A dynamic blind area monitoring system and control method based on multi-path camera of tractor and semitrailer

Abstract

The application provides a dynamic blind area monitoring system and control method based on multi-path camera of tractor and semitrailer, and relates to the technical field of vehicle safe driving assistance. The system comprises a sensing module at the end of the tractor, a left lateral wide-angle camera and a right lateral wide-angle camera arranged on both sides of the tractor, a multi-path video split-screen display terminal and a central control module, and lateral wide-angle cameras arranged on both sides of the semitrailer. In view of the problem that the fixed-view-angle camera cannot dynamically adjust the blind area following the posture change of the semitrailer, the central control module divides different driving levels according to the vehicle speed, matches corresponding steering trigger standards and dangerous area sizes for each level, and automatically switches to the camera on the inside of the turning in combination with the gear position, steering wheel angle and two-vehicle hinge angle signals, and separately displays the picture of the dangerous blind area after intercepting the picture. The application does not need complex mechanical structure, can highlight the key dangerous area, and effectively reduces the scratching and collision risk of the semitrailer when turning and reversing.

Description

Technical Field

[0001] This invention relates to the field of vehicle safety driving assistance technology, and in particular to a dynamic blind spot monitoring system and control method based on multi-channel cameras of a tractor and a semi-trailer. Background Technology

[0002] With the development of road freight transport, semi-trailers, consisting of a tractor unit pulling a single semi-trailer, are widely used for cargo transportation. These vehicles are relatively long, and there is a change in the connection angle between the tractor unit and the semi-trailer, which can make it difficult for the driver to observe when turning or reversing.

[0003] When a semi-trailer is turning forward, it follows the tractor unit's change of direction, creating an area on the inside of the trailer that is not easily visible to the driver. If there are pedestrians, non-motorized vehicles, curbs, or other obstacles in this area, the vehicles may scrape or collide. When the vehicle is reversing, the angle between the tractor unit and the semi-trailer changes, making it difficult for the driver to see the dangerous areas on the inside and rear sides of the semi-trailer in time using ordinary rearview mirrors or fixed-angle cameras. This is especially true when the articulation angle is greater than 30°, at which point the fixed-angle camera will completely lose coverage of the area inside the semi-trailer's cab.

[0004] Existing vehicle camera systems typically simply display the camera feed directly to the driver with a fixed viewing angle. They cannot automatically highlight dangerous areas based on the vehicle's turning or reversing position, requiring the driver to manually locate hazards within the full view, leading to long reaction times and potential distraction. Some solutions employ rotatable cameras or complex mechanical structures to adjust the viewing angle; however, these structures are susceptible to damage from rain, mud, and vibration during long-term use in freight vehicles, resulting in high failure rates, short lifespans, and high maintenance costs.

[0005] Therefore, there is a need for an auxiliary observation system that is simple in structure, easy to install, and capable of automatically displaying the blind spots of the semi-trailer based on the angle changes between the tractor and the semi-trailer. Summary of the Invention

[0006] This invention provides a dynamic blind spot monitoring system and control method based on multi-channel cameras of the tractor and semi-trailer, which helps the driver to more clearly observe the dangerous areas around the semi-trailer when turning forward and reversing.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a dynamic blind spot monitoring system based on multiple cameras of a tractor and a semi-trailer, including a tractor end assembly disposed at the tractor end and a left semi-trailer side wide-angle camera and a right semi-trailer side wide-angle camera respectively fixed to the left and right sides of the semi-trailer. The tractor-mounted assembly includes a steering wheel angle sensing module, a vehicle speed sensing module, a gear position sensing module, an articulation angle sensing module, a left-side wide-angle camera of the tractor, a right-side wide-angle camera of the tractor, a multi-channel video split-screen display terminal, and a central control module. The articulation angle sensing module is arranged at the connection between the tractor and the semi-trailer to collect the articulation angle signal between the tractor and the semi-trailer in real time. The left-side wide-angle camera and the right-side wide-angle camera of the tractor are respectively located on both sides of the tractor and are used to collect global field-of-view images of the side and rear of the tractor. The multi-channel video split-screen display terminal is installed in the driver's cab of the tractor, and the multi-channel video split-screen display terminal is equipped with a main viewing window of the tractor and a blind spot viewing window. The central control module is connected to the steering wheel angle sensing module, vehicle speed sensing module, gear sensing module, articulation angle sensing module, left-side wide-angle camera of the tractor, right-side wide-angle camera of the tractor, multi-channel video split-screen display terminal, left-side semi-trailer wide-angle camera and right-side semi-trailer wide-angle camera respectively. The central control module is used to classify vehicle speed levels based on vehicle speed signals and adjust steering judgment conditions and blind spot range expansion coefficients according to vehicle speed levels. The central control module is also used to determine the location of the high-risk inner blind spot of the semi-trailer when it is turning forward or reversing based on gear signals, steering wheel angle signals, articulation angle signals and vehicle speed signals, determine the steering direction based on the positive or negative of the steering wheel angle, control the switching to the corresponding semi-trailer side wide-angle camera on the inner side of the steering, and control the cropping of the video image of the corresponding semi-trailer side wide-angle camera and the display of the dangerous blind spot window.

[0008] As a preferred technical solution, the central control module incorporates a semi-trailer trajectory geometry calculation unit. This unit uses the articulation angle signal as one of its input parameters, combined with the steering wheel angle signal, gear signal, and vehicle speed signal, to calculate the real-time driving trajectory of the semi-trailer through vehicle kinematic geometry, thereby determining the inner basic blind zone boundary when the semi-trailer is turning forward or reversing. The geometry calculation unit presets a blind zone range expansion coefficient corresponding to the vehicle speed level. This coefficient is a proportional coefficient that expands the basic blind zone boundary. The geometry calculation unit multiplies the basic blind zone boundary by the blind zone range expansion coefficient corresponding to the vehicle speed level to obtain the final high-risk blind zone boundary, with higher speeds corresponding to larger blind zone range expansion coefficients. The central control module also presets steering judgment conditions corresponding to the vehicle speed level. These conditions include a steering wheel angle threshold and a steering angle duration threshold, with higher speeds corresponding to smaller steering wheel angle thresholds and shorter steering angle duration thresholds.

[0009] As a preferred technical solution, the image output terminals of the left-side wide-angle camera and the right-side wide-angle camera of the tractor are connected to a multi-channel video split-screen display terminal to continuously transmit the collected images to the main window of the tractor for display.

[0010] As a preferred technical solution, the central control module is also used to perform fault self-checking and degradation processing. The fault self-checking and degradation processing includes: when any sensor module or camera signal is detected to be abnormal, controlling the blind spot window to display fault prompt information; when the central control module fails to perform blind spot position calculation or output screen switching and cropping control signals, the multi-channel video split-screen display terminal automatically displays the complete images of the left-side wide-angle camera and the right-side wide-angle camera of the tractor.

[0011] As a preferred technical solution, both the left and right semi-trailer side-facing wide-angle cameras are installed on the semi-trailer body at positions that can completely cover the entire inner length of their respective sides. The lenses face the rear of the semi-trailer and tilt inward, with a horizontal field of view covering the adjacent lanes and a preset area behind the semi-trailer. In forward turning mode, the central control module determines the vehicle's turning direction based on the steering wheel angle signal. When the vehicle turns left, it switches to the left semi-trailer side-facing wide-angle camera for image cropping and display, and when the vehicle turns right, it switches to the right semi-trailer side-facing wide-angle camera for image cropping and display. In reverse mode, the central control module determines the location of the high-risk blind spot on the inner side of the semi-trailer during reversing based on the gear signal, steering wheel angle signal, articulation angle signal, and vehicle speed signal, and switches to the semi-trailer side-facing wide-angle camera corresponding to the location of the high-risk blind spot for image cropping and display.

[0012] Secondly, the present invention provides a dynamic blind spot monitoring and control method based on multi-channel cameras of a tractor and a semi-trailer, applied to the system described above, the method comprising: It reads steering wheel angle signal, gear signal, hinge angle signal, vehicle speed signal, and video streams from each camera in real time; Vehicle speed levels are classified based on vehicle speed signals, and corresponding steering judgment conditions and blind spot range expansion coefficients are matched for each vehicle speed level. When the current gear is forward or reverse, determine whether the absolute value and duration of the steering wheel angle meet the steering judgment conditions for the corresponding vehicle speed level. If the turning judgment condition is met, the position of the high-risk blind spot inside the semi-trailer when turning forward or reversing is determined by the articulation angle signal, steering wheel angle signal and vehicle speed signal, combined with the blind spot range expansion coefficient of the corresponding vehicle speed level. The turning direction is determined by the positive or negative of the steering wheel angle, and the camera is switched to the side wide-angle camera of the semi-trailer corresponding to the turning inside. The high-risk blind spot image is cropped and displayed in the dangerous blind spot window (8). If the steering judgment condition is not met or the gear is neutral / parking, the blind spot window (8) will be put into standby mode.

[0013] As a preferred technical solution, vehicle speed levels are divided based on vehicle speed signals, and corresponding steering judgment conditions and blind spot range expansion coefficients are matched for each vehicle speed level, including: Based on the vehicle speed signal, the vehicle driving status is divided into three speed levels: low speed, medium speed and high speed, and corresponding steering judgment conditions and blind spot range expansion coefficients are matched for each speed level. The low-speed level corresponds to the first steering wheel angle threshold, the first angle duration threshold, and the first blind spot range expansion coefficient; the medium-speed level corresponds to the second steering wheel angle threshold, the second angle duration threshold, and the second blind spot range expansion coefficient; and the high-speed level corresponds to the third steering wheel angle threshold, the third angle duration threshold, and the third blind spot range expansion coefficient. The first steering wheel angle threshold is greater than the second steering wheel angle threshold, and the second steering wheel angle threshold is greater than the third steering wheel angle threshold; The first corner duration threshold is longer than the second corner duration threshold, and the second corner duration threshold is longer than the third corner duration threshold; The first blind zone range expansion coefficient is less than the second blind zone range expansion coefficient, and the second blind zone range expansion coefficient is less than the third blind zone range expansion coefficient.

[0014] As a preferred technical solution, methods for determining the location of the high-risk inner blind spot of a semi-trailer when it is turning forward or reversing include: The real-time driving trajectory of the semi-trailer is calculated by the vehicle's kinematic geometry to obtain the inner basic blind zone boundary. The basic blind zone boundary is then multiplied by the blind zone range expansion coefficient corresponding to the current vehicle speed level to obtain the final high-risk blind zone boundary. The higher the vehicle speed level, the larger the corresponding blind zone range expansion coefficient.

[0015] As a preferred technical solution, the method of cropping out the high-risk blind area image and displaying it in the dangerous blind area window (8) includes: selecting the smallest rectangular area that can completely contain the boundary of the high-risk blind area as the cropping area, and adapting it to the display ratio of the dangerous blind area window (8) for full-screen display.

[0016] As a preferred technical solution, the method further includes fault self-checking and degradation processing, wherein the fault self-checking and degradation processing includes: The system monitors the signal status of each sensor module and camera in real time. When an abnormal signal is detected, a fault message is displayed in the blind spot window (8), while the main window (7) of the tractor remains in normal display. When the central control module fails to perform blind spot calculation and display control, the multi-channel video split-screen display terminal (6) automatically displays the complete images of the lateral wide-angle cameras on both sides of the tractor.

[0017] The dynamic blind spot monitoring system and control method based on multi-channel cameras of tractor and semi-trailer provided by the present invention has at least the following beneficial effects: 1. The system of this invention uses fixed-installation sensor modules and side-facing wide-angle cameras, with no rotating mechanical parts. It can adapt to harsh operating environments such as rain, mud, and vibration in freight vehicles, resulting in a low failure rate and low maintenance costs. Furthermore, the system can directly utilize the vehicle's existing steering wheel angle, speed, and gear signals, in conjunction with the articulation angle sensor module located at the connection between the tractor and semi-trailer, to complete driving status data collection. This eliminates the need for large-scale modifications to the vehicle's original system, ensuring good compatibility and facilitating mass installation and widespread adoption.

[0018] 2. The system of this invention classifies vehicle speed levels based on vehicle speed signals through a central control module. For each speed level, it matches corresponding steering judgment conditions and blind spot range expansion coefficients. Combining gear position signals, steering wheel angle signals, articulation angle signals, and vehicle speed signals, it determines the location of the high-risk blind spot on the inside of the semi-trailer when it is turning forward or reversing. It automatically switches to the corresponding wide-angle side camera on the inside of the turning direction, crops out the high-risk blind spot image, and displays it in the blind spot view window. This system can automatically highlight the high-risk blind spot area on the inside of the current turning direction, avoiding driver distraction, significantly shortening hazard recognition time, effectively improving observation safety during forward turning and reversing, and reducing the risk of scrapes and collisions when the semi-trailer is turning or reversing. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] Figure 1 A schematic diagram of the hardware topology of a dynamic blind spot monitoring system based on multiple cameras of a tractor and a semi-trailer provided in an embodiment of the present invention; Figure 2 A flowchart of a dynamic blind spot elimination control logic based on separate judgments of forward and reverse gears is provided for an embodiment of the present invention. Figure 3A schematic diagram illustrating the arrangement and field of view coverage of onboard cameras in a tractor and a single semi-trailer, provided as an embodiment of the present invention; Figure 4 The flowchart illustrates a dynamic blind spot monitoring and control method based on multi-channel cameras of a tractor and a semi-trailer, as provided in an embodiment of the present invention.

[0021] Figure label: 1. Steering wheel angle sensor module; 2. Vehicle speed sensor module; 3. Gear position sensor module; 4. Articulation angle sensor module; 5a. Left-side wide-angle camera of the tractor; 5b. Right-side wide-angle camera of the tractor; 6. Multi-channel video split-screen display terminal; 7. Main window of the tractor; 8. Hazard blind spot window; 9. Central control module; 10a. Left-side wide-angle camera of the semi-trailer; 10b. Right-side wide-angle camera of the semi-trailer.

[0022] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0023] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0024] Example 1: This invention provides a dynamic blind spot monitoring system based on multiple cameras of a tractor and a semi-trailer. This system is applicable to vehicle scenarios where a tractor pulls a semi-trailer and is used to assist the driver in observing dangerous blind spots around the semi-trailer when the vehicle is moving forward, turning, or reversing.

[0025] like Figure 1As shown, the dynamic blind spot monitoring system based on multi-channel cameras on the tractor and semi-trailer includes a tractor-side assembly and a left-side semi-trailer wide-angle camera 10a and a right-side semi-trailer wide-angle camera 10b, respectively fixed to the left and right sides of the semi-trailer. The tractor-side assembly includes a steering wheel angle sensing module 1, a vehicle speed sensing module 2, a gear position sensing module 3, an articulation angle sensing module 4, a left-side and right-side wide-angle camera 5a and 5b respectively located on both sides of the tractor, a multi-channel video split-screen display terminal 6, and a central control module 9. The steering wheel angle sensing module 1 is used to collect steering wheel angle signals. The vehicle speed sensing module 2 is used to collect vehicle speed signals and transmit them to the central control module 9, allowing the central control module 9 to classify vehicle speed levels, adjust steering judgment conditions, and adjust the blind spot range expansion coefficient. The gear position sensing module 3 is used to collect vehicle gear position signals. The articulation angle sensing module 4 is installed at the connection between the tractor and the semi-trailer to collect the articulation angle signal between them. The left-side wide-angle camera 5a and the right-side wide-angle camera 5b of the tractor are installed on both sides of the tractor to collect the overall view of the tractor's rear side. The left-side wide-angle camera 10a and the right-side wide-angle camera 10b of the semi-trailer, respectively fixed to the left and right sides, are used to collect images of the rear side of the corresponding side of the semi-trailer and the area near the inside of the turn. The semi-trailer's side wide-angle cameras are installed at positions on the semi-trailer body that can completely cover the entire inner length of their side, with the lenses facing the rear side of the semi-trailer and tilted inward at a certain angle. Their horizontal field of view meets the requirement of simultaneously covering the adjacent lane inside the semi-trailer and a certain range behind it. In forward turning mode, the central control module 9 determines the vehicle's turning direction based on the steering wheel angle signal. When the vehicle turns left, it switches to the left semi-trailer side wide-angle camera 10a for image cropping and display, and when the vehicle turns right, it switches to the right semi-trailer side wide-angle camera 10b for image cropping and display. In reverse mode, the central control module 9 determines the location of the inner high-risk blind spot during the semi-trailer's reversing process based on the gear position signal, steering wheel angle signal, articulation angle signal, and vehicle speed signal, and switches to the semi-trailer side wide-angle camera corresponding to the inner high-risk blind spot location for image cropping and display. The central control module 9 receives steering wheel angle signal, vehicle speed signal, gear position signal, articulation angle signal, and camera video stream, and controls the display of the blind spot image according to the vehicle's current state. The multi-channel video split-screen display terminal 6 is located in the tractor cab, including the tractor main viewing window 7 and the blind spot viewing window 8. The tractor's main view 7 is used to display the images captured by the left-side wide-angle camera 5a and the right-side wide-angle camera 5b of the tractor, which are set on both sides of the tractor. The blind spot view 8 is used to display the high-risk blind spot images cropped from the left-side semi-trailer wide-angle camera 10a and the right-side semi-trailer wide-angle camera 10b.

[0026] When the vehicle is running, the system first collects steering wheel angle signals, gear position signals, articulation angle signals, vehicle speed signals, and camera video streams. The system then determines whether the current gear is a drive gear. If the current gear is a drive gear, the system continues to determine whether the absolute value and duration of the steering wheel angle meet the corresponding steering judgment conditions based on the current vehicle speed level. The steering judgment conditions are dynamically linked to the vehicle speed level and are composed of two parameters: a steering wheel angle threshold and a steering angle duration threshold. At low speeds, a larger steering wheel angle threshold and a longer steering angle duration threshold are used. Blind spot display is only triggered when the absolute value of the steering wheel angle reaches the threshold and the duration meets the requirements, avoiding accidental triggering due to minor steering wheel adjustments by the driver. At medium speeds, a smaller steering wheel angle threshold and a shorter steering angle duration threshold are used to accommodate the faster steering response at medium speeds. At high speeds, the smallest steering wheel angle threshold and the shortest steering angle duration threshold are used to ensure timely triggering of blind spot display in scenarios such as high-speed lane changes and emergency avoidance, providing early warning of potential hazards.

[0027] It is important to note that the specific parameters of the aforementioned steering judgment conditions are not fixed values. Instead, they are determined using the same calibration principles as the blind spot range expansion coefficient, based on specific parameters such as the semi-trailer's body length, wheelbase, track width, and steering system transmission ratio, as well as typical vehicle usage scenarios, through real-vehicle calibration before the vehicle leaves the factory. For different semi-trailer models, the system's trigger timing can be optimized by adjusting the steering judgment condition parameters, avoiding false triggers that might interfere with the driver while ensuring timely response in dangerous scenarios. When the absolute value of the steering wheel angle does not reach the steering wheel angle threshold for the corresponding vehicle speed level, or when it reaches the angle threshold but the duration does not reach the corresponding level's angle duration threshold, it is determined that the steering judgment condition has not been met, the system does not trigger the high-risk blind spot display, and the blind spot window remains in standby mode.

[0028] like Figure 2 The diagram shown is a flowchart of a dynamic blind spot elimination control logic based on the judgment of forward gear and reverse gear according to an embodiment of the present invention. The system is controlled by a central control module 9, which is configured to execute the following steps S101-S112.

[0029] S101: Collect vehicle driving data in real time, read steering wheel angle, gear position signal, hinge angle signal and camera video stream, and execute S102.

[0030] S102: Determine if the current gear is a forward gear. If yes, execute S103; otherwise, execute S107.

[0031] S103: Determine whether the absolute value and duration of the steering wheel angle meet the steering judgment conditions for the corresponding vehicle speed level. If yes, execute S104; otherwise, execute S108.

[0032] S104: Determine the position of the inner blind spot of the semi-trailer when it is turning forward based on the articulation angle signal and the steering wheel angle signal. After completion, execute S105.

[0033] S105: Based on the determined blind spot location, crop the high-risk blind spot area in the side wide-angle camera image of the semi-trailer, and then execute S106.

[0034] S106: Display the cropped high-risk blind spot image in the danger blind spot window, and the process ends.

[0035] S107: Determine if the current gear is reverse. If yes, execute S109; otherwise, execute S108.

[0036] S108: Keep the tractor's main view window displaying normally, keep the blind spot view window on standby, process complete.

[0037] S109: Determine whether the absolute value and duration of the steering wheel angle meet the steering judgment conditions for the corresponding vehicle speed level. If yes, execute S110; otherwise, execute S108.

[0038] S110: Determine the position of the inner blind spot of the semi-trailer when reversing based on the articulation angle signal and the steering wheel angle signal, and then execute S111.

[0039] S111: Based on the determined blind spot location, crop the high-risk blind spot area for reversing from the side wide-angle camera image of the semi-trailer, and then execute S112.

[0040] S112: Display the cropped image of the high-risk blind spot when reversing in the blind spot view window, and the process ends.

[0041] It should be noted that the process ending mentioned above refers to the end of the processing of the vehicle driving data collected at the current moment. After the process ends, the system will return to step S101 by default and execute the processing of the vehicle driving data collected at the next moment. When the system detects that the vehicle is turned off or the power is cut off, the driver actively turns off the dynamic blind spot elimination function, or the system triggers the serious fault degradation mechanism, the above loop process will be exited.

[0042] In some embodiments, if the steering wheel angle reaches the corresponding level of steering judgment conditions, the system determines the inner blind spot position of the semi-trailer when it is turning forward, based on the articulation angle signal and the steering wheel angle signal. During the calculation process, the central control module 9 divides the vehicle's driving state into three levels: low speed, medium speed, and high speed, based on the current vehicle speed. The classification of vehicle speed levels is based on the typical driving conditions of the vehicle. Low speed corresponds to low-speed driving scenarios such as urban roads and factory areas, medium speed corresponds to driving scenarios on ordinary roads, and high speed corresponds to driving scenarios on highways. Different blind spot range expansion coefficients are used accordingly to ensure that a larger potential danger area can be covered when driving at high speeds.

[0043] Specifically, at low speeds, a basic calculation coefficient is used, at which point the high-risk blind spot range is basically the same as the basic blind spot range, covering only the area that the semi-trailer will actually sweep when turning. At medium speeds, a medium-speed coefficient greater than the basic calculation coefficient is used, extending the high-risk blind spot range inward and forward to a certain extent from the basic blind spot range. At high speeds, a high-speed coefficient greater than the medium-speed coefficient is used, further extending the high-risk blind spot range to cover areas that the vehicle may enter due to inertia. The blind spot range extension coefficients corresponding to each speed level are not fixed values, but are determined through actual vehicle calibration before the vehicle leaves the factory, based on specific parameters of the semi-trailer, including body length, wheelbase, track width, and maximum gross weight, as well as the typical usage scenarios of the vehicle. For different models of semi-trailers, the blind spot display effect of the system can be optimized by adjusting the calculation coefficients.

[0044] In some embodiments, the system crops out the corresponding high-risk blind spot areas from the images captured by the left semi-trailer side-facing wide-angle camera 10a and the right semi-trailer side-facing wide-angle camera 10b. The cropped area is the smallest rectangular area that can completely contain the calculated high-risk blind spot boundary, and automatically adapts to the display ratio of the blind spot window for full-screen display, ensuring that the driver can clearly observe all obstacles inside the blind spot, and displays the cropped image in the blind spot window 8. If the current gear is not forward, the system continues to determine whether the current gear is reverse. If the current gear is reverse, the system enters reverse blind spot monitoring. The system also determines whether the absolute value and duration of the steering wheel angle meet the corresponding steering judgment conditions based on the current vehicle speed level. If they do, it determines the location of the high-risk inner blind spot of the semi-trailer when reversing based on the articulation angle signal, steering wheel angle signal, and vehicle speed signal, combined with the blind spot range expansion coefficient of the corresponding vehicle speed level. The calculation logic is the same as when turning forward, that is, first calculate the inner basic blind spot boundary when reversing, and then multiply it by the blind spot range expansion coefficient of the corresponding vehicle speed level to obtain the final high-risk blind spot boundary when reversing.

[0045] In some embodiments, the system crops out the high-risk blind spot area for reversing from the images captured by the wide-angle camera 10a on the left side of the semi-trailer and the wide-angle camera 10b on the right side of the semi-trailer, and displays the cropped image in the blind spot window 8. If the current gear is neither forward nor reverse, or if the gear is reverse but the steering judgment condition is not met, the system does not enter forward steering blind spot monitoring or reversing blind spot monitoring. At this time, the tractor's main view window 7 remains normally displayed, and the blind spot window 8 is in standby mode.

[0046] In some embodiments, such as Figure 3 As shown, the left-side wide-angle camera 5a and the right-side wide-angle camera 5b of the tractor are positioned on the sides of the tractor to capture images from the rear side of the tractor. The left-side semi-trailer wide-angle camera 10a and the right-side semi-trailer wide-angle camera 10b are positioned on the sides of the semi-trailer to capture images from the rear side of the semi-trailer and the area near the inner side of the semi-trailer. When the vehicle turns or reverses, the angle between the tractor and the semi-trailer changes, creating a blind spot on the inner side of the semi-trailer that is difficult for the driver to directly observe. The system determines the location of this blind spot using articulation angle signals, steering wheel angle signals, and vehicle speed signals, and crops this area from the images of the left-side semi-trailer wide-angle camera 10a and the right-side semi-trailer wide-angle camera 10b for display.

[0047] This embodiment uses a fixed camera and existing vehicle driving status information to display the blind spots of a semi-trailer when it is moving forward, turning, or reversing. The solution is simple in structure, easy to install, and helps drivers see the dangerous areas inside the semi-trailer more quickly, thereby reducing the risk of vehicle scrapes and collisions.

[0048] In some embodiments, the system has fault self-diagnosis and degradation processing functions. When the system detects an abnormal signal from any sensor module or camera, the blind spot window will display a clear fault warning message while maintaining the normal display of the tractor's main window. When the central control module 9 experiences a serious fault, the multi-channel video split-screen display terminal will automatically switch to directly display the complete images from the left-side wide-angle camera and the right-side wide-angle camera of the tractor, ensuring that basic driving assistance functions are not affected.

[0049] It should be noted that the serious faults mentioned in this embodiment include, but are not limited to, the failure of the central control module (9) to perform blind spot position calculation and to output screen switching and cropping control signals. Specifically, when the central control module (9) fails to complete the blind spot position calculation within a preset time, or fails to output valid high-risk blind spot boundary data for multiple consecutive control cycles, or when the calculation results show missing coordinates, out-of-bounds, abnormal area, or significant inconsistency with the trend of steering wheel angle and hinge angle changes, it is determined that it cannot perform blind spot position calculation; when the central control module (9) issues a camera switching or screen cropping control command, and fails to receive an execution confirmation signal from the multi-channel video split-screen display terminal (6) within a preset feedback time, or when the feedback display channel, cropping area, video frame timestamp is inconsistent with the output command, it is determined that it cannot output valid screen switching and cropping control signals. If the above abnormalities meet the preset serious fault judgment conditions, a downgrade process is triggered, so that the multi-channel video split-screen display terminal (6) automatically displays the complete images of the lateral wide-angle cameras on both sides of the tractor.

[0050] Example 2: This invention provides a dynamic blind spot monitoring and control method based on multi-channel cameras of a tractor and semi-trailer, applicable to systems as described in any of the embodiments in Example 1, such as... Figure 4 As shown, the method includes the following steps S10-S50.

[0051] S10: Real-time reading of steering wheel angle signal, gear signal, hinge angle signal, vehicle speed signal, and video streams from each camera.

[0052] In some embodiments, the steering wheel angle signal is acquired in real time by the steering wheel angle sensing module 1 at a sampling frequency of not less than 50Hz and sent to the central control module 9 in the form of CAN bus messages; the vehicle speed signal is obtained from the vehicle ABS wheel speed sensor or the transmission output shaft speed sensor by the vehicle speed sensing module 2; the gear position signal is obtained from the vehicle automatic transmission control unit or gear position switch by the gear position sensing module 3; the articulation angle signal is acquired in real time by the articulation angle sensing module 4 at a sampling frequency of not less than 20Hz, and the articulation angle sensing module 4 adopts an angular displacement sensor or a Hall effect angle sensor installed at the connection between the tractor and the semi-trailer saddle; the video streams of each camera are transmitted to the central control module 9 in real time through coaxial cables or LVDS differential signal lines at a frame rate of not less than 30 frames / second.

[0053] S20: Classify vehicle speed levels based on vehicle speed signals, and match corresponding steering judgment conditions and blind spot range expansion coefficients for each vehicle speed level.

[0054] In some embodiments, the vehicle driving state is divided into three speed levels: low speed, medium speed, and high speed, based on the vehicle speed signal, and corresponding steering judgment conditions and blind spot range expansion coefficients are matched for each speed level.

[0055] For example, the low speed level corresponds to a vehicle speed range of 0 km / h. v ≤15 km / h, the medium speed level corresponds to a speed range of 15 km / h< v ≤50 km / h, the speed range corresponding to the highway level is v >50 km / h; The above speed class threshold can be adjusted by actual vehicle calibration before the vehicle leaves the factory according to parameters such as the body length, wheelbase, and maximum gross weight of the semi-trailer. Different speed class thresholds can be set for different models of semi-trailers. This is just an example and does not constitute a limitation of the present invention.

[0056] The low-speed level corresponds to the first steering wheel angle threshold, the first angle duration threshold, and the first blind spot range expansion coefficient; the medium-speed level corresponds to the second steering wheel angle threshold, the second angle duration threshold, and the second blind spot range expansion coefficient; and the high-speed level corresponds to the third steering wheel angle threshold, the third angle duration threshold, and the third blind spot range expansion coefficient. The first steering wheel angle threshold is greater than the second steering wheel angle threshold, and the second steering wheel angle threshold is greater than the third steering wheel angle threshold; The first turning time duration threshold is longer than the second turning time duration threshold, and the second turning time duration threshold is longer than the third turning time duration threshold; The expansion coefficient of the first blind zone is less than that of the second blind zone, and the expansion coefficient of the second blind zone is less than that of the third blind zone.

[0057] For example, taking a certain model of semi-trailer (body length 13 meters, wheelbase 8 meters) as an example, the first steering wheel angle threshold for the low-speed level is 150°, the first steering angle duration threshold is 1.5 s, and the first blind spot range expansion coefficient is 1.0; the second steering wheel angle threshold for the medium-speed level is 90°, the second steering angle duration threshold is 0.8 s, and the second blind spot range expansion coefficient is 1.4; the third steering wheel angle threshold for the high-speed level is 45°, the third steering angle duration threshold is 0.4 s, and the third blind spot range expansion coefficient is 2.0. The specific values ​​of the above thresholds and coefficients are determined by actual vehicle calibration before the vehicle leaves the factory based on parameters such as the semi-trailer's body length, wheelbase, track width, steering system transmission ratio, and maximum gross vehicle weight, and do not constitute a limitation of the present invention.

[0058] S30: When the current gear is forward or reverse, determine whether the absolute value and duration of the steering wheel angle meet the steering judgment conditions for the corresponding vehicle speed level.

[0059] Specifically, the central control module 9 first determines whether the current gear is a drive or reverse gear based on the gear position signal. If the current gear is a drive or reverse gear, it further determines whether the absolute value of the steering wheel angle is greater than or equal to the steering wheel angle threshold corresponding to the current vehicle speed level. If so, it continues to determine whether the time during which the steering wheel angle continuously reaches or exceeds the angle threshold is greater than or equal to the angle duration threshold corresponding to the current vehicle speed level. Only when both of the above conditions are met simultaneously is it determined that the steering judgment condition has been met. If the current gear is neither a drive nor a reverse gear, it is directly determined that the steering judgment condition has not been met, and subsequent blind spot monitoring is not triggered.

[0060] S40: If the steering judgment condition is met, the position of the high-risk blind spot inside the semi-trailer when it is turning forward or reversing is determined based on the articulation angle signal, steering wheel angle signal and vehicle speed signal, combined with the blind spot range expansion coefficient of the corresponding vehicle speed level. The steering direction is determined according to the positive or negative of the steering wheel angle, and the camera is switched to the side wide-angle camera of the semi-trailer corresponding to the inside of the steering direction. The high-risk blind spot image is cropped and displayed in the danger blind spot window.

[0061] In some embodiments, the method for determining the location of the inner high-risk blind spot when the semi-trailer is turning forward or reversing includes: calculating the real-time driving trajectory of the semi-trailer through the vehicle's kinematic geometry to obtain the inner basic blind spot boundary, and multiplying the basic blind spot boundary by the blind spot range expansion coefficient corresponding to the current vehicle speed level to obtain the final high-risk blind spot boundary; wherein the higher the vehicle speed level, the larger the corresponding blind spot range expansion coefficient.

[0062] Specifically, the semi-trailer trajectory geometry calculation unit built into the central control module 9 uses the current articulation angle... γ Steering wheel angle θ and vehicle speed v The calculations are performed based on the kinematic model of the semi-trailer, using the input parameters. The front wheel steering angle of the tractor unit is considered. α Its relationship with the steering wheel angle θ The relationship between them is α = θ / i ,in i The vehicle steering system transmission ratio; the semi-trailer trajectory geometry calculation unit calculates the vehicle's trajectory based on the tractor's front wheel turning angle α, the tractor's wheelbase, the semi-trailer's wheelbase, and the real-time articulation angle between the two vehicles. γ The geometric constraints are used to calculate the real-time motion trajectory and articulation angle change rate of the semi-trailer through the vehicle kinematic equations, thereby determining the motion trajectory of the inner end point of the semi-trailer cab and the inner contour line of the vehicle body; the boundary of the inner basic blind zone is determined by the area between the innermost envelope formed by the inner contour line of the semi-trailer body sweeping on its motion trajectory and the current inner edge of the semi-trailer body.

[0063] After determining the basic blind spot boundary, the geometric calculation unit extends the basic blind spot boundary outward in a direction perpendicular to the inner side of the semi-trailer body. The maximum distance between the extended high-risk blind spot boundary and the inner edge of the semi-trailer body is multiplied by the maximum distance between the basic blind spot boundary and the inner edge of the semi-trailer body, and the blind spot range expansion coefficient corresponding to the current vehicle speed level, to obtain the final high-risk blind spot boundary. For example, if the current vehicle speed level is low speed level (0 km / h < v If the speed is ≤15 km / h and the blind spot range expansion coefficient is 1.0, then the boundary of the high-risk blind spot coincides with the boundary of the basic blind spot; if the current speed level is medium speed level (15 km / h < v ≤50 km / h), the blind spot range expansion coefficient is 1.4, then the high-risk blind spot boundary expands outward by 40% from the basic blind spot boundary; if the current vehicle speed level is high speed level ( v If the speed is >50 km / h and the blind zone expansion coefficient is 2.0, then the high-risk blind zone boundary extends outward by 100% from the basic blind zone boundary.

[0064] The method for determining the steering direction is: when the steering wheel angle signal... θ When the value is >0, it is determined to be a right turn, and the camera switches to the right-side wide-angle camera 10b on the semi-trailer side; when the steering wheel angle signal... θ When the value is less than 0, it is determined to be a left turn, and the camera switches to the left-side wide-angle camera 10a on the side of the semi-trailer.

[0065] In some embodiments, the method of cropping out the high-risk blind spot image and displaying it in the dangerous blind spot window 8 includes: selecting the smallest rectangular area that can completely contain the boundary of the high-risk blind spot as the cropping area, and displaying it in full screen according to the display ratio of the dangerous blind spot window.

[0066] Specifically, the central control module 9 determines the pixel coordinate range of the high-risk blind zone boundary in the image based on the calculated mapping position of the high-risk blind zone boundary in the image coordinate system of the selected semi-trailer side wide-angle camera; the central control module 9 uses the smallest rectangular area that can completely contain the high-risk blind zone boundary as the cropping window, and extracts the pixel array corresponding to the cropping window from the original video frame; the central control module 9 scales the obtained pixel array to adapt to the display resolution of the dangerous blind zone window 8, and displays the scaled image in the dangerous blind zone window 8 in full-screen mode.

[0067] For example, when the vehicle turns right and the current speed is at a medium speed level, the central control module 9 maps the boundary of the high-risk blind spot onto the image of the right-side wide-angle camera 10b of the semi-trailer, obtains the pixel coordinates of the high-risk blind spot boundary in the image, uses the smallest rectangular area that can completely contain the boundary of the high-risk blind spot as the cropping window, crops the rectangular image corresponding to the cropping window from the original video frame, scales the cropped rectangular image to adapt to the display ratio of the blind spot window 8, and displays it in full screen in the blind spot window 8; the driver does not need to search in the complete wide-angle image, but can directly observe the obstacle situation in the high-risk blind spot in the blind spot window 8.

[0068] S50: If the steering judgment condition is not met or the gear is neutral / park, the blind spot window will be put into standby mode.

[0069] Specifically, when the blind spot window 8 is in standby mode, the window remains black or displays a static interface consistent with the background of the tractor's main window 7, without displaying any video footage from the cameras, in order to avoid unnecessary screen switching that might distract the driver. In this state, the tractor's main window 7 continuously displays the global field of view images of the tractor's side and rear side captured by the left-side wide-angle camera 5a and the right-side wide-angle camera 5b, unaffected by changes in the status of the blind spot window 8.

[0070] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic blind spot monitoring system based on multi-channel cameras of a tractor and a semi-trailer, characterized in that, It includes a tractor end assembly installed at the tractor end and a left semi-trailer side wide-angle camera (10a) and a right semi-trailer side wide-angle camera (10b) respectively fixed to the left and right sides of the semi-trailer. The tractor end assembly includes a steering wheel angle sensing module (1), a vehicle speed sensing module (2), a gear sensing module (3), an articulation angle sensing module (4), a left-side wide-angle camera (5a) of the tractor, a right-side wide-angle camera (5b) of the tractor, a multi-channel video split-screen display terminal (6), and a central control module (9). The articulation angle sensing module (4) is arranged at the connection between the tractor and the semi-trailer to collect the articulation angle signal between the tractor and the semi-trailer in real time. The left-side wide-angle camera (5a) and the right-side wide-angle camera (5b) of the tractor are respectively located on both sides of the tractor and are used to collect global field-of-view images of the tractor's side and rear. The multi-channel video split-screen display terminal (6) is installed in the driver's cab of the tractor. The multi-channel video split-screen display terminal (6) is equipped with a tractor main viewing window (7) and a blind spot viewing window (8). The central control module (9) is connected to the steering wheel angle sensing module (1), vehicle speed sensing module (2), gear sensing module (3), articulation angle sensing module (4), left-side wide-angle camera (5a) of the tractor, right-side wide-angle camera (5b) of the tractor, multi-channel video split-screen display terminal (6), left-side semi-trailer wide-angle camera (10a) and right-side semi-trailer wide-angle camera (10b) respectively; The central control module (9) is used to classify vehicle speed levels according to vehicle speed signals and adjust steering judgment conditions and blind spot range expansion coefficients according to vehicle speed levels. The central control module (9) is also used to determine the position of the high-risk blind spot inside the semi-trailer when it is turning forward or reversing according to gear signals, steering wheel angle signals, articulation angle signals and vehicle speed signals, determine the steering direction according to the positive and negative of the steering wheel angle, control the switching to the corresponding semi-trailer side wide-angle camera on the inside of the steering, and control the cropping of the video image of the corresponding semi-trailer side wide-angle camera and the display of the dangerous blind spot window (8).

2. The dynamic blind spot monitoring system based on multi-channel cameras of the tractor and semi-trailer as described in claim 1, characterized in that, The central control module (9) has a built-in semi-trailer trajectory geometry calculation unit. The geometry calculation unit uses the articulation angle signal as one of the input parameters, and combines the steering wheel angle signal, gear signal and vehicle speed signal to calculate the real-time driving trajectory of the semi-trailer through the vehicle kinematic geometry relationship, thereby determining the inner basic blind zone boundary of the semi-trailer when it is turning forward or reversing. The geometry calculation unit has a preset blind zone range expansion coefficient corresponding to the vehicle speed level. The blind zone range expansion coefficient is a proportional coefficient for expanding the basic blind zone boundary. The geometry calculation unit multiplies the basic blind zone boundary by the blind zone range expansion coefficient corresponding to the vehicle speed level to obtain the final high-risk blind zone boundary. The higher the vehicle speed, the larger the corresponding blind zone range expansion coefficient. The central control module (9) also has preset steering judgment conditions corresponding to the vehicle speed level. The steering judgment conditions include a steering wheel angle threshold and a steering angle duration threshold. The higher the vehicle speed, the smaller the steering wheel angle threshold and the shorter the steering angle duration threshold.

3. The dynamic blind spot monitoring system based on multi-channel cameras of the tractor and semi-trailer as described in claim 1, characterized in that, The image output terminals of the left-side wide-angle camera (5a) and the right-side wide-angle camera (5b) of the tractor are connected to a multi-channel video split-screen display terminal (6) to continuously transmit the collected images to the main window (7) of the tractor for display.

4. The dynamic blind spot monitoring system based on multi-channel cameras of the tractor and semi-trailer as described in claim 1, characterized in that, The central control module (9) is also used to perform fault self-check and degradation processing. The fault self-check and degradation processing includes: when any sensor module or camera signal is detected to be abnormal, controlling the blind spot window (8) to display fault prompt information; when the central control module (9) fails to perform blind spot position calculation or output screen switching and cropping control signals, the multi-channel video split screen display terminal (6) automatically displays the complete images of the left-side wide-angle camera (5a) and the right-side wide-angle camera (5b) of the tractor.

5. The dynamic blind spot monitoring system based on multi-channel cameras of the tractor and semi-trailer as described in claim 1, characterized in that, The left-side semi-trailer wide-angle camera (10a) and the right-side semi-trailer wide-angle camera (10b) are both installed on the semi-trailer body in a position that can completely cover the inner area of ​​the entire length of this side. The lens faces the rear side of the semi-trailer and tilts inward. The horizontal field of view covers the adjacent lanes inside the semi-trailer and the preset range behind the side. In the forward turning condition, the central control module (9) determines the vehicle turning direction according to the steering wheel angle signal, and switches to the left-side semi-trailer wide-angle camera (10a) for image cropping when the vehicle turns left, and switches to the right-side semi-trailer wide-angle camera (10b) for image cropping when the vehicle turns right. In the reversing condition, the central control module (9) determines the inner high-risk blind spot position of the semi-trailer during the reversing process according to the gear signal, steering wheel angle signal, articulation angle signal and vehicle speed signal, and switches to the semi-trailer wide-angle camera corresponding to the inner high-risk blind spot position for image cropping.

6. A dynamic blind spot monitoring and control method based on multi-channel cameras of a tractor and a semi-trailer, applied to the system described in any one of claims 1 to 5, characterized in that, The method includes: It reads steering wheel angle signal, gear signal, hinge angle signal, vehicle speed signal, and video streams from each camera in real time; Vehicle speed levels are classified based on vehicle speed signals, and corresponding steering judgment conditions and blind spot range expansion coefficients are matched for each vehicle speed level. When the current gear is forward or reverse, determine whether the absolute value and duration of the steering wheel angle meet the steering judgment conditions for the corresponding vehicle speed level. If the turning judgment condition is met, the position of the high-risk blind spot inside the semi-trailer when turning forward or reversing is determined by the articulation angle signal, steering wheel angle signal and vehicle speed signal, combined with the blind spot range expansion coefficient of the corresponding vehicle speed level. The turning direction is determined by the positive or negative of the steering wheel angle, and the camera is switched to the side wide-angle camera of the semi-trailer corresponding to the turning inside. The high-risk blind spot image is cropped and displayed in the dangerous blind spot window (8). If the steering judgment condition is not met or the gear is neutral / parking, the blind spot window (8) will be put into standby mode.

7. The dynamic blind spot monitoring and control method based on multi-channel cameras of the tractor and semi-trailer according to claim 6, characterized in that, Vehicle speed is classified into different levels based on the vehicle speed signal, and each speed level is matched with corresponding steering judgment conditions and blind spot range expansion coefficients, including: Based on the vehicle speed signal, the vehicle driving status is divided into three speed levels: low speed, medium speed and high speed, and corresponding steering judgment conditions and blind spot range expansion coefficients are matched for each speed level. The low-speed level corresponds to the first steering wheel angle threshold, the first angle duration threshold, and the first blind spot range expansion coefficient; the medium-speed level corresponds to the second steering wheel angle threshold, the second angle duration threshold, and the second blind spot range expansion coefficient; and the high-speed level corresponds to the third steering wheel angle threshold, the third angle duration threshold, and the third blind spot range expansion coefficient. The first steering wheel angle threshold is greater than the second steering wheel angle threshold, and the second steering wheel angle threshold is greater than the third steering wheel angle threshold; The first corner duration threshold is longer than the second corner duration threshold, and the second corner duration threshold is longer than the third corner duration threshold; The first blind zone range expansion coefficient is less than the second blind zone range expansion coefficient, and the second blind zone range expansion coefficient is less than the third blind zone range expansion coefficient.

8. The dynamic blind spot monitoring and control method based on multi-channel cameras of the tractor and semi-trailer according to claim 6, characterized in that, Methods for determining the location of the high-risk blind spot on the inside of a semi-trailer when it is turning forward or reversing include: The real-time driving trajectory of the semi-trailer is calculated by the vehicle's kinematic geometry to obtain the inner basic blind zone boundary. The basic blind zone boundary is then multiplied by the blind zone range expansion coefficient corresponding to the current vehicle speed level to obtain the final high-risk blind zone boundary. The higher the vehicle speed level, the larger the corresponding blind zone range expansion coefficient.

9. The dynamic blind spot monitoring and control method based on multi-channel cameras of the tractor and semi-trailer according to claim 6, characterized in that, The methods for cropping out the high-risk blind area and displaying it in the dangerous blind area window (8) include: selecting the smallest rectangular area that can completely contain the boundary of the high-risk blind area as the cropping area, and adapting it to the display ratio of the dangerous blind area window (8) for full-screen display.

10. The dynamic blind spot monitoring and control method based on multi-channel cameras of the tractor and semi-trailer according to claim 6, characterized in that, The method further includes fault self-checking and degradation processing, wherein the fault self-checking and degradation processing includes: The system monitors the signal status of each sensor module and camera in real time. When an abnormal signal is detected, a fault message is displayed in the blind spot window (8), while the main window (7) of the tractor remains in normal display. When the central control module fails to perform blind spot calculation and display control, the multi-channel video split-screen display terminal (6) automatically displays the complete images of the lateral wide-angle cameras on both sides of the tractor.

Images