Method for realizing visual field follow-up of electronic rearview mirror based on rearview visual field image recognition

Abstract

The invention discloses a method for realizing visual field follow-up of an electronic rearview mirror based on rearview visual field image recognition. The method comprises the following steps: S1, image acquisition and transmission: acquiring and transmitting an image through a camera module mounted in an outer ear rearview mirror shell outside a vehicle body; integrated image acquisition, processing and execution devices are combined to form a mode of directly perceiving road conditions through image recognition, the problems that after-loading market signals are difficult to acquire and potential safety hazards exist due to broken line signal acquisition are solved, the system adapts to the whole front and back loading market, the view angle is dynamically adjusted, a pull steering blind area is specifically eliminated, and the steering efficiency is improved. The adaptability is improved by dynamically optimizing follow-up parameters in combination with the double-judgment principle, the view requirements of different road conditions and trailer models can be accurately matched, the blind area of the driving view is effectively eliminated, the driving safety and convenience are improved, the integrated structural design is achieved, extra wiring and breaking operation are not needed, and the practicability is high. No potential safety hazard exists, and the practicability is high.

Description

Technical Field

[0001] This invention relates to the field of electronic rearview mirror control technology, and in particular to a method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition. Background Technology

[0002] The field-following function of electronic rearview mirrors is a key feature for improving driving safety. Its core requirement is to dynamically adjust the field of view of the rearview mirror according to the driving conditions and eliminate blind spots. There are two main ways to implement the existing electronic rearview mirror field-following technology: one is to read the steering wheel rotation angle and adjust the rearview mirror field of view accordingly; the other is to read the turn signal status and switch the fixed field of view angle of the rearview mirror.

[0003] However, existing technologies have significant drawbacks: 1. Current methods adjust visibility indirectly through vehicle body signals, which do not meet the actual visibility requirements of driving conditions, resulting in poor adjustment accuracy and adaptability; 2. They rely on the vehicle body to provide corresponding steering signals, such as steering wheel angle. Most domestic models are not equipped with steering wheel angle sensors, and aftermarket products are unlikely to obtain this signal. Obtaining this signal by cutting wires also poses certain safety hazards; 3. Turn signal signals can only provide two states: "turning" and "not turning," without specific adjustment angles, and cannot actually reflect the visibility adjustment requirements of actual driving conditions; 4. They cannot match suitable visibility adjustment requirements for trailer models because each trailer has a different length, and the required adjustment angle is not fixed, which can easily create blind spots.

[0004] Therefore, there is an urgent need for an electronic rearview mirror field-following solution that does not rely on vehicle body sensor signals, can dynamically adjust according to actual road conditions, and is compatible with both OEM and trailer markets, in order to solve many pain points of existing technologies. In view of this, this application proposes a method for realizing electronic rearview mirror field-following based on rearview field image recognition. Summary of the Invention

[0005] Based on the technical problems existing in the background technology, the present invention proposes a method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition.

[0006] The method for achieving electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention includes the following steps: S1: Image Acquisition and Transmission: The camera module installed in the outer ear rearview mirror housing of the vehicle body acquires the visual video signal of the side and rear of the vehicle body in real time, and transmits the video signal frame by frame to the video computing processor in the car through the transmission line. S2: Image Recognition and Feature Extraction: The video computing processor uses the built-in BSD image recognition algorithm and employs convolutional neural networks or traditional machine learning methods to analyze the video signal frame by frame and extract key feature information of reference objects, including pedestrians, vehicles, two-wheeled vehicles and calibration reference points. S3: Field of view tracking judgment: Based on the feature information output by the BSD image recognition algorithm, the relative attitude parameters between the reference object and the vehicle body are calculated by the field of view tracking judgment algorithm. Combined with the driving status, the field of view adjustment needs are predicted. The relative attitude parameters include the horizontal angle between the reference object and the vehicle body, the relative distance and the relative speed. S4: Execute field of view adjustment: Based on the field of view tracking judgment result, the video computing power processor outputs a drive signal to the field of view tracking execution device, which drives the camera module to rotate around the pivot, thereby realizing the precise tracking adjustment of the field of view of the electronic rearview mirror. S5: Adaptive Optimization: Records the driver's vision adjustment habits and dynamically optimizes the vision adjustment range based on the vision follow-up effect under different road conditions to adapt to diverse driving scenarios.

[0007] Preferably, the video computing processor in S1 includes a display module, a main control chip PCBA, and a display housing. The main control chip and the display module can be integrated or arranged separately. The video signal is displayed in real time after low-latency processing. The camera module includes a camera and a camera bracket. The camera resolution is not less than 1080P, the frame rate is not less than 30fps, and shielded cable is used for transmission with a transmission delay of ≤100ms.

[0008] Preferably, the BSD image recognition algorithm in S2 adopts a lightweight convolutional neural network model with an inference latency of ≤50ms; the horizontal angle calculation of the reference object adopts a trigonometric function algorithm with a calculation error of ≤±0.5°, providing data support for subsequent angle parameters including but not limited to ∠A, ∠B, and ∠C1.

[0009] Preferably, the field-of-view tracking algorithm in S3 includes a basic judgment principle PD1 and an extended judgment principle PD2: PD1: Based on the BSD image recognition algorithm, obtain the distance X1 perpendicular to the vehicle body and the distance Y1 parallel to the vehicle body of the reference object in state one, and calculate the horizontal angle ∠A; obtain the distances X2 and Y2 in state two, calculate the horizontal angle ∠B, and compare the changes in ∠A and ∠B to judge the changes in vehicle body posture. PD2: Based on the BSD image recognition algorithm, the vehicle body calibration reference points Q1 / Q2 are introduced. The rear reference point and Q1 / Q2 are used to determine the parallel plane of the vehicle body. The three included angles ∠C1 and ∠E1 and the four included angles ∠C2 and ∠E2 are obtained, and ∠F1 and ∠F2 are calculated. Combined with the relative distance between the reference object and the vehicle body, the motion trajectory is predicted to optimize the judgment accuracy.

[0010] Preferably, the field-following actuator in S4 includes a stepper motor, a worm gear transmission mechanism, a rotating shaft, and an encoder. The top of the rotating shaft is connected to the camera module, and its bottom is rotatably connected to the bottom of the inner side of the rearview mirror housing via a bearing. A transparent cover for protecting the camera module is fixed on the surface of the rearview mirror housing. The stepper motor is connected to the rotating shaft via the worm gear transmission mechanism. The encoder detects the rotation angle of the camera module in real time and provides feedback, forming an angle closed-loop compensation control with an adjustment accuracy of ≤±0.1°.

[0011] Preferably, the stepper motor step angle is ≤1.8°, the worm gear transmission ratio of the worm gear transmission mechanism is ≥1:50, and the encoder resolution is ≥1024 lines, to ensure precise control of the field of view adjustment angle such as ∠E5.

[0012] Preferably, the method is applicable to towed vehicles, and the specific adaptation steps are as follows: A: Configure the rear of the trailer vehicle with the designated points Q3 / Q4 / Q5; B: Obtain the horizontal angles ∠E3 and ∠E4 between the reference object and Q3 / Q4 / Q5 under different states using the PD1 algorithm; C: The field-of-view follow-up actuator adjusts the camera's field-of-view angle ∠E5, expanding the observation range and eliminating blind spots during trailer steering.

[0013] Preferably, the method can work independently or is compatible with existing vision adjustment methods based on steering wheel rotation angle or turn signal status. By combining actual image judgment with vehicle body signals through a vision detection arbitration system, the follow-up effect is optimized.

[0014] Preferably, in step S5, adaptive optimization establishes a road condition-adjustment amplitude mapping model by statistically analyzing driver operation data under different road conditions, and dynamically adjusts the field of vision follow-up sensitivity by 0.3-1.2 times the benchmark value.

[0015] Preferably, the outer rearview mirror housing and the camera module are an integrated fixed structure, which is compatible with both the OEM and aftermarket, and installation does not require cutting wires to obtain signals; the calibration reference points include vehicle body calibration reference points Q1 / Q2 and towed vehicle-specific calibration points Q3 / Q4 / Q5 to ensure accurate acquisition of the included angle parameters.

[0016] Compared with existing technologies, the beneficial effects of this invention are: 1. Through an integrated image acquisition, processing and execution device, combined with the calculation of calibration reference points such as Q1 / Q2, Q3 / Q4 / Q5 and the included angle parameters such as ∠A and ∠B, the road conditions can be perceived directly through image recognition, solving the problems of difficult signal acquisition in the aftermarket and safety hazards of cutting lines to obtain signals. It does not need to rely on vehicle body sensor signals and is compatible with the entire aftermarket. 2. The camera and lightweight algorithm ensure the real-time performance of data acquisition and processing. Closed-loop control ensures that the adjustment angle accuracy of ∠E5 is ≤ ±0.1°, and the calculation error of the included angle of ∠A, ∠B is ≤ ±0.5°. It can accurately match the field of vision requirements of actual road conditions, with high adjustment accuracy and strong real-time performance. 3. By coordinating the calculation of the included angle parameters such as exclusive calibration points Q3 / Q4 / Q5 and ∠E3, ∠E4, ∠E5, the field of vision angle is dynamically adjusted to specifically eliminate blind spots in towed vehicle steering, fill existing technological gaps, improve the adaptability of towed vehicle models, accurately match the field of vision requirements of different road conditions and towed vehicle models, effectively eliminate blind spots in driving vision, and improve driving safety and convenience; 4. By recording the driver's operating habits, the follow-up parameters are dynamically optimized. At the same time, it is compatible with existing adjustment methods. Combining the PD1 and PD2 dual judgment principles improves adaptability, enhances the flexibility and user experience, and has the advantages of strong adaptability and compatibility. 5. The integrated structural design eliminates the need for additional wiring or wire cutting, making installation convenient, safe, and highly practical.

[0017] This invention combines an integrated image acquisition, processing, and execution device to directly perceive road conditions through image recognition, solving the problems of difficult signal acquisition in the aftermarket and the safety hazards of cutting wires to obtain signals. It is compatible with both aftermarket and aftermarket products. By dynamically adjusting the field of vision angle, it specifically eliminates blind spots for towed vehicles, improving the adaptability of towed vehicle models. By dynamically optimizing follow-up parameters and combining a dual judgment principle, it enhances adaptability, accurately matching the field of vision requirements of different road conditions and towed vehicle models, effectively eliminating blind spots, improving driving safety and convenience. Moreover, the integrated structural design eliminates the need for additional wiring and wire cutting operations, making installation convenient and safe, and highly practical. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention; Figure 2 This is a schematic diagram of the basic judgment principle PD1 in the method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention; Figure 3 This is a schematic diagram of the extended judgment principle PD2 in the method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention; Figure 4 This is a schematic diagram illustrating the adaptation of a trailer vehicle in the method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention. Figure 5This is a schematic diagram of the camera at different angles in the external rearview mirror of the vehicle body, which is used in the method of realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention. Figure 6 This is a schematic diagram of the vehicle's external rearview mirror with an integrated camera module, used in the method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition proposed in this invention. Figure 7 This is a schematic diagram of the interior of the external rearview mirror of the vehicle body, which is an integrated camera module for realizing the electronic rearview mirror field of view tracking based on rearview field of view image recognition as proposed in this invention. Figure 8 This is a top-view diagram of the internal structure of the vehicle's external rearview mirror, which is part of the integrated camera module for the electronic rearview mirror's field of view tracking method based on rear-view image recognition proposed in this invention.

[0019] In the diagram: 1. Encoder; 2. Camera bracket; 3. Camera; 4. Worm gear transmission mechanism; 6. Bearing; 7. Stepper motor; 8. Transparent cover. Detailed Implementation

[0020] The present invention will be further explained below with reference to specific embodiments. Example

[0021] Reference Figure 1-8 This embodiment proposes a method for realizing electronic rearview mirror field-of-view tracking based on rearview field-of-view image recognition, including the following steps: S1: Image Acquisition and Transmission: The camera module installed in the outer ear rearview mirror housing of the vehicle body acquires the visual video signal of the side and rear of the vehicle body in real time, and transmits the video signal frame by frame to the video computing processor in the car through the transmission line. The video computing processor includes a display module, a main control chip PCBA, and a display housing. The main control chip and the display module can be integrated or arranged separately. The video signal is displayed in real time after low-latency processing. The camera module includes a camera 3 and a camera bracket 2. The camera 3 has a resolution of no less than 1080P and a frame rate of no less than 30fps. It uses shielded cable transmission and the transmission latency is ≤100ms. S2: Image Recognition and Feature Extraction: The video computing processor uses the built-in BSD image recognition algorithm and employs convolutional neural networks or traditional machine learning methods to analyze the video signal frame by frame and extract key feature information of reference objects, including pedestrians, vehicles, two-wheeled vehicles and calibration reference points. The BSD image recognition algorithm uses a lightweight convolutional neural network model with an inference latency of ≤50ms; the horizontal angle calculation of the reference object uses a trigonometric function algorithm with a calculation error of ≤±0.5°, providing data support for subsequent angle parameters including but not limited to ∠A, ∠B, and ∠C1. S3: Field of View Follow-up Judgment: Based on the feature information output by the BSD image recognition algorithm, the relative attitude parameters between the reference object and the vehicle body are calculated by the field of view follow-up judgment algorithm. Combined with the driving status, the field of view adjustment needs are predicted. The relative attitude parameters include the horizontal angle between the reference object and the vehicle body, the relative distance, and the relative speed. The field-of-view tracking judgment algorithm includes the basic judgment principle PD1 and the extended judgment principle PD2: PD1: Based on the BSD image recognition algorithm, obtain the distance X1 perpendicular to the vehicle body and the distance Y1 parallel to the vehicle body of the reference object in state one, and calculate the horizontal angle ∠A; obtain the distances X2 and Y2 in state two, calculate the horizontal angle ∠B, and compare the changes in ∠A and ∠B to judge the changes in vehicle body posture. PD2: Based on the BSD image recognition algorithm, the vehicle body calibration reference points Q1 / Q2 are introduced. The rear reference point and Q1 / Q2 are used to determine the parallel plane of the vehicle body; the three included angles ∠C1 and ∠E1 in the state and the four included angles ∠C2 and ∠E2 in the state are obtained, and ∠F1 and ∠F2 are calculated; combined with the relative distance between the reference object and the vehicle body, the motion trajectory is predicted to optimize the judgment accuracy. S4: Execute field of view adjustment: Based on the field of view tracking judgment result, the video computing power processor outputs a drive signal to the field of view tracking execution device, which drives the camera module to rotate around the pivot, thereby realizing the precise tracking adjustment of the field of view of the electronic rearview mirror. The vision-following actuator includes a stepper motor 7, a worm gear transmission mechanism 4, a rotating shaft, and an encoder 1. The top of the rotating shaft is connected to the camera module, and its bottom is rotatably connected to the bottom of the inner side of the rearview mirror housing via a bearing 6. A transparent cover 8 for protecting the camera module is fixed on the surface of the rearview mirror housing. The stepper motor 7 is connected to the rotating shaft via the worm gear transmission mechanism 4. The encoder 1 detects the rotation angle of the camera module in real time and provides feedback to form an angle closed-loop compensation control with an adjustment accuracy of ≤±0.1°. Among them, the stepper motor 7 has a step angle ≤1.8°, the worm gear transmission mechanism 4 has a worm gear transmission ratio ≥1:50, and the encoder 1 has a resolution ≥1024 lines, ensuring precise control of the field of view adjustment angle such as ∠E5; S5: Adaptive Optimization: Records the driver's vision adjustment habits and dynamically optimizes the vision follow-up adjustment range based on the vision follow-up effect under different road conditions to adapt to diverse driving scenarios. The adaptive optimization model establishes a road condition-adjustment range mapping model by statistically analyzing driver operation data under different road conditions (straight, turning, high speed, low speed congestion) and dynamically adjusts the field of vision follow-up sensitivity by 0.3-1.2 times the baseline value. The outer rearview mirror housing and camera module are integrated into a fixed structure, suitable for both OEM and aftermarket installations, and installation does not require cutting wires to obtain signals; the calibration reference points include the vehicle body calibration reference points Q1 / Q2 and the towed vehicle-specific calibration points Q3 / Q4 / Q5, ensuring accurate acquisition of the included angle parameters.

[0022] Furthermore, the method applies to towed vehicles, and the specific adaptation steps are as follows: A: Configure the rear markers of trailer vehicles with Q3 / Q4 / Q5 (using high-contrast reflective markings); B: Obtain the horizontal angles ∠E3 and ∠E4 between the reference object and Q3 / Q4 / Q5 under different states using the PD1 algorithm; C: The field-of-view follow-up actuator adjusts the camera's field-of-view angle ∠E5, expanding the observation range and eliminating blind spots during trailer steering.

[0023] Furthermore, the method can work independently or be compatible with existing vision adjustment methods based on steering wheel rotation angle or turn signal status. By combining actual image judgment with vehicle body signals through a vision detection arbitration system, the follow-up effect is optimized.

[0024] This embodiment combines an integrated image acquisition, processing, and execution device to directly perceive road conditions through image recognition, solving the problems of difficult signal acquisition in the aftermarket and the safety hazards of cutting wires to obtain signals. It is compatible with both aftermarket and aftermarket products. By dynamically adjusting the field of vision angle, it specifically eliminates blind spots for towed vehicles, improving the adaptability of towed vehicle models. By dynamically optimizing follow-up parameters and combining them with a dual judgment principle, it enhances adaptability and can accurately match the field of vision requirements of different road conditions and towed vehicle models, effectively eliminating blind spots and improving driving safety and convenience. Moreover, the integrated structural design eliminates the need for additional wiring and wire cutting operations, making installation convenient and safe, and highly practical.

[0025] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for realizing electronic rearview mirror field-of-view tracking based on rearview field-of-view image recognition, characterized in that, Includes the following steps: S1: Image Acquisition and Transmission: The camera module installed in the outer ear rearview mirror housing of the vehicle body acquires the visual video signal of the side and rear of the vehicle body in real time, and transmits the video signal frame by frame to the video computing processor in the car through the transmission line. S2: Image Recognition and Feature Extraction: The video computing processor uses the built-in BSD image recognition algorithm and employs convolutional neural networks or traditional machine learning methods to analyze the video signal frame by frame and extract key feature information of reference objects, including pedestrians, vehicles, two-wheeled vehicles and calibration reference points. S3: Field of view tracking judgment: Based on the feature information output by the BSD image recognition algorithm, the relative attitude parameters between the reference object and the vehicle body are calculated by the field of view tracking judgment algorithm. Combined with the driving status, the field of view adjustment needs are predicted. The relative attitude parameters include the horizontal angle between the reference object and the vehicle body, the relative distance and the relative speed. S4: Execute field of view adjustment: Based on the field of view tracking judgment result, the video computing power processor outputs a drive signal to the field of view tracking execution device, which drives the camera module to rotate around the pivot, thereby realizing the precise tracking adjustment of the field of view of the electronic rearview mirror. S5: Adaptive Optimization: Records the driver's vision adjustment habits and dynamically optimizes the vision adjustment range based on the vision follow-up effect under different road conditions to adapt to diverse driving scenarios.

2. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The video computing processor in S1 includes a display module, a main control chip PCBA and a display housing. The main control chip and the display module can be integrated or arranged separately. The video signal is displayed in real time after low-latency processing. The camera module includes a camera (3) and a camera bracket (2). The camera (3) has a resolution of not less than 1080P and a frame rate of not less than 30fps. It uses shielded wire transmission and the transmission delay is ≤100ms.

3. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The BSD image recognition algorithm in S2 adopts a lightweight convolutional neural network model with an inference latency of ≤50ms; the horizontal angle calculation of the reference object adopts a trigonometric function algorithm with a calculation error of ≤±0.5°, providing data support for subsequent angle parameters including but not limited to ∠A, ∠B, and ∠C1.

4. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The field-of-view tracking judgment algorithm in S3 includes basic judgment principle PD1 and extended judgment principle PD2: PD1: Based on the BSD image recognition algorithm, obtain the distance X1 perpendicular to the vehicle body and the distance Y1 parallel to the vehicle body of the reference object in state one, and calculate the horizontal angle ∠A; obtain the distances X2 and Y2 in state two, calculate the horizontal angle ∠B, and compare the changes in ∠A and ∠B to judge the changes in vehicle body posture. PD2: Based on the BSD image recognition algorithm, the vehicle body calibration reference points Q1 / Q2 are introduced. The rear reference point and Q1 / Q2 are used to determine the parallel plane of the vehicle body. The three included angles ∠C1 and ∠E1 and the four included angles ∠C2 and ∠E2 are obtained, and ∠F1 and ∠F2 are calculated. Combined with the relative distance between the reference object and the vehicle body, the motion trajectory is predicted to optimize the judgment accuracy.

5. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The field-following actuator in S4 includes a stepper motor (7), a worm gear transmission mechanism (4), a rotating shaft, and an encoder (1). The top of the rotating shaft is connected to the camera module, and its bottom is rotatably connected to the bottom of the inner side of the rearview mirror housing through a bearing (6). A transparent cover (8) for protecting the camera module is fixed on the surface of the rearview mirror housing. The stepper motor (7) is connected to the rotating shaft through the worm gear transmission mechanism (4). The encoder (1) detects the rotation angle of the camera module in real time and provides feedback to form an angle closed-loop compensation control with an adjustment accuracy of ≤ ±0.1°.

6. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 5, characterized in that, The stepper motor (7) has a step angle ≤ 1.8°, the worm gear transmission mechanism (4) has a worm gear transmission ratio ≥ 1:50, and the encoder (1) has a resolution ≥ 1024 lines, ensuring precise control of the field of view adjustment angle such as ∠E5.

7. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The method is applicable to towed vehicles, and the specific adaptation steps are as follows: A: Configure the rear of the trailer vehicle with the designated points Q3 / Q4 / Q5; B: Obtain the horizontal angles ∠E3 and ∠E4 between the reference object and Q3 / Q4 / Q5 under different states using the PD1 algorithm; C: The field-of-view follow-up actuator adjusts the camera's field-of-view angle ∠E5, expanding the observation range and eliminating blind spots during trailer steering.

8. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The method can work independently or is compatible with existing vision adjustment methods based on steering wheel rotation angle or turn signal status. It optimizes the follow-up effect by combining actual image judgment with vehicle body signals through a vision detection arbitration system.

9. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, In S5, adaptive optimization establishes a road condition-adjustment range mapping model by statistically analyzing driver operation data under different road conditions, and dynamically adjusts the field of vision follow-up sensitivity by 0.3-1.2 times the benchmark value.

10. The method for realizing electronic rearview mirror field of view tracking based on rearview field of view image recognition according to claim 1, characterized in that, The outer rearview mirror housing and camera module are integrated into a fixed structure, suitable for both OEM and aftermarket installations, and installation does not require cutting wires to obtain signals; the calibration reference points include vehicle body calibration reference points Q1 / Q2 and towed vehicle-specific calibration points Q3 / Q4 / Q5, ensuring accurate acquisition of included angle parameters.

Images