A display method, system, and storage medium for an in-vehicle electronic rearview mirror.

Abstract

This invention relates to the field of vehicle-mounted equipment control technology, and particularly to a display method for an in-vehicle electronic rearview mirror. The method includes: after the vehicle is started, acquiring the driver's pupil position and an image of the vehicle's external traffic conditions; if the pupil position is located within the electronic rearview mirror display area of ​​a preset pupil coordinate system, and the pupil position continuously moves within the electronic rearview mirror display area, then determining that the vehicle is in lane-changing mode; in lane-changing mode, cropping the external traffic condition image to obtain a cropped external traffic condition image, and displaying the cropped external traffic condition image on the vehicle's corresponding display screen, wherein the size of the image displayed on the corresponding display screen is smaller than the size of the external traffic condition image. This method enables the content displayed on the in-vehicle display screen to be synchronously adjusted according to the driver's line of sight movement, tracking the driver's observation range, greatly adapting to different driver observation habits, improving driver convenience, and ensuring vehicle driving safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle-mounted device control, and in particular to a display method and system of a vehicle-mounted electronic rearview mirror and a storage medium. BACKGROUND

[0002] At present, a traditional vehicle rearview mirror is a convex mirror, and the display area of the convex mirror is directly related to the size of the mirror body. The display range of the traditional vehicle rearview mirror is very limited, and when it is rainy, snowy, cloudy or at night, the mirror surface of the rearview mirror is covered with water, which causes poor visibility when the water is exposed to light. The driver observes the transmitted rearview mirror through the vehicle glass, which results in very low visibility and affects the driving of the driver, thereby causing safety risks. In order to change the disadvantages of the traditional rearview mirror, a vehicle-mounted electronic rearview mirror system is generated.

[0003] The vehicle-mounted electronic rearview mirror includes a high-definition camera, a processor and an in-vehicle display screen. The high-definition camera is arranged outside the vehicle body and is used to send the collected image outside the vehicle to the processor directly, and the processor drives the display screen to display the image outside the vehicle, so that the driver can obtain the image outside the vehicle in a larger range by observing the in-vehicle display screen, and thus more accurately judge the road conditions. Since the high-definition camera has a wide field of view and strong adaptability to various environments, the problem of poor observation conditions and limited observation range of the traditional rearview mirror in complex weather can be solved.

[0004] However, due to individual differences of the driver, the sitting posture of the driver on the driver's seat is different, and the observation habits and the observation positions of interest of different drivers are also different. These differences result in that there is still a blind area when the driver watches the display screen, or the driver needs to constantly adjust the observation direction and the posture on the seat to obtain a wider observation range. These limitations all cause inconvenience in using the electronic rearview mirror system. SUMMARY

[0005] The display method, system and storage medium of the vehicle-mounted electronic rearview mirror provided by the embodiments of the present application solve the technical problem that the electronic rearview mirror system is inconvenient to use due to individual differences and use habit differences of the driver in the prior art, and achieve the technical effects that the display content of the in-vehicle display screen can be adjusted in real time according to the line of sight movement of the driver of the vehicle, the observation range of the driver is tracked, different observation habits of the driver are greatly adapted, the convenience of the driver of the vehicle is improved, and the driving safety of the vehicle is ensured.

[0006] In a first aspect, the embodiments of the present application provide a display method of a vehicle-mounted electronic rearview mirror, comprising:

[0007] After the vehicle is started, the pupil position of the driver of the vehicle and the image of the road conditions outside the vehicle are acquired;

[0008] If the pupil position is located in the electronic rearview mirror display area of ​​the preset pupil coordinate system, and the pupil position is detected to be continuously moving in the electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode.

[0009] In the lane-changing mode, the external vehicle condition image is cropped to obtain a cropped external vehicle condition image, and the cropped external vehicle condition image is displayed on the vehicle's corresponding display screen, wherein the size of the image displayed on the corresponding display screen is smaller than the size of the external vehicle condition image.

[0010] Preferably, the step of determining that the vehicle is in lane-changing mode if the pupil position is located within the electronic rearview mirror display area of ​​a preset pupil coordinate system, and the pupil position is continuously moving within the electronic rearview mirror display area, includes:

[0011] If the pupil position is located within the left electronic rearview mirror display area of ​​the electronic rearview mirror display area, and the pupil position is detected to be continuously moving within the left electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode; or,

[0012] If the pupil position is located in the right electronic rearview mirror display area of ​​the electronic rearview mirror display area, and the pupil position is detected to be continuously moving in the right electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode.

[0013] Preferably, in the lane-changing mode, cropping the external vehicle condition image to obtain a cropped external vehicle condition image, and displaying the cropped external vehicle condition image on the vehicle's corresponding display screen, includes:

[0014] In the left lane change mode of the lane change mode, the left side of the vehicle exterior image is cropped to obtain a cropped left side vehicle exterior image, and the cropped left side vehicle exterior image is displayed on the left side display screen inside the vehicle, wherein the corresponding display screen includes the left side display screen.

[0015] Preferably, in the lane-changing mode, cropping the external vehicle condition image to obtain a cropped external vehicle condition image, and displaying the cropped external vehicle condition image on the vehicle's corresponding display screen, includes:

[0016] In the right-side lane change mode of the lane change mode, the right-side exterior vehicle condition image of the exterior vehicle condition image is cropped to obtain a cropped right-side exterior vehicle condition image, and the cropped right-side exterior vehicle condition image is displayed on the right-side display screen inside the vehicle, wherein the corresponding display screen includes the right-side display screen.

[0017] Preferably, after acquiring the driver's pupil position and the external vehicle condition image, the method further includes:

[0018] If the pupil position is located in the fatigue warning area of ​​the preset pupil coordinate system, then the vehicle is determined to be in straight-line driving mode.

[0019] In the straight-line driving mode, the external vehicle condition image is directly displayed on the corresponding display screen, and the driver's facial image is continuously captured;

[0020] If, within a preset time period, the frequency of detecting the absence of the driver's pupil information in the face image is not less than a preset detection frequency, then fatigue warning information is sent to the vehicle's body controller. The body controller then determines the driver's current fatigue warning level based on the detection frequency of the fatigue warning information, and outputs vehicle warning information based on the current fatigue warning level.

[0021] Preferably, after acquiring the driver's pupil position and the external vehicle condition image, the method further includes:

[0022] If the pupil position is not located in the preset pupil coordinate system, the vehicle is determined to be in a dangerous driving mode.

[0023] In the dangerous driving mode, dangerous driving information is sent to the body controller, so that the body controller outputs vehicle alarm information based on the dangerous driving information.

[0024] Preferably, after the vehicle is started, and before acquiring the driver's pupil position and the external vehicle condition image, the method further includes:

[0025] Obtain the driver's pupil information and facial information;

[0026] If the pupil information matches the pre-stored pupil information, and / or the facial information matches the pre-stored facial information, then the driver's seat of the vehicle will be adjusted to the pre-stored driver's seat position.

[0027] Based on the same inventive concept, in a second aspect, the present invention also provides a display system for an in-vehicle electronic rearview mirror, which operates on the display method of the in-vehicle electronic rearview mirror as described above. The system includes: a SOC module, and an in-vehicle camera, an external camera, and a display screen connected to the SOC module. The in-vehicle camera is located directly in front of the driver's seat of the vehicle, and the external camera is located outside the vehicle body. The resolution of the display screen is lower than the resolution of the external camera, and the external camera transmits images digitally.

[0028] The in-vehicle camera is used to acquire the position of the driver's pupils after the vehicle is started.

[0029] The external camera is used to acquire images of the vehicle's external conditions after the vehicle is started.

[0030] The SOC module is configured to determine that the vehicle is in lane-changing mode if the pupil position is located in the electronic rearview mirror display area of ​​a preset pupil coordinate system and the pupil position continues to move in the electronic rearview mirror display area; in the lane-changing mode, the external vehicle condition image is cropped to obtain a cropped external vehicle condition image, and the cropped external vehicle condition image is displayed on the vehicle's corresponding display screen, wherein the size of the image displayed on the corresponding display screen is smaller than the size of the external vehicle condition image.

[0031] Preferably, the exterior camera includes a left-side exterior camera and a right-side exterior camera, with the left-side exterior camera located on the left side of the vehicle body and the right-side exterior camera located on the right side of the vehicle body.

[0032] The external left-side camera is used to acquire the external vehicle condition image of the vehicle's external left-side condition.

[0033] The external right-side camera is used to acquire images of the external vehicle conditions on the right side of the vehicle.

[0034] The display screen includes a left-side display screen and a right-side display screen inside the vehicle. The left-side display screen is located on the left side of the steering wheel inside the vehicle, and the right-side display screen is located on the right side of the steering wheel inside the vehicle.

[0035] The left-side display screen is used to display the image of the vehicle's left-side exterior condition;

[0036] The right-side display screen is used to display images of the vehicle's right-side exterior.

[0037] Based on the same inventive concept, in a third aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a display method for an in-vehicle electronic rearview mirror.

[0038] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:

[0039] In this embodiment of the invention, after the vehicle is started, the driver's pupil position and an image of the external traffic conditions are acquired to lay a solid foundation for the subsequent adjustment and display of the external traffic conditions image. If the pupil position is located within the electronic rearview mirror display area of ​​a preset pupil coordinate system, and the pupil position is detected to continuously move within the electronic rearview mirror display area, the vehicle is determined to be in lane-changing mode. In lane-changing mode, the external traffic conditions image is cropped to obtain a cropped external traffic conditions image, and the cropped external traffic conditions image is displayed on the vehicle's corresponding display screen, wherein the size of the image displayed on the corresponding display screen is smaller than the size of the external traffic conditions image.

[0040] Thus, through the display method of this invention, the content displayed on the in-vehicle display screen can be adjusted synchronously in real time according to the driver's line of sight, completing the tracking of the driver's observation range, greatly adapting to different drivers' observation habits, improving the convenience of use for drivers, and ensuring vehicle driving safety. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0042] Figure 1 A flowchart illustrating the steps of the display method for an in-vehicle electronic rearview mirror according to an embodiment of the present invention is shown.

[0043] Figure 2 A schematic diagram of the display system of the vehicle electronic rearview mirror in an embodiment of the present invention is shown;

[0044] Figure 3 A schematic diagram of the preset pupil coordinate system in an embodiment of the present invention is shown;

[0045] Figure 4 This diagram illustrates a comparison between the size of the vehicle exterior image and the display frame of the screen in an embodiment of the present invention. Detailed Implementation

[0046] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0047] Example 1

[0048] The first embodiment of the present invention provides a display method for a vehicle-mounted electronic rearview mirror, such as... Figure 1 As shown, it includes:

[0049] S101, after the vehicle is started, acquire the driver's pupil position and the vehicle's external vehicle condition image;

[0050] S102, if the pupil position is located in the electronic rearview mirror display area of ​​the preset pupil coordinate system, and the detected pupil position continues to move in the electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode.

[0051] S103, in lane change mode, crop the image of the external vehicle conditions to obtain a cropped image of the external vehicle conditions, and display the cropped image of the external vehicle conditions on the vehicle's corresponding display screen. The size of the image displayed on the corresponding display screen is smaller than the size of the image of the external vehicle conditions.

[0052] The display method of the vehicle-mounted electronic rearview mirror in this embodiment is applied to the display system of the vehicle-mounted electronic rearview mirror, specifically to the SOC module 201 of the display system, so that the SOC module 201 executes the steps of the display method of the vehicle-mounted electronic rearview mirror in this embodiment. For example... Figure 2 As shown, the display system includes: a SOC (System on Chip) module 201, and an in-vehicle camera 202, an external camera 203, and a display screen 204 connected to the SOC module 201. The in-vehicle camera 202 is positioned directly in front of the driver's seat, such as directly in front of the steering wheel. The external camera 203 is positioned outside the vehicle body. The resolution of the display screen 204 is lower than that of the external camera 203, and the external camera 203 transmits images digitally. Specifically, the external camera 203 is connected to the CPU (Central Processing Unit) module of the SOC module 201, the in-vehicle camera 202 is connected to the GPU (Graphics Processing Unit) module of the SOC module 201, and the body controller 205 is connected to the NPU (Neural Network Processing Unit) module of the SOC module 201 via an MCU (Motor Control Unit) module.

[0053] The in-vehicle camera 202 is used to acquire the driver's pupil position and facial information after the vehicle is started. The facial information includes a facial image, and the pupil position is the driver's pupil gaze location within a preset pupil coordinate system. Specifically, the in-vehicle camera 202 is an infrared camera. The in-vehicle camera 202 is specifically used for: facial recognition and / or pupil recognition authentication of the driver after the vehicle is started; and tracking the driver's pupil position and pupil movement trajectory while driving. To recognize and capture the driver's eye movements when looking at the left and right displays, this camera is best positioned directly in front of the driver. Technically, it uses a 940nm near-infrared camera mode, with a built-in 940nm infrared auxiliary light. When working, the auxiliary light emits infrared light, and the in-vehicle camera 202 receives the reflected infrared light to form an infrared image. The image is displayed at 1080p resolution, detecting the driver's face and pupils, accurately describing and recording the driver's pupil movement direction and trajectory. When pupil recognition and positioning are not required, the system can use the camera to detect the driver's state, whether the driver is in a state of fatigue driving with eyes closed or yawning, and whether there are dangerous driving behaviors such as smoking, making or receiving phone calls, or looking around, and complete the actions of fatigue driving warning and dangerous driving alarm.

[0054] The exterior camera 203 is used to acquire images of the external traffic conditions after the vehicle is started. Specifically, the exterior camera 203 includes a left-side exterior camera and a right-side exterior camera. The left-side exterior camera is located on the left side of the vehicle body, and the right-side exterior camera is located on the right side of the vehicle body. The left-side exterior camera acquires images of the left side of the vehicle's exterior traffic conditions. The right-side exterior camera acquires images of the right side of the vehicle's exterior traffic conditions.

[0055] The exterior camera 203 includes two digital high-definition cameras: a left-side camera and a right-side camera. Since this method uses cameras to display rear-view images instead of traditional physical rearview mirrors, the image display must be real-time or near real-time. To meet standard latency requirements, the signal transmission of the exterior camera 203 must minimize delay to meet the driver's real-time observation needs. Therefore, the exterior camera 203 must use digital transmission to minimize signal transmission latency. Secondly, to meet the requirements for adjustable field of view, the exterior camera 203 uses a 1080p resolution, while the in-vehicle display screen 204 uses a 720p resolution screen. The image size captured by the exterior camera 203 is larger than the image display size of the display screen 204 to facilitate image cropping during field of view adjustments and ensure sufficient image clarity after cropping. The field of view of the exterior camera 203 is greater than 60 degrees both horizontally and vertically.

[0056] The display screen 204 includes a left-side display screen and a right-side display screen inside the vehicle. The left-side display screen is located to the left of the steering wheel inside the vehicle, and the right-side display screen is located to the right of the steering wheel inside the vehicle. The left-side display screen is used to display an image of the left side of the vehicle's exterior, and the right-side display screen is used to display an image of the right side of the vehicle's exterior.

[0057] The display system works as follows: the exterior camera 203 and the interior camera 202 respectively handle signal input and conversion from several cameras inside and outside the vehicle. The digital image signals captured by the cameras are transmitted to the SOC module 201 for processing. The SOC module 201 includes a CPU module, a GPU module, and an NPU module. The GPU module receives the digital image signals input from the exterior camera 203, converts the processed image signals, and outputs them to the left and right displays respectively. For example, the image of the left side of the vehicle captured by the left exterior camera is transmitted to the left display, and the image of the right side of the vehicle captured by the right exterior camera is transmitted to the right display. The NPU module receives the input signals from the interior camera 202, runs face recognition and pupil recognition algorithms, and outputs the recognition content and results to the CPU module. The recognition content and results include pupil position, pupil movement trajectory, and direction. The CPU module receives the recognition content and results from the NPU module, runs a control strategy based on the recognition content and results, and controls the GPU module to output the processed image to the display screen 204. The CPU module also outputs identity information to the MCU module. The MCU module handles the interaction between the SOC module 201 and the body controller 205, sending identification information to the body controller 205 for relevant feature settings. The MCU module also receives signals from the body controller 205, receiving some vehicle driving information such as reversing, left turn, and right turn. The SOC module 201 controls the output of the entire vehicle's electronic rearview mirror display. The in-vehicle display screen 204 is the display terminal of the entire system, displaying the signals processed by the SOC module 201 on the screen in real time.

[0058] First, the SOC module 201 receives input signals from the left and right cameras outside the vehicle and outputs the left and right images to the two left and right displays inside the vehicle, respectively, so that the left and right displays can display them in real time. The in-vehicle camera 202 and the external camera 203 are activated simultaneously to detect the driver's eyes. They actively emit near-infrared light towards the face to obtain the face image and transmit the infrared image to the NPU module of the SOC module 201. The NPU runs face recognition, pupil recognition, and fatigue warning algorithms to detect the driver's face, perform identity authentication, and send the authentication result to the body controller 205; it also detects the position of the driver's pupils, judges the up, down, left, and right movement direction of the pupils, detects the driver's driving status, and provides fatigue warning.

[0059] The display system in this embodiment is compatible with drivers with different viewing habits, greatly expanding the flexibility of this function.

[0060] Below, in conjunction with Figures 1-2 The following details the specific implementation steps of the display method for the vehicle-mounted electronic rearview mirror provided in this embodiment:

[0061] When a driver uses the display system of the vehicle electronic rearview mirror of this embodiment for the first time, they need to register and calibrate with the display system. The registration and calibration process is as follows: the driver needs to first write their own identity information, then adjust the driver's seat height and fore-aft position to a suitable position, fasten the seat belt, look ahead, and then determine the relative position of the driver's face and pupils in the image captured by the in-vehicle camera 202.

[0062] During the driver's seat adjustment process, the SOC module 201 of the display system pre-sets a range of relative head movement values ​​for the driver. This range is set according to actual needs. If the driver's current relative head movement value is outside this set range, it is determined that the driver is still adjusting their position and is not ready for registration or calibration. If the driver's current relative head movement value is within this set range, it is determined that the driver's seat position has been adjusted and they are ready for registration and calibration. At this time, the SOC module 201 sends a ready signal to the vehicle's CAN bus. The body controller 205 receives this information and issues a voice prompt, such as "Please maintain your posture and look straight ahead." Meanwhile, the in-vehicle camera 202 sends the facial image to the SOC module 201 of the display system in real time. The SOC module 201 runs the facial recognition algorithm and / or pupil recognition algorithm, records facial identity information and / or pupil information, and records the driver's initial facial and pupil positions. Subsequently, based on this, the movement state of the driver's face and pupils is judged to determine the driver's identity information and fatigue state. Then, the field of view of the external camera 203 and the display content of the in-vehicle display screen 204 are adjusted according to these movement states.

[0063] After calibration, the NPU module of SOC module 201 builds models of the face and pupils and maps them to a virtual plane coordinate system to obtain a preset pupil coordinate system. For example... Figure 3As shown, within the preset pupil coordinate system, which is an XY-axis coordinate system, the origin is the midpoint of the line of sight when the pupil is facing forward. The driver's face and pupils have been positioned within this coordinate system after calibration. In this coordinate system, the positions and ranges of the left and right displays of the in-vehicle electronic rearview mirror system observed by the driver, as well as the detection range of the fatigue warning function, are mapped to corresponding areas. Specifically, the dotted pattern area of ​​the preset pupil coordinate system is the fatigue warning area; the lower left grid of the fatigue warning area represents the position and display range of the left display screen, denoted as the left electronic rearview mirror display area; and the lower right grid of the fatigue warning area represents the position and display range of the right display screen, denoted as the right electronic rearview mirror display area.

[0064] The next time the driver uses the vehicle, these internal parameters will be directly accessed based on the driver's identity information. When the driver's face and pupils move, the camera will detect their positions in real time. The recognition algorithm uses the real-time reported pupil positions, along with the calibrated initial face and pupil positions of the driver, to pinpoint the location of the gaze point in this coordinate system. It also calculates the driver's gaze trajectory and direction, i.e., the pupil movement trajectory and direction, to determine the vehicle's operating mode and adjust the image orientation of the external camera 203, thereby adjusting the display content of the in-vehicle display screen 204 in real time.

[0065] When the driver uses the display system of the vehicle electronic rearview mirror of this embodiment again, after the vehicle is started and before executing step S101, the driver's pupil information and facial information are first obtained through the in-vehicle camera 202 to authenticate the driver's identity. If the pupil information matches the pre-stored pupil information, and / or the facial information matches the pre-stored facial information, the driver's seat is adjusted to the pre-stored driver's seat position, and the display system of this embodiment is turned on to realize the automatic adjustment function of the display system, making the display system convenient to operate and intelligently user-friendly. If the pupil information does not match the pre-stored pupil information, and / or the facial information does not match the pre-stored facial information, the display system of this embodiment is turned off.

[0066] After the driver authenticates his identity through the display system, step S101 is executed: after the vehicle is started, the driver's pupil position and the vehicle's external condition image are obtained.

[0067] Specifically, the vehicle uses a left-side exterior camera to acquire real-time images of the left side of the vehicle, a right-side exterior camera to acquire real-time images of the right side of the vehicle, and an interior camera 202 to acquire real-time images of the driver's pupil position. Since both the exterior and interior cameras 203 and 202 use 1080p resolution, the images acquired by these cameras are all 1080p resolution images, providing a basis for subsequent image processing and display by the SOC module 201.

[0068] Next, step S102 is executed. If the pupil position is located in the electronic rearview mirror display area of ​​the preset pupil coordinate system, and the detected pupil position continues to move in the electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode.

[0069] Specifically, if the pupil position is located in the left electronic rearview mirror display area (i.e., the pupil position is within the preset pupil coordinate system's left electronic rearview mirror display area), and the pupil position is detected to be continuously moving within the left electronic rearview mirror display area, it indicates that the driver intends to change lanes to the left or turn to the left. Therefore, the vehicle is determined to be in lane-changing mode, specifically the left-side lane-changing mode. Alternatively, if the pupil position is located in the right electronic rearview mirror display area (i.e., the pupil position is within the preset pupil coordinate system's right electronic rearview mirror display area), and the pupil position is detected to be continuously moving within the right electronic rearview mirror display area, it indicates that the driver intends to change lanes to the right or turn to the right. Therefore, the vehicle is determined to be in lane-changing mode, specifically the right-side lane-changing mode. Lane-changing modes include left-side lane-changing mode and right-side lane-changing mode.

[0070] It should also be noted that the process of detecting continuous pupil movement within the left rearview mirror display area or continuous pupil movement within the right rearview mirror display area is determined by the NPU module. If the frequency of detecting the pupil position within a preset detection time is not less than the preset pupil position landing point frequency, then it is determined that the pupil position is continuously moving within the rearview mirror display area. Furthermore, the pupil movement trajectory and direction can be formed based on the pupil position landing point (i.e., the pupil line of sight) within this preset detection time. The preset detection time and preset pupil position landing point frequency can be set according to actual needs. For example, within a preset detection time of 3 seconds, the driver's pupil position is detected every 50 milliseconds (ms), and the frequency of the pupil position within that 3 seconds is counted. Assuming the frequency of the pupil position being within the left rearview mirror display area is 50 times, which is greater than the preset pupil position landing point frequency of 30 times, then it is determined that the pupil position is continuously moving within the left rearview mirror display area. The pupil movement trajectory and direction can also be obtained based on the pupil position within that 3 seconds.

[0071] If the frequency of the area where the pupil is located is less than the frequency of the preset pupil position landing point within the preset detection time, it is determined that the pupil position is not continuously moving within the electronic rearview mirror display area, indicating that the driver is only observing the image of the vehicle outside the vehicle.

[0072] In lane change mode, step S103 is executed to crop the external vehicle condition image to obtain a cropped external vehicle condition image, and the cropped external vehicle condition image is displayed on the vehicle's corresponding display screen 204. The size of the image displayed on the corresponding display screen 204 is smaller than the size of the external vehicle condition image.

[0073] Specifically, in the left lane change mode of the lane change mode, the left side of the vehicle exterior image is cropped to obtain the cropped left side of the vehicle exterior image, and the cropped left side of the vehicle exterior image is displayed on the left display screen inside the vehicle, wherein the corresponding display screen 204 includes the left side display screen.

[0074] It should be explained that because the exterior camera 203 uses a 1080p resolution screen, while the interior display screen 204 uses a 720p resolution screen, the size of the external vehicle condition image captured by the exterior camera 203 is larger than the size of the image displayed on the display screen 204. For example... Figure 4 As shown, the pixel size of the image of the vehicle's exterior is 1920×1080, and the pixel size of the image displayed on screen 204 is 1280×720.

[0075] Assuming Figure 4 Taking the left-side external traffic image as an example, in straight-line driving mode, the center point of the display frame on the left-side display coincides with the center point of the left-side external traffic image. The left-side display shows the image corresponding to the display frame, which is a 1280×720 external traffic image with the center points of both coinciding. In the left-side lane-changing mode, the NPU module obtains information such as pupil position, pupil movement trajectory, and direction, and transmits this information to the CPU module. The CPU module controls the GPU module to crop the left-side external traffic image. Specifically, the cropping process involves moving the left-side display frame to the lower left of the left-side external traffic image, cropping the image corresponding to the display frame, and then displaying the cropped left-side external traffic image on the left-side display inside the vehicle.

[0076] In the right lane change mode of the lane change mode, the right-side vehicle condition image of the vehicle exterior is cropped to obtain the cropped right-side vehicle condition image, and the cropped right-side vehicle condition image is displayed on the right-side display screen inside the vehicle, wherein the corresponding display screen 204 includes the right-side display screen.

[0077] Therefore, the content displayed on the in-vehicle display screen 204 can be adjusted synchronously in real time according to the driver's line of sight, completing the tracking of the driver's observation range, greatly adapting to different drivers' observation habits, improving the convenience of use for the driver, and ensuring the driving safety of the vehicle.

[0078] After acquiring the driver's pupil position and the vehicle's external conditions, if the pupil position is located in the fatigue warning area of ​​the preset pupil coordinate system, the vehicle is determined to be in straight-line driving mode.

[0079] In straight-line driving mode, the external vehicle condition image is directly displayed on the corresponding display screen 204, and the driver's facial image is continuously acquired. The specific steps for directly displaying the external vehicle condition image on the corresponding display screen 204 are as follows: assuming... Figure 4 Taking the image of the vehicle's exterior left side as an example, when the vehicle is in straight-line driving mode, the center point of the display frame on the left side of the display screen coincides with the center point of the image of the vehicle's exterior left side. The left side of the display screen displays the image corresponding to the display frame on the left side of the display screen. The image corresponding to the display frame on the left side of the display screen is an image of the vehicle's exterior with a size of 1280×720 under the condition that the center points of the two coincide.

[0080] Furthermore, during the continuous acquisition of the driver's facial images, if the frequency of detecting the absence of the driver's pupil information in the facial images within a preset time period is not less than a preset detection frequency, it indicates that the driver is frequently opening and closing their eyes, exhibiting symptoms of drowsiness and frequent yawning, indicating a risk of driver fatigue. In this case, fatigue warning information is sent to the vehicle's body controller 205. The body controller 205 then determines the driver's current fatigue warning level based on the detection frequency of the fatigue warning information and outputs vehicle warning information accordingly. The preset time and preset detection frequency can both be set according to actual needs.

[0081] Specifically, in straight-line driving mode, during the continuous acquisition of the driver's facial image, if the driver's pupil information is frequently not detected within a preset time period (i.e., the frequency of detections where the driver's pupil information is not present in the facial image is not less than a preset detection frequency), it indicates that the driver is frequently opening and closing their eyes, thus determining that the driver is in a state of fatigue driving. The SOC module 201 will send fatigue warning information to the body controller 205. The body controller 205 receives the fatigue warning information, determines the driver's current fatigue warning level based on the detection frequency contained in the fatigue warning information, and provides vehicle warning information reminders according to the current fatigue warning level. If the current fatigue warning level is Level 1, the body controller 205 will remind the driver by vibrating the steering wheel and / or playing a fatigue warning voice message. If the current fatigue warning level is Level 2, the body controller 205 will remind the driver by flashing the warning lights and / or playing a louder voice message. If the current fatigue warning level is Level 3, the body controller 205 will limit the vehicle's speed or directly control the vehicle to stop, forcing the driver to rest.

[0082] In straight-line driving mode, during the continuous acquisition of the driver's face image, if the driver's pupil information is not frequently not detected within a preset time, that is, if the detection frequency of the absence of the driver's pupil information in the face image is less than the preset detection frequency, it indicates that the driver is driving normally, and the driver's pupil position is continuously acquired.

[0083] After acquiring the driver's pupil position and the vehicle's external traffic conditions, if the pupil position is not within the preset pupil coordinate system, it indicates that the driver is engaging in dangerous driving behavior, such as smoking, and the vehicle is determined to be in dangerous driving mode. In dangerous driving mode, dangerous driving information is sent to the body controller 205, so that the body controller 205 outputs vehicle alarm information based on the dangerous driving information.

[0084] In this embodiment, the display method not only accommodates drivers with different viewing habits and tracks the driver's field of vision, but also enables the content displayed on the in-vehicle display screen 204 to be synchronously adjusted in real time according to the driver's line of sight, improving the driver's ease of use and ensuring driving safety. Furthermore, it integrates the in-vehicle fatigue warning system method into a single system, improving overall vehicle integration, saving resources, reducing manufacturing costs, and greatly demonstrating the flexibility of the display system.

[0085] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:

[0086] In this embodiment of the invention, after the vehicle is started, the driver's pupil position and an image of the external traffic conditions are acquired to lay a solid foundation for the subsequent adjustment and display of the external traffic conditions image. If the pupil position is located within the electronic rearview mirror display area of ​​a preset pupil coordinate system, and the pupil position is detected to continuously move within the electronic rearview mirror display area, the vehicle is determined to be in lane-changing mode. In lane-changing mode, the external traffic conditions image is cropped to obtain a cropped external traffic conditions image, and the cropped external traffic conditions image is displayed on the vehicle's corresponding display screen, wherein the size of the image displayed on the corresponding display screen is smaller than the size of the external traffic conditions image.

[0087] Thus, through the display method of this invention, the content displayed on the in-vehicle display screen can be adjusted synchronously in real time according to the driver's line of sight, completing the tracking of the driver's observation range, greatly adapting to different drivers' observation habits, improving the convenience of use for drivers, and ensuring vehicle driving safety.

[0088] Example 2

[0089] Based on the same inventive concept, the second embodiment of the present invention also provides a display system for a vehicle-mounted electronic rearview mirror, which operates on the display method of the vehicle-mounted electronic rearview mirror as described in Embodiment 1, such as... Figure 2 As shown, the system includes: a SOC module 201, and an in-vehicle camera 202, an external camera 203, and a display screen 204 connected to the SOC module 201. The in-vehicle camera 202 is located directly in front of the driver's seat of the vehicle, and the external camera 203 is located outside the vehicle body. The resolution of the display screen 204 is lower than that of the external camera 203, and the external camera 203 transmits images digitally.

[0090] The in-vehicle camera 202 is used to acquire the pupil position of the driver of the vehicle after the vehicle is started.

[0091] The external camera 203 is used to acquire images of the vehicle's external conditions after the vehicle is started.

[0092] The SOC module 201 is used to determine that the vehicle is in lane-changing mode if the pupil position is located in the electronic rearview mirror display area of ​​the preset pupil coordinate system and the pupil position continues to move in the electronic rearview mirror display area; in the lane-changing mode, the external vehicle condition image is cropped to obtain a cropped external vehicle condition image, and the cropped external vehicle condition image is displayed on the display screen 204 corresponding to the vehicle, wherein the size of the image displayed on the corresponding display screen 204 is smaller than the size of the external vehicle condition image.

[0093] As an optional embodiment, the exterior camera 203 includes a left-side exterior camera and a right-side exterior camera, with the left-side exterior camera located on the left side of the vehicle body and the right-side exterior camera located on the right side of the vehicle body.

[0094] The external left-side camera is used to acquire the external vehicle condition image of the vehicle's external left-side condition.

[0095] The external right-side camera is used to acquire images of the external vehicle conditions on the right side of the vehicle.

[0096] The display screen 204 includes a left-side display screen and a right-side display screen inside the vehicle. The left-side display screen is located on the left side of the steering wheel inside the vehicle, and the right-side display screen is located on the right side of the steering wheel inside the vehicle.

[0097] The left-side display screen is used to display the image of the vehicle's left-side exterior condition;

[0098] The right-side display screen is used to display images of the vehicle's right-side exterior.

[0099] As an optional embodiment, determining that the vehicle is in lane-changing mode if the pupil position is located within the electronic rearview mirror display area of ​​a preset pupil coordinate system, and the pupil position continuously moves within the electronic rearview mirror display area, includes:

[0100] If the pupil position is located within the left electronic rearview mirror display area of ​​the electronic rearview mirror display area, and the pupil position is detected to be continuously moving within the left electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode; or,

[0101] If the pupil position is located in the right electronic rearview mirror display area of ​​the electronic rearview mirror display area, and the pupil position is detected to be continuously moving in the right electronic rearview mirror display area, then it is determined that the vehicle is in lane change mode.

[0102] As an optional embodiment, in the lane-changing mode, cropping the external vehicle condition image to obtain a cropped external vehicle condition image, and displaying the cropped external vehicle condition image on the vehicle's corresponding display screen 204, includes:

[0103] In the left lane change mode of the lane change mode, the left side of the vehicle exterior image is cropped to obtain a cropped left side vehicle exterior image, and the cropped left side vehicle exterior image is displayed on the left side display screen inside the vehicle, wherein the corresponding display screen 204 includes the left side display screen.

[0104] As an optional embodiment, in the lane-changing mode, cropping the external vehicle condition image to obtain a cropped external vehicle condition image, and displaying the cropped external vehicle condition image on the vehicle's corresponding display screen 204, includes:

[0105] In the right lane change mode of the lane change mode, the right-side vehicle condition image of the vehicle exterior is cropped to obtain a cropped right-side vehicle condition image, and the cropped right-side vehicle condition image is displayed on the right-side display screen inside the vehicle, wherein the corresponding display screen 204 includes the right-side display screen.

[0106] As an optional embodiment, the SOC module 201 is used to: determine that the vehicle is in a straight-line driving mode if the pupil position is located in the fatigue warning area of ​​the preset pupil coordinate system;

[0107] In the straight-line driving mode, the external vehicle condition image is directly displayed on the corresponding display screen 204, and the driver's facial image is continuously captured;

[0108] If, within a preset time period, the frequency of detecting the absence of the driver's pupil information in the face image is not less than a preset detection frequency, then fatigue warning information is sent to the vehicle's body controller 205, so that the body controller 205 determines the driver's current fatigue warning level based on the detection frequency of the fatigue warning information, and then outputs vehicle warning information based on the current fatigue warning level.

[0109] As an optional embodiment, the SOC module 201 is used to: determine that the vehicle is in a dangerous driving mode if the pupil position is not located in the preset pupil coordinate system;

[0110] In the dangerous driving mode, dangerous driving information is sent to the body controller 205 so that the body controller 205 outputs vehicle alarm information based on the dangerous driving information.

[0111] As an optional embodiment, the in-vehicle camera 202 is used to: after the vehicle is started, and before acquiring the pupil position of the driver and the external vehicle condition image of the vehicle, acquire the driver's pupil information and facial information.

[0112] SOC module 201 is used to: if the pupil information is consistent with the pre-stored pupil information, and / or the face information is consistent with the pre-stored face information, then adjust the driver's seat of the vehicle to the pre-stored driver's seat position.

[0113] Since the vehicle-mounted electronic rearview mirror display system described in this embodiment is the system used to implement the vehicle-mounted electronic rearview mirror display method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the vehicle-mounted electronic rearview mirror display system of this embodiment based on the vehicle-mounted electronic rearview mirror display method described in Embodiment 1 of this application. Therefore, how the vehicle-mounted electronic rearview mirror display system implements the method in Embodiment 1 of this application will not be described in detail here. Any system used by those skilled in the art to implement the vehicle-mounted electronic rearview mirror display method in Embodiment 1 of this application falls within the scope of protection of this application.

[0114] Example 3

[0115] Based on the same inventive concept, the third embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the display methods of the vehicle electronic rearview mirror described in the first embodiment above.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0121] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A display method of a vehicle-mounted electronic rearview mirror, characterized by, The method comprises the following steps: After the vehicle starts, the pupil position of the driver of the vehicle and the out-of-vehicle vehicle condition image of the vehicle are acquired; If the pupil position is located in the electronic rearview mirror display area of the preset pupil coordinate system, and it is detected that the pupil position continuously moves in the electronic rearview mirror display area, it is determined that the vehicle is in a lane-changing mode; wherein the preset pupil coordinate system is an X-Y axis coordinate system, the midpoint of the line of sight when the pupil directly faces the front is the coordinate origin, the driver's face and pupil have determined positions in the coordinate system through a calibration action, the middle area of the preset pupil coordinate system is a fatigue warning area, the lower left grid of the fatigue warning area is the position and display range of the left display screen, and is recorded as the left electronic rearview mirror display area; the lower right grid of the fatigue warning area is the position and display range of the right display screen, and is recorded as the right electronic rearview mirror display area; In the lane-changing mode, the out-of-vehicle vehicle condition image is cropped to obtain a cropped out-of-vehicle vehicle condition image, and the cropped out-of-vehicle vehicle condition image is displayed on the corresponding display screen of the vehicle; wherein the size of the image displayed on the corresponding display screen is smaller than the size of the out-of-vehicle vehicle condition image, and the corresponding display screen comprises the left display screen; In the left lane-changing mode of the lane-changing mode, the out-of-vehicle left vehicle condition image of the out-of-vehicle vehicle condition image is cropped to obtain a cropped out-of-vehicle left vehicle condition image, and the cropped out-of-vehicle left vehicle condition image is displayed on the left display screen in the vehicle; wherein the corresponding display screen comprises the left display screen; The cropping of the out-of-vehicle left vehicle condition image of the out-of-vehicle vehicle condition image to obtain the cropped out-of-vehicle left vehicle condition image comprises moving the left display screen display frame to the lower left of the out-of-vehicle left vehicle condition image, and then cropping the image corresponding to the left display screen display frame to obtain the cropped out-of-vehicle left vehicle condition image; When the driver uses the display system of the vehicle-mounted electronic rearview mirror for the first time, the driver needs to register and calibrate the system. The registration and calibration process is as follows: the driver needs to write his / her own identity information first, then the driver adjusts the height and front-back position of the driver's seat to a suitable position, fastens the seat belt, looks at the front, and then determines the relative position of the driver's face and pupil in the initial state in the image collected by the in-vehicle camera; After the calibration is completed, the NPU module of the SOC module establishes a model of the face and pupil, and maps it to a virtual plane coordinate system to obtain a preset pupil coordinate system. The SOC module is a system-on-chip module, and the NPU module is a neural network processor module.

2. The method of claim 1, wherein, If the pupil position is located in the left electronic rearview mirror display area of the electronic rearview mirror display area, and it is detected that the pupil position continuously moves in the left electronic rearview mirror display area, it is determined that the vehicle is in a lane-changing mode; or, ​ If the pupil position is located in a right electronic rearview mirror display area of the electronic rearview mirror display area, and it is detected that the pupil position continuously moves in the right electronic rearview mirror display area, it is determined that the vehicle is in a lane changing mode.

3. The method of claim 1, wherein, In the lane changing mode, the out-of-vehicle vehicle condition image is cropped to obtain a cropped out-of-vehicle vehicle condition image, and the cropped out-of-vehicle vehicle condition image is displayed on the corresponding display screen of the vehicle. In a right lane changing mode of the lane changing mode, an out-of-vehicle right side vehicle condition image of the out-of-vehicle vehicle condition image is cropped to obtain a cropped out-of-vehicle right side vehicle condition image, and the cropped out-of-vehicle right side vehicle condition image is displayed on a right side display screen in the vehicle, wherein the corresponding display screen includes the right side display screen.

4. The method of claim 1, wherein, After obtaining the pupil position of the driver of the vehicle and the out-of-vehicle vehicle condition image of the vehicle, further comprising: If the pupil position is located in a fatigue warning area of the preset pupil coordinate system, it is determined that the vehicle is in a straight driving mode. In the straight driving mode, the out-of-vehicle vehicle condition image is directly displayed on the corresponding display screen, and the face image of the driver is continuously collected. If, within a preset time, it is detected that the detection frequency of the absence of the driver's pupil information in the face image is not less than a preset detection frequency, fatigue warning information is sent to the vehicle body controller to determine the current fatigue warning level of the driver according to the detection frequency of the fatigue warning information, and vehicle warning information is output according to the current fatigue warning level.

5. The method of claim 4, wherein, After obtaining the pupil position of the driver of the vehicle and the out-of-vehicle vehicle condition image of the vehicle, further comprising: If the pupil position is not located in the preset pupil coordinate system, it is determined that the vehicle is in a dangerous driving mode. In the dangerous driving mode, dangerous driving information is sent to the vehicle body controller to output vehicle alarm information according to the dangerous driving information.

6. The method of claim 1, wherein, After the vehicle is started, and before obtaining the pupil position of the driver of the vehicle and the out-of-vehicle vehicle condition image of the vehicle, further comprising: Obtaining the pupil information and face information of the driver; If the pupil information is consistent with the pre-stored pupil information, and / or the face information is consistent with the pre-stored face information, the driver's seat of the vehicle is adjusted to the pre-stored driver's seat position.

7. A display system for a vehicle electronic rearview mirror, characterized in that The display method for the vehicle-mounted electronic rearview mirror according to any one of claims 1-6, the system comprising: a SOC module, and an in-vehicle camera, an out-of-vehicle camera, and a display screen connected to the SOC module, the in-vehicle camera being arranged in front of the driver's seat of the vehicle, and the out-of-vehicle camera being arranged outside the vehicle body, wherein the resolution of the display screen is less than the resolution of the out-of-vehicle camera, and the out-of-vehicle camera transmits images in a digital transmission mode; The in-vehicle camera is used to obtain the pupil position of the driver of the vehicle after the vehicle is started. The out-of-vehicle camera is used to obtain the out-of-vehicle vehicle condition image of the vehicle after the vehicle is started. The SOC module is configured to determine that the vehicle is in a lane-changing mode if the pupil position is located in an electronic rearview mirror display region of a preset pupil coordinate system and the pupil position is detected to continuously move in the electronic rearview mirror display region; in the lane-changing mode, the out-of-vehicle vehicle condition image is cropped to obtain a cropped out-of-vehicle vehicle condition image, and the cropped out-of-vehicle vehicle condition image is displayed on a corresponding display screen of the vehicle, wherein a size of an image displayed on the corresponding display screen is smaller than a size of the out-of-vehicle vehicle condition image.

8. The system of claim 7, wherein, The out-of-vehicle camera includes an out-of-vehicle left camera and an out-of-vehicle right camera, the out-of-vehicle left camera is arranged on a left side of the vehicle body, and the out-of-vehicle right camera is arranged on a right side of the vehicle body. The out-of-vehicle left camera is configured to acquire an out-of-vehicle left vehicle condition image of the out-of-vehicle vehicle condition image. The out-of-vehicle right camera is configured to acquire an out-of-vehicle right vehicle condition image of the out-of-vehicle vehicle condition image. The display screen includes a left display screen and a right display screen in the vehicle, the left display screen is arranged on a left side of a steering wheel in the vehicle, and the right display screen is arranged on a right side of the steering wheel in the vehicle. The left display screen is configured to display the out-of-vehicle left vehicle condition image. The right display screen is configured to display the out-of-vehicle right vehicle condition image.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the method steps of any one of claims 1-6.

Images