Electronic rearview mirror system and electronic rearview mirror display host

Abstract

The present invention belongs to the technical field of electronic rearview mirrors, and specifically relates to an electronic rearview mirror system, an electronic rearview mirror image jam detection method and a device. During the operation of the electronic rearview mirror system, the method selects at least one state from the video data stream state, the connection state of the video transmission link, and the working state of the image display module to determine whether the electronic rearview mirror image is jammed, and the selected state at least includes the video data stream state. When determining the video data stream state, different time nodes of the video stream are extracted, and the differences between the extracted image data are compared. Based on the differences, it is possible to accurately determine whether the video data stream is normal. Moreover, during the comparison process, the selected area is generally not the complete image data, which can meet the judgment accuracy while reducing the amount of calculation. When selecting three states for judgment, it is possible to detect whether the video data stream is normal from the entire video transmission link, thereby reducing the occurrence of potential accidents.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic rearview mirrors, and in particular relates to an electronic rearview mirror system and an electronic rearview mirror display host. Background Art

[0002] An electronic rearview mirror, also known as a CMS, streaming media rearview mirror, or indirect vision device, is an electronic device that captures images from a camera mounted in a specific location on the vehicle and displays them on a display screen. The camera and the display unit responsible for image processing are typically separate components: the camera is mounted on the exterior of the vehicle, while the display unit is installed in a convenient location inside the cab. Depending on the camera's mounting position, an electronic rearview mirror can replace a traditional glass rearview mirror to help the driver observe the vehicle's surroundings, or serve as an auxiliary device, allowing the driver to observe areas not visible with traditional glass rearview mirrors, eliminating blind spots.

[0003] According to the composition of the product, the failure of electronic rearview mirrors can be divided into three categories: image acquisition device (camera) failure, video transmission failure and display host image processing and display failure. As an on-board electronic device related to driving safety, when a safety-related function fails, the driver should be informed of the device's failure status through warning signals or information prompts.

[0004] When a malfunction occurs in the image acquisition device (camera) or video transmission link, the display host, without effective detection and judgment, will continue to display the last frame of image data retained in memory, failing to reflect real-time changes in the vehicle's surroundings. This is known as image freeze (or image lag). Image freeze creates a false impression for the driver, preventing them from distinguishing between a normal image and a stuck image. This can lead to serious traffic accidents when obstacles or pedestrians appear near the vehicle. Therefore, for electronic rearview mirror products, image freeze is generally considered more serious than other types of failures in terms of its consequences. To mitigate the driving safety risks caused by image freeze, the electronic rearview mirror host must quickly and accurately identify the occurrence of image freeze and notify the driver of the fault through the display, indicator lights, or sound. Summary of the Invention

[0005] The object of the present invention is to provide an electronic rearview mirror system and an electronic rearview mirror display host, so as to solve the problem that the electronic rearview mirror image is stuck and easily causes safety hazards.

[0006] In order to solve the above technical problems, the present invention provides an electronic rearview mirror display host, which includes: a main control processing unit, an image decoding and receiving module, an image capture module, a data buffer module, an image display module and an image data sampling module;

[0007] The image decoding and receiving module is configured to decode image data in an encoded format transmitted by the camera and output the decoded image data to the image capture module; the image capture module is configured to output the received image data to the data buffer module; the image display module is configured to display and output the image data in the data buffer module and output status feedback information to the main control processing unit; the image data sampling module is configured to simultaneously extract and store one frame of image data from at least two of the image capture module, the data buffer module, and the image display module; the main control processing unit is configured to determine the connection status of the video transmission link by detecting the status parameters of the image decoding and receiving module during operation of the electronic rearview mirror system; if the connection status of the video transmission link is normal, the main control processing unit sends a data sampling trigger instruction to the image data sampling module; the main control processing unit detects the status feedback information of the image display module; if the image display module is operating normally, the main control processing unit compares the differences in the same area of ​​each frame of image data; if the differences in the same area of ​​all frames of image data are less than a set requirement, it indicates that the video data stream is abnormal and it is determined that the electronic rearview mirror image is stuck; otherwise, it indicates that the video data stream is normal and it is determined that the electronic rearview mirror image is not stuck.

[0008] The beneficial effects are as follows: the present invention selects at least one state from the video data stream state, the connection state of the video transmission link, and the working state of the image display module for each module included in the electronic rearview mirror system to determine whether the electronic rearview mirror image is stuck, and the selected state at least includes the video data stream state. When determining the video data stream state, different time nodes of the video stream are extracted, and the differences between the extracted image data are compared. Based on the difference results, it is possible to accurately determine whether the video data stream is normal. Moreover, during the comparison process, the selected area is generally not the complete image data, which can meet the judgment accuracy while reducing the amount of calculation. In addition, when fewer states are selected for judgment, a higher processing speed can be guaranteed to promptly detect the stuck electronic rearview mirror image; when three states are selected for judgment, it is possible to detect whether the video data stream is normal from the entire video transmission link, reducing the occurrence of potential accidents.

[0009] Furthermore, in step 1) determining the video data stream status, the difference includes regional color difference and / or brightness difference, and the corresponding difference is less than the set requirement as follows: if only color is included, then if the color difference of the same area of ​​each frame image data is less than the set color difference threshold, then it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if only brightness is included, then if the brightness difference of the same area of ​​each frame image data is less than the set brightness difference threshold, then it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if color and brightness are included, then if the color difference of the same area of ​​each frame image data is less than the set color difference threshold and the brightness difference is less than the set brightness difference threshold, then it indicates that the difference of the same area of ​​all frame image data is less than the set requirement.

[0010] Its beneficial effect is: judging whether the status of the video data stream is normal from the changes in color and / or brightness. This is because even if the object photographed by the electronic rearview mirror does not change, the light will change over time. The change in light will cause the color and / or brightness of the image data to change, thereby accurately judging whether the video data stream is normal.

[0011] Furthermore, in the process of determining the video data stream state in step 1), the same area is an area close to the middle position of the entire image data.

[0012] The beneficial effect is that the middle area of ​​the electronic rearview mirror generally covers the main target of the shooting, and the main target generally changes. In the face of this phenomenon, the area close to the middle position of the entire image data is selected. This area is the key area, thereby achieving the best balance between data calculation amount and reliability.

[0013] Furthermore, the same area is: an area occupying 1 / 3 to 1 / 2 of the entire image data in the horizontal direction and an area from the center of the image data downward in the vertical direction.

[0014] The beneficial effect is that the selected area is the area with frequent changes, and the areas with infrequent pixel changes such as the vehicle body and the sky are avoided, which can improve the detection effect.

[0015] In order to solve the above technical problems, the present invention also provides an electronic rearview mirror system, including a camera and a host, the host including an image decoding and receiving module, an image capture module, a data buffer module, an image display module, an image data sampling module and a main control processing unit; the image decoding and receiving module is used to decode the image data in the coded format transmitted by the camera and output it to the image capture module; the image capture module is used to output the received image data to the data buffer module; the image display module is used to display and output the image data in the data buffer module; the image data sampling module is used to simultaneously extract a frame of image data from at least two modules among the image capture module, the data buffer module and the image display module The main control processing unit is used to determine the connection status of the video transmission link by detecting the status parameters of the image decoding receiving module during the operation of the electronic rearview mirror system. If the connection status of the video transmission link is normal, a data sampling trigger instruction is sent to the image data sampling module; the main control processing unit detects the status feedback information of the image display module. If the image display module is in normal working condition, the main control processing unit compares the differences of the same areas of the extracted image data of each frame. If the differences of the same areas of all the frame image data are less than the set requirements, it indicates that the video data stream is abnormal, and it is determined that the electronic rearview mirror image is stuck. Otherwise, it indicates that the video data stream is normal, and it is determined that the electronic rearview mirror image is not stuck.

[0016] The beneficial effects are as follows: the present invention selects at least one state from the video data stream state, the connection state of the video transmission link, and the working state of the image display module for each module included in the electronic rearview mirror system to determine whether the electronic rearview mirror image is stuck, and the selected state at least includes the video data stream state. When determining the video data stream state, different time nodes of the video stream are extracted, and the differences between the extracted image data are compared. Based on the difference results, it is possible to accurately determine whether the video data stream is normal. Moreover, during the comparison process, the selected area is generally not the complete image data, which can meet the judgment accuracy while reducing the amount of calculation. In addition, when fewer states are selected for judgment, a higher processing speed can be guaranteed to promptly detect the stuck electronic rearview mirror image; when three states are selected for judgment, it is possible to detect whether the video data stream is normal from the entire video transmission link, reducing the occurrence of potential accidents.

[0017] Furthermore, in the process of determining the video data stream status, the main control processing unit includes regional color difference and / or brightness difference, and the corresponding difference is less than the set requirement: if only color is included, if the color difference of the same area of ​​each frame image data is less than the set color difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if only brightness is included, if the brightness difference of the same area of ​​each frame image data is less than the set brightness difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if color and brightness are included, if the color difference of the same area of ​​each frame image data is less than the set color difference threshold and the brightness difference is less than the set brightness difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement.

[0018] Its beneficial effect is: judging whether the status of the video data stream is normal from the changes in color and / or brightness. This is because even if the object photographed by the electronic rearview mirror does not change, the light will change over time. The change in light will cause the color and / or brightness of the image data to change, thereby accurately judging whether the video data stream is normal.

[0019] Furthermore, in the process of the main control processing unit determining the video data stream state, the same area is an area close to the middle position of the entire image data.

[0020] The beneficial effect is that the middle area of ​​the electronic rearview mirror generally covers the main target of the shooting, and the main target generally changes. In the face of this phenomenon, the area close to the middle position of the entire image data is selected. This area is the key area, thereby achieving the best balance between data calculation amount and reliability.

[0021] Furthermore, the same area is: an area occupying 1 / 3 to 1 / 2 of the entire image data in the horizontal direction and an area from the center of the image data downward in the vertical direction.

[0022] The beneficial effect is that the selected area is the area with frequent changes, and the areas with infrequent pixel changes such as the vehicle body and the sky are avoided, which can improve the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a structural diagram of the electronic rearview mirror system of the present invention;

[0024] Figure 2 is a flow chart of a method for detecting image stuckness in an electronic rearview mirror according to the present invention;

[0025] Figure 3 It is a structural diagram of the electronic rearview mirror image jam detection device of the present invention. DETAILED DESCRIPTION

[0026] During the operation of the electronic rearview mirror, the present invention acquires at least one of the following: the video data stream status, the connection status of the video transmission link, and the operating status of the image display module. The acquired status includes at least the video data stream status. Based on the acquired status, it is determined whether the electronic rearview mirror is stuck (or frozen). The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0027] Electronic rearview mirror system embodiment:

[0028] The electronic rearview mirror system embodiment of the present invention is mainly aimed at a vehicle-mounted electronic rearview mirror, and its structure is as follows Figure 1 As shown, the electronic rearview mirror includes a camera and a host. The camera includes an image sensing module and an image encoding and transmitting module. The host includes an image decoding and receiving module, an image capture module, an image display module, a data buffer module, and a main control processing unit. In addition, the host also includes two image data sampling modules: a first image data sampling module and a second image data sampling module.

[0029] The image sensing module is responsible for the acquisition and signal conversion of video image data and is the original source of video image data.

[0030] The image coding and sending module is responsible for encoding the image data output by the image sensor module according to a specific coding format and transmitting the signal through a transmission medium.

[0031] The image decoding and receiving module receives image data in a specific coding format through a transmission medium, decodes the image data, and converts the image data into a data format that can be recognized by the image capturing module.

[0032] The image capture module is responsible for receiving the image data output by the image decoding and receiving module, and outputting the final image data to the data buffer through the data bus.

[0033] The image display module is responsible for displaying and outputting image data and outputting status feedback information to the main control processing unit.

[0034] The first image data sampling module, based on a trigger instruction from the main control processing unit, backs up a specified area aera (x, y, w, h) of a frame of video image data from the image capture module to a designated data buffer. Here, x and y represent the coordinates of the backup data's starting point, w represents the backup data's pixel width, and h represents the backup data's pixel height.

[0035] The second image data sampling module backs up the specified area aera (x, y, w, h) of the current frame video image data of the data buffer module to the specified data buffer according to the trigger instruction of the main control processing unit.

[0036] The main control processing unit is responsible for the status detection of the video image transmission link, the sampling trigger control of the image data sampling module, the verification calculation and comparison of the sampling data, and the status feedback detection of the image display module, so as to determine whether the electronic image is stuck.

[0037] Based on the electronic rearview mirror system described above, an electronic rearview mirror image stuck detection method of the present invention can be implemented, which is described in detail below.

[0038] S1, during the operation of the electronic rearview mirror system, the main control processing unit determines the connection status of the video transmission link by detecting the status parameters of the image decoding and receiving module to determine whether the camera is connected normally. If the video transmission link status is normal, it indicates that the camera image data can be received, and then S2 is executed. Otherwise, it indicates that the video transmission link status is abnormal, and then S7 is executed.

[0039] S2, the main control processing unit sends a data sampling trigger instruction to the first image data sampling module and the second image data sampling module at the same time, and then executes S3 and S4.

[0040] In step S3, after receiving the data sampling trigger instruction from the main control processing unit, the first image data sampling module backs up the designated area aera (x, y, w, h) of the current video frame of the image capture module to data buffer A, and then executes step S5. The designated area here is preferably near the center of the entire image data, because the image data content in the entire middle area usually changes. In addition, the designated area can start at 1 / 3 of the entire image horizontally and have a width of approximately 1 / 3 to 1 / 2 of the entire image width. Vertically, it can be located 1 / 2 of the area below the center of the image. This area is typically where the image changes frequently, which can reduce the amount of computation while avoiding areas where pixels do not change frequently, such as the vehicle body and sky.

[0041] S4: After receiving the data sampling trigger instruction from the main control processing unit, the second image data sampling module backs up the specified area aera (x, y, w, h) of the current frame video image data in the data buffer module to data buffer B, and then executes S5. The values ​​of x, y, w, h in this step are the same as those in step S3.

[0042] S5, the main control processing unit detects the status feedback information of the image display module. If the image display module is in a normal working state, execute S6, otherwise execute S8.

[0043] In step S6, the main control processing unit compares the color and brightness differences of the two image data in data buffer A and data buffer B. Since data buffer A and data buffer B are taken from different time nodes of the video stream, if the color and brightness differences between the two are less than the corresponding set difference threshold Δp, it indicates that the video data stream is abnormal and S7 is executed. Otherwise, the video data stream is normal and the entire process ends.

[0044] This step can be performed pixel by pixel. For example, when comparing color, the colors of pixels (in RGBA or YUV422 format) at the phase positions of the two image data are compared to determine if they are identical. The number of pixels with the same color is then accumulated. After the comparison is complete, the accumulated number is divided by the total number of pixels in the entire image data to obtain a percentage. A threshold is set. If the percentage is less than the threshold, it indicates that the color difference between the two image data is less than the set color difference threshold. The same process can be used to compare brightness.

[0045] S7, the main control processing unit displays a corresponding warning symbol or other sound and light alarm on the display screen according to the type of fault, and prompts the driver of the fault.

[0046] S8, the main control processing unit notifies the driver of the fault on the display screen through an audible and visual alarm, and the entire process ends.

[0047] In this embodiment, two image data sampling modules are provided. The two image data sampling modules extract one frame of image data from the image capture module and the data buffer module respectively according to the trigger instruction of the main control processing unit and back up the data to the designated data buffer. Correspondingly, in step S6, the image data extracted from the image capture module and the data buffer module are compared for differences. As other implementation methods, the following methods can also be adopted: Method ①, the two image data sampling modules extract one frame of image data from the data buffer module and the image display module respectively and back up the data to the designated data buffer. Correspondingly, in step S6, the image data extracted from the data buffer module and the image display module are compared. Method ②, the two image data sampling modules extract one frame of image data from the image capture module and the image display module respectively and back up the data to the designated data buffer. Correspondingly, in step S6, the image data extracted from the image capture module and the image display module are compared. Method ③, setting up another image data sampling module, that is, setting up three image data sampling modules in total. These three image data sampling modules respectively extract one frame of image data from the image capture module, the data buffer module and the image display module and back them up to the designated data buffer. Correspondingly, the image data extracted from the image capture module, the data buffer module and the image display module are compared in step S6.

[0048] In step S6 of this embodiment, when comparing differences, the comparison is performed on two factors, color and brightness. In other implementations, the comparison may be performed on only one factor, that is, only color or only brightness.

[0049] In this embodiment, when judging whether the electronic rearview mirror image is stuck as a whole, three states are judged: the video data stream state, the connection state of the video transmission link, and the working state of the image display module. The judgment order is the connection state of the video transmission link, the working state of the image display module, and the video data stream state. As other embodiments, when judging all three states, the judgment order can be changed arbitrarily, and the corresponding effect can be achieved. Moreover, in order to improve computational efficiency, only one or two states can be selected for judgment. However, no matter how many are selected, the video data stream state needs to be judged. Moreover, when selecting two states for judgment, for example, the video data stream state and the connection state of the video transmission link, the electronic rearview mirror image is considered to be free of stuck only when both states are normal.

[0050] In summary, the system and the corresponding implementation method have the following characteristics:

[0051] 1) The image data sampling module of the present invention collects the image data of the specific area to be verified into the backup area. When performing data verification calculation, it does not affect the real-time performance of normal image display, thus meeting the real-time performance requirement of the electronic rearview mirror image display.

[0052] 2) The image jam detection method applied to the electronic rearview mirror system of the present invention covers the detection of the image coding transmission link, data verification of the video data stream, and fault status detection of the display processing module. It can detect whether the video data stream is normal from the entire video transmission link, thereby reducing the occurrence of potential accidents.

[0053] 3) The image stuck detection method applied to the electronic rearview mirror system of the present invention can specify the area for image verification and can verify only the key area, thereby achieving the best balance between data calculation amount and reliability.

[0054] Embodiment of the electronic rearview mirror image jam detection device:

[0055] An embodiment of an electronic rearview mirror image stuck detection device of the present invention has a structure as follows Figure 3As shown, the electronic rearview mirror system includes a memory, a processor, and an internal bus. The processor and memory communicate and exchange data with each other via the internal bus. The memory includes at least one software function module stored in the memory. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the electronic rearview mirror image stuck detection method described in the electronic rearview mirror system embodiment of the present invention.

[0056] The processor may be a microprocessor (MCU), a programmable logic device (FPGA), or other processing device. The memory may be any type of memory that stores information electrically, such as RAM and ROM; any type of memory that stores information magnetically, such as hard disks, floppy disks, magnetic tapes, magnetic core memory, bubble memory, and USB flash drives; any type of memory that stores information optically, such as CDs and DVDs; or any other type of memory, such as quantum memory and graphene memory.

[0057] Example of a method for detecting image jamming in an electronic rearview mirror:

[0058] An embodiment of the electronic rearview mirror image stuck detection method of the present invention has the following overall process: Figure 2 As shown, the method for detecting image jamming of the electronic rearview mirror is consistent with that described in the electronic rearview mirror system, and will not be described in detail here.

Claims

1. An electronic rearview mirror display host, characterized in that: The host comprises: a main control processing unit, an image decoding and receiving module, an image capture module, a data buffer module, an image display module and an image data sampling module; the image decoding and receiving module is used to decode the image data in the coded format transmitted by the camera and output it to the image capture module; the image capture module is used to output the received image data to the data buffer module; the image display module is used to display and output the image data in the data buffer module and output status feedback information to the main control processing unit; the image data sampling module is used to simultaneously extract one frame of image data from at least two modules of the image capture module, the data buffer module and the image display module and store the data; The main control processing unit is used to determine the connection status of the video transmission link by detecting the status parameters of the image decoding and receiving module during the operation of the electronic rearview mirror system. If the connection status of the video transmission link is normal, the main control processing unit sends a data sampling trigger instruction to the image data sampling module; the main control processing unit detects the status feedback information of the image display module. If the image display module is in normal working condition, the main control processing unit compares the differences in the same area of ​​each frame image data extracted: if the differences in the same area of ​​all frame image data are less than the set requirements, it indicates that the video data stream is abnormal, and it is determined that the electronic rearview mirror image is stuck; otherwise, it indicates that the video data stream is normal, and it is determined that the electronic rearview mirror image is not stuck.

2. The electronic rearview mirror display host according to claim 1, characterized in that: In the process of determining the state of the video data stream, the difference includes the color difference and / or brightness difference of the area, and the corresponding difference is less than the set requirement: if only color is included, if the color difference of the same area of ​​each frame image data is less than the set color difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if only brightness is included, if the brightness difference of the same area of ​​each frame image data is less than the set brightness difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if color and brightness are included, if the color difference of the same area of ​​each frame image data is less than the set color difference threshold and the brightness difference is less than the set brightness difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement.

3. The electronic rearview mirror display host according to claim 1 or 2, characterized in that: In the process of determining the video data stream state, the same area is an area close to the middle position of the entire image data.

4. The electronic rearview mirror display host according to claim 3, characterized in that: The same area is an area that occupies 1 / 3 to 1 / 2 of the entire image data in the horizontal direction and extends downward from the center of the image data in the vertical direction.

5. An electronic rearview mirror system, comprising a camera and a host, characterized in that: The host includes an image decoding and receiving module, an image capture module, a data buffer module, an image display module, an image data sampling module and a main control processing unit; the image decoding and receiving module is used to decode the image data in the coded format transmitted by the camera and output it to the image capture module; the image capture module is used to output the received image data to the data buffer module; the image display module is used to display and output the image data in the data buffer module; The image data sampling module is used to simultaneously extract one frame of image data from at least two modules of the image capture module, the data buffer module and the image display module and store the data; The main control processing unit is used to determine the connection status of the video transmission link by detecting the status parameters of the image decoding and receiving module during the operation of the electronic rearview mirror system. If the connection status of the video transmission link is normal, a data sampling trigger instruction is sent to the image data sampling module; the main control processing unit detects the status feedback information of the image display module. If the image display module is in normal working condition, the main control processing unit compares the differences in the same area of ​​each frame image data extracted. If the differences in the same area of ​​all frame image data are less than the set requirements, it indicates that the video data stream is abnormal and it is determined that the electronic rearview mirror image is stuck. Otherwise, it indicates that the video data stream is normal and it is determined that the electronic rearview mirror image is not stuck.

6. The electronic rearview mirror system according to claim 5, characterized in that: In the process of determining the video data stream status, the main control processing unit includes the color difference and / or brightness difference of the area, and the corresponding difference is less than the set requirement: if only color is included, if the color difference of the same area of ​​each frame image data is less than the set color difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if only brightness is included, if the brightness difference of the same area of ​​each frame image data is less than the set brightness difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement; if color and brightness are included, if the color difference of the same area of ​​each frame image data is less than the set color difference threshold and the brightness difference is less than the set brightness difference threshold, it indicates that the difference of the same area of ​​all frame image data is less than the set requirement.

7. The electronic rearview mirror system according to claim 5 or 6, characterized in that: In the process of the main control processing unit determining the video data stream state, the same area is an area close to the middle position of the entire image data.

8. The electronic rearview mirror system according to claim 7, characterized in that: The same area is an area that occupies 1 / 3 to 1 / 2 of the entire image data in the horizontal direction and extends downward from the center of the image data in the vertical direction.

Images