Image processing method and device, storage medium, electronic rearview mirror system and vehicle

Abstract

This application relates to an image processing method, apparatus, storage medium, electronic rearview mirror system, and vehicle. The method includes: an electronic rearview mirror control unit acquiring first image data from one or more cameras and sending the first image data to a domain controller. The domain controller receives the first image data sent by the electronic rearview mirror control unit. The electronic rearview mirror control unit processes the first image data to obtain processed image data and outputs the processed image data to a display device. According to embodiments of this application, the electronic rearview mirror function can still operate normally even when the domain controller fails, without relying on the domain controller, ensuring the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Simultaneously, it allows the electronic rearview mirror and the domain controller to share image data acquired by the cameras, saving costs and deployment space.

Description

[0001] This application is a divisional application. The original application has the application number 202180021631.X and the original application date is December 16, 2021. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of automotive technology, and more particularly to an image processing method, apparatus, storage medium, electronic rearview mirror system, and vehicle. Background Technology

[0003] With technological advancements, electronic rearview mirrors have begun to be installed in vehicles, providing visual information about the vehicle's surroundings and assisting with safe driving. Electronic rearview mirrors, also known as camera monitor systems (CMS), consist of a camera and a display screen. The camera captures real-time images and transmits them to the screen for display. Compared to optical rearview mirrors, they offer advantages such as a wider field of view and less susceptibility to environmental influences.

[0004] Currently, implementing electronic rearview mirror functionality in vehicles typically requires a dedicated electronic control unit (ECU) to process data collected by cameras and display it on a screen. However, the current approach to installing electronic rearview mirrors in vehicles is still immature, often suffering from slow startup and high latency, impacting user experience and hindering large-scale application. Therefore, there is an urgent need for safer and more efficient image processing technologies to improve these issues, enhance user experience, and enable the widespread adoption of this technology. Summary of the Invention

[0005] In view of this, an image processing method, apparatus, storage medium, electronic rearview mirror system, and vehicle are proposed.

[0006] In a first aspect, embodiments of this application provide an image processing method. This method is used in an electronic rearview mirror control unit, and includes: acquiring first image data collected by one or more cameras; sending the first image data to a domain controller (DC); processing the first image data to obtain processed image data; and outputting the processed image data to a display device corresponding to the electronic rearview mirror control unit.

[0007] According to the embodiments of this application, the electronic rearview mirror control unit acquires image data collected by the camera and outputs the processed image data to the display device. This allows the camera to be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without relying on the domain controller. Even when the domain controller fails, the electronic rearview mirror function can still operate normally, ensuring the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Simultaneously, by sending the first image data to the domain controller through the electronic rearview mirror control unit, the electronic rearview mirror and the domain controller can share the image data collected by the camera, saving costs and deployment space.

[0008] According to the first aspect, in a first possible implementation of the image processing method, the method further includes: receiving second image data sent by the domain controller, the second image data being determined by the domain controller based on the first image data and / or sensor data collected by one or more sensors; processing the second image data to obtain processed image data; and outputting the processed image data to the display device.

[0009] According to the embodiments of this application, the first image data is sent to the domain controller for processing by the electronic rearview mirror control unit to obtain the second image data. This allows the domain controller to use its computing power to perform calculations, thereby enhancing the function of the electronic rearview mirror and further improving safety and comfort during driving.

[0010] According to a first possible implementation of the first aspect, in a second possible implementation of the image processing method, receiving the second image data sent by the domain controller includes: receiving the second image data sent by the domain controller via a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

[0011] This allows for higher transmission efficiency when the electronic rearview mirror control unit receives the second image data.

[0012] In a third possible implementation of the image processing method, according to the first or second possible implementation of the first aspect, the field of view of the second image data corresponds to the vehicle driving scene.

[0013] This allows for more refined image information and a better driving experience.

[0014] According to the first, second, or third possible implementation of the first aspect, in the fourth possible implementation of the image processing method, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

[0015] This allows for a wider field of view in the image, resulting in a better driving experience.

[0016] According to the first aspect or the first, second, third or fourth possible implementation of the first aspect, in the fifth possible implementation of the image processing method, the method further includes: receiving indication information sent by the domain controller, the indication information being determined by the domain controller based on the first image data and / or, sensing data collected by one or more sensors; and outputting corresponding display information to the display device according to the indication information.

[0017] According to the embodiments of this application, the first image data is sent to the domain controller for processing by the electronic rearview mirror control unit to obtain instruction information. This enables the domain controller to use its computing power to perform calculations, thereby enhancing the function of the electronic rearview mirror and further improving safety and comfort during driving.

[0018] According to the first aspect or any of the first, second, third, fourth, or fifth possible implementations of the first aspect, in the sixth possible implementation of the image processing method, the electronic rearview mirror control unit includes a deserializer and a processing unit. The deserializer is used to connect with the one or more cameras, acquire the first image data, and send the first image data to the processing unit. The processing unit is used to process the first image data to obtain processed image data and output the processed image data to the display device corresponding to the electronic rearview mirror control unit.

[0019] Therefore, the electronic rearview mirror function can be realized independently.

[0020] According to the sixth possible implementation of the first aspect, in the seventh possible implementation of the image processing method, sending the first image data to the domain controller DC includes: the deserializer sending the first image data to the domain controller.

[0021] This enables an electronic rearview mirror control unit to send first image data to a domain controller, allowing the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0022] According to the sixth possible implementation of the first aspect, in the eighth possible implementation of the image processing method, sending the first image data to the domain controller DC includes: the processing unit sending the first image data to the domain controller.

[0023] This provides an alternative way for the electronic rearview mirror control unit to send the first image data to the domain controller, enabling the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0024] In a ninth possible implementation of the image processing method, according to the first aspect or one or two possible implementations of the first aspect, sending the first image data to the domain controller DC includes: sending the first image data to the domain controller via a Low Voltage Differential Signaling (LVDS) interface.

[0025] This allows the domain controller to utilize its computing power to perform calculations that enhance the functionality of the electronic rearview mirror and improve the user experience.

[0026] According to the first aspect or any of the first, second, third, fourth, fifth, sixth, seventh, eighth, or ninth possible implementations of the first aspect, in the tenth possible implementation of the image processing method, the one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera for displaying the electronic rearview mirror image captured by the first camera; a second display device corresponding to the second camera for displaying the electronic rearview mirror image captured by the second camera; and a third display device corresponding to the third camera for displaying the electronic rearview mirror image captured by the third camera.

[0027] Therefore, by utilizing multiple cameras and display devices, the functions of electronic rearview mirrors can be better realized, enhancing the user experience.

[0028] Secondly, embodiments of this application provide an image processing method. This method is used in a domain controller (DC) and includes: receiving first image data sent by an electronic rearview mirror control unit, the first image data being acquired by the electronic rearview mirror control unit from one or more cameras; the first image data being processed by the electronic rearview mirror control unit to obtain processed image data; and the processed image data being output by the electronic rearview mirror control unit to a display device corresponding to the electronic rearview mirror control unit.

[0029] According to the embodiments of this application, by receiving first image data, which is image data acquired by the camera and obtained by the electronic rearview mirror control unit, and processing and outputting this image data to a display device, the camera can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without relying on a domain controller. Even if the domain controller fails, the electronic rearview mirror function can still be implemented normally, ensuring the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Simultaneously, by receiving the first image data sent by the electronic rearview mirror control unit, it is also possible to share the image data acquired by the camera with the electronic rearview mirror, saving costs and deployment space.

[0030] According to the second aspect, in a first possible implementation of the image processing method, the method further includes: acquiring sensing data collected by one or more sensors; determining second image data based on the first image data and / or the sensing data collected by the one or more sensors; sending the second image data to the electronic rearview mirror control unit, wherein the second image data is processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is output by the electronic rearview mirror control unit to the display device.

[0031] According to the embodiments of this application, by acquiring sensor data through a domain controller and using the first image data and / or the sensor data to determine the second image data, the domain controller can use its computing power to perform calculations to enhance the function of the electronic rearview mirror and further improve safety and comfort during driving.

[0032] According to the first possible implementation of the second aspect, in the second possible implementation of the image processing method, sending the second image data to the electronic rearview mirror control unit includes: sending the second image data to the electronic rearview mirror control unit via a controller area network (CAN) interface or an Ethernet (ETH) interface.

[0033] This allows for higher transmission efficiency when the domain controller sends the second image data.

[0034] According to the first or second possible implementation of the second aspect, in the third possible implementation of the image processing method, the field of view of the second image data corresponds to the vehicle driving scene.

[0035] This allows for more refined image information and a better driving experience.

[0036] According to the first, second, or third possible implementation of the second aspect, in the fourth possible implementation of the image processing method, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

[0037] This allows for a wider field of view in the image, resulting in a better driving experience.

[0038] According to the second aspect or the first, second, third or fourth possible implementation of the second aspect, in a fifth possible implementation of the image processing method, the method further includes: acquiring sensing data collected by one or more sensors; determining indication information based on the first image data, and / or the sensing data collected by the one or more sensors; sending the indication information to the electronic rearview mirror control unit, wherein the indication information is used by the electronic rearview mirror control unit to output corresponding display information to the display device according to the indication information.

[0039] According to the embodiments of this application, by acquiring sensor data through a domain controller and using the first image data and / or the sensor data to determine indication information, the domain controller can use its computing power to perform calculations to enhance the function of the electronic rearview mirror and further improve safety and comfort during driving.

[0040] According to the second aspect or the first, second, third, fourth or fifth possible implementation of the second aspect, in the sixth possible implementation of the image processing method, the electronic rearview mirror control unit includes a deserializer, and receiving first image data sent by the electronic rearview mirror control unit includes: receiving the first image data sent by the deserializer.

[0041] This enables a way for a domain controller to receive first image data, allowing the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0042] According to the second aspect or the first, second, third, fourth or fifth possible implementation of the second aspect, in the seventh possible implementation of the image processing method, the electronic rearview mirror control unit includes a processing unit that receives first image data sent by the electronic rearview mirror control unit, including: receiving the first image data sent by the processing unit.

[0043] This allows for an alternative way for the domain controller to receive the first image data, enabling the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0044] According to the second aspect or the first, second, third, fourth, fifth, sixth, or seventh possible implementation of the second aspect, in the eighth possible implementation of the image processing method, receiving the first image data sent by the electronic rearview mirror control unit includes: receiving the first image data sent by the electronic rearview mirror control unit through the low voltage differential signal LVDS interface.

[0045] This allows the domain controller to utilize its computing power to perform calculations that enhance the functionality of the electronic rearview mirror and improve the user experience.

[0046] According to the second aspect or any of the first, second, third, fourth, fifth, sixth, seventh, or eighth possible implementations of the second aspect, in the ninth possible implementation of the image processing method, the one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera for displaying the electronic rearview mirror image captured by the first camera; a second display device corresponding to the second camera for displaying the electronic rearview mirror image captured by the second camera; and a third display device corresponding to the third camera for displaying the electronic rearview mirror image captured by the third camera.

[0047] Therefore, by utilizing multiple cameras and display devices, the functions of electronic rearview mirrors can be better realized, enhancing the user experience.

[0048] Thirdly, embodiments of this application provide an image processing apparatus. This apparatus is used in an electronic rearview mirror control unit, and includes: a first acquisition module for acquiring first image data captured by one or more cameras; a first transmission module for transmitting the first image data to a domain controller (DC); and a first processing output module for processing the first image data to obtain processed image data, and outputting the processed image data to a display device corresponding to the electronic rearview mirror control unit.

[0049] According to a third aspect, in a first possible implementation of the image processing apparatus, the apparatus further includes: a first receiving module for receiving second image data sent by the domain controller, the second image data being determined by the domain controller based on the first image data and / or sensing data collected by one or more sensors; and a second processing output module for processing the second image data to obtain processed image data and outputting the processed image data to the display device.

[0050] According to the first possible implementation of the third aspect, in the second possible implementation of the image processing device, the first receiving module is configured to: receive the second image data sent by the domain controller via the Controller Area Network (CAN) interface or the Ethernet (ETH) interface.

[0051] According to the first or second possible implementation of the third aspect, in the third possible implementation of the image processing device, the field of view of the second image data corresponds to the vehicle driving scene.

[0052] According to the first, second, or third possible implementation of the third aspect, in the fourth possible implementation of the image processing device, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

[0053] According to the third aspect or the first, second, third or fourth possible implementation of the third aspect, in a fifth possible implementation of the image processing device, the device further includes: a second receiving module, configured to receive indication information sent by the domain controller, the indication information being determined by the domain controller based on the first image data and / or, sensing data collected by one or more sensors; and a third processing output module, configured to output corresponding display information to the display device according to the indication information.

[0054] According to the third aspect or the first, second, third, fourth or fifth possible implementation of the third aspect, in the sixth possible implementation of the image processing device, the electronic rearview mirror control unit includes a deserializer and a processing unit. The deserializer is used to connect with the one or more cameras, acquire the first image data, and send the first image data to the processing unit. The processing unit is used to process the first image data to obtain processed image data, and output the processed image data to the display device corresponding to the electronic rearview mirror control unit.

[0055] According to the sixth possible implementation of the third aspect, in the seventh possible implementation of the image processing device, the first sending module is configured to: send the first image data to the domain controller via the deserializer.

[0056] According to the sixth possible implementation of the third aspect, in the eighth possible implementation of the image processing apparatus, the first sending module is configured to: send the first image data to the domain controller by the processing unit.

[0057] According to the third aspect or the first, second, third, fourth, fifth, sixth, seventh, or eighth possible implementation of the third aspect, in the ninth possible implementation of the image processing device, the first transmitting module is configured to: transmit the first image data to the domain controller via a low-voltage differential signaling (LVDS) interface.

[0058] According to the third aspect or any of the first, second, third, fourth, fifth, sixth, seventh, eighth, or ninth possible implementations of the third aspect, in the tenth possible implementation of the image processing device, the one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera for displaying the electronic rearview mirror image captured by the first camera; a second display device corresponding to the second camera for displaying the electronic rearview mirror image captured by the second camera; and a third display device corresponding to the third camera for displaying the electronic rearview mirror image captured by the third camera.

[0059] Fourthly, embodiments of this application provide an image processing apparatus. This apparatus is used in a domain controller (DC) and includes: a third receiving module for receiving first image data sent by an electronic rearview mirror control unit, wherein the first image data is acquired by the electronic rearview mirror control unit from one or more cameras, and is processed by the electronic rearview mirror control unit to obtain processed image data, which is then output by the electronic rearview mirror control unit to a display device corresponding to the electronic rearview mirror control unit.

[0060] According to the fourth aspect, in a first possible implementation of the image processing device, the device further includes: a second acquisition module for acquiring sensing data collected by one or more sensors; a first determination module for determining second image data based on the first image data and / or the sensing data collected by the one or more sensors; and a second transmission module for transmitting the second image data to the electronic rearview mirror control unit, wherein the second image data is processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is output by the electronic rearview mirror control unit to the display device.

[0061] According to the first possible implementation of the fourth aspect, in the second possible implementation of the image processing device, the second transmitting module is used to: transmit the second image data to the electronic rearview mirror control unit via a controller area network (CAN) interface or an Ethernet (ETH) interface.

[0062] According to the first or second possible implementation of the fourth aspect, in the third possible implementation of the image processing device, the field of view of the second image data corresponds to the vehicle driving scene.

[0063] According to the first, second, or third possible implementation of the fourth aspect, in the fourth possible implementation of the image processing device, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

[0064] According to the fourth aspect or the first, second, third or fourth possible implementation of the fourth aspect, in a fifth possible implementation of the image processing device, the device further includes: a third acquisition module, configured to acquire sensing data collected by one or more sensors; a second determination module, configured to determine indication information based on the first image data and / or the sensing data collected by the one or more sensors; and a third transmission module, configured to transmit the indication information to the electronic rearview mirror control unit, wherein the indication information is used by the electronic rearview mirror control unit to output corresponding display information to the display device according to the indication information.

[0065] According to the fourth aspect or the first, second, third, fourth or fifth possible implementation of the fourth aspect, in the sixth possible implementation of the image processing device, the electronic rearview mirror control unit includes a deserializer and a third receiving module for: receiving the first image data sent by the deserializer.

[0066] According to the fourth aspect or the first, second, third, fourth or fifth possible implementation of the fourth aspect, in the seventh possible implementation of the image processing device, the electronic rearview mirror control unit includes a processing unit and a third receiving module for: receiving the first image data sent by the processing unit.

[0067] According to the fourth aspect or the first, second, third, fourth, fifth, sixth, or seventh possible implementation of the fourth aspect, in the eighth possible implementation of the image processing device, the third receiving module is configured to: receive the first image data sent by the electronic rearview mirror control unit through the low-voltage differential signal LVDS interface.

[0068] According to the fourth aspect or the first, second, third, fourth, fifth, sixth, seventh, or eighth possible implementations of the fourth aspect, in the ninth possible implementation of the image processing device, the one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera for displaying the electronic rearview mirror image captured by the first camera; a second display device corresponding to the second camera for displaying the electronic rearview mirror image captured by the second camera; and a third display device corresponding to the third camera for displaying the electronic rearview mirror image captured by the third camera.

[0069] Fifthly, embodiments of this application provide an image processing apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to, when executing the instructions, implement one or more of the image processing methods described in the first aspect or multiple possible implementations of the first aspect, or implement one or more of the image processing methods described in the second aspect or multiple possible implementations of the second aspect.

[0070] Sixthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement one or more of the image processing methods described in the first aspect or in various possible implementations of the first aspect, or implement one or more of the image processing methods described in the second aspect or in various possible implementations of the second aspect.

[0071] In a seventh aspect, embodiments of this application provide a terminal device that can execute one or more of the image processing methods described in the first aspect or in various possible implementations of the first aspect, or execute one or more of the image processing methods described in the second aspect or in various possible implementations of the second aspect.

[0072] Eighthly, embodiments of this application provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in an electronic device, the processor in the electronic device executes one or more of the image processing methods described in the first aspect or multiple possible implementations of the first aspect, or executes one or more of the image processing methods described in the second aspect or multiple possible implementations of the second aspect.

[0073] Ninthly, embodiments of this application provide an electronic rearview mirror system, including: one or more cameras; a display device corresponding to the one or more cameras; and an electronic rearview mirror control unit for executing one or more image processing methods from the first aspect or various possible implementations of the first aspect.

[0074] In a tenth aspect, embodiments of this application provide a vehicle including one or more cameras; a display device corresponding to the one or more cameras; an electronic rearview mirror control unit for executing one or more image processing methods of the first aspect or multiple possible implementations thereof; and a domain controller DC for executing one or more image processing methods of the second aspect or multiple possible implementations thereof.

[0075] Eleventhly, embodiments of this application provide a vehicle including an electronic rearview mirror system as described in the ninth aspect above; a domain controller DC for executing one or more image processing methods from the second aspect or various possible implementations of the second aspect above.

[0076] These and other aspects of this application will become more apparent in the description of the following embodiments(s). Attached Figure Description

[0077] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0078] Figure 1 This demonstrates a possible deployment method for a smart driving domain controller and an electronic rearview mirror to share a camera.

[0079] Figure 2 A schematic diagram of a system architecture according to an embodiment of this application is shown.

[0080] Figure 3 A flowchart illustrating an image processing method according to an embodiment of this application is shown.

[0081] Figure 4 A schematic diagram illustrating a transmission method of first image data according to an embodiment of this application is shown.

[0082] Figure 5 A flowchart illustrating an image processing method according to an embodiment of this application is shown.

[0083] Figure 6 This diagram illustrates a transmission method of second image data according to an embodiment of the present application.

[0084] Figure 7A flowchart illustrating an image processing method according to an embodiment of this application is shown.

[0085] Figure 8 A structural diagram of an image processing apparatus according to an embodiment of this application is shown.

[0086] Figure 9 A structural diagram of an image processing apparatus according to an embodiment of this application is shown.

[0087] Figure 10 This diagram illustrates the structure of an electronic device 100 according to an embodiment of the present application. Detailed Implementation

[0088] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0089] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0090] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0091] With technological advancements, vehicles often have cameras installed in various locations to capture images of their surroundings. The intelligent driving domain controller (ADAS) then fuses this data from sensors, including cameras, and uses this fused data for vehicle control. When electronic rearview mirrors are installed, the locations of the cameras connected to the rearview mirror and the cameras connected to the ADAS may overlap. Therefore, it's necessary to address the issue of sharing between the ADAS and rearview mirror cameras at these locations. Figure 1 This illustrates a possible deployment method where an intelligent driving domain controller and an electronic rearview mirror share a camera. See also Figure 1In this deployment method, the camera sends the captured image data to the intelligent driving domain controller. The intelligent driving domain controller can fuse the image data with data acquired from other sensors to control the vehicle and realize the intelligent driving function. The image data is also sent by the intelligent driving domain controller to the electronic rearview mirror ECU. After acquiring the image data, the electronic rearview mirror ECU processes the data and sends it to the corresponding display screen for display, thus realizing the electronic rearview mirror function.

[0092] In this configuration, when the intelligent driving domain controller fails, the electronic rearview mirror ECU cannot obtain image data from the camera, thus rendering the electronic rearview mirror unusable and failing to meet safety requirements. Furthermore, after the vehicle is powered on, the electronic rearview mirror requires the intelligent driving domain controller to start up before it can function properly, resulting in slow startup, high latency, and negatively impacting the user experience.

[0093] To address the aforementioned technical problems, this application provides an image processing method. This method enables an electronic rearview mirror control unit to acquire first image data from one or more cameras, process the first image data to obtain processed image data, and output the processed image data to a display device corresponding to the control unit, thereby realizing the function of an electronic rearview mirror. The electronic rearview mirror control unit can also send the first image data to a domain controller (DC), which receives the data to enable other intelligent driving functions. According to this application, cameras can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without relying on a domain controller. This ensures that the electronic rearview mirror function can still operate normally even when the domain controller fails, guaranteeing the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Simultaneously, by sending the first image data to the domain controller, the electronic rearview mirror and the domain controller can share the image data acquired by the camera, saving costs and space for camera deployment.

[0094] Figure 2 A schematic diagram of a system architecture according to an embodiment of this application is shown. Figure 2As shown, the electronic rearview mirror system of this application embodiment can be deployed on a vehicle and may include one or more of the following: an electronic rearview mirror control unit 101, one or more cameras, and one or more display devices. For example, the electronic rearview mirror system may only include the electronic rearview mirror control unit 101, or include the electronic rearview mirror control unit 101 and one or more display devices, or include the electronic rearview mirror control unit 101 and one or more cameras, or include the electronic rearview mirror control unit 101, one or more cameras, and one or more display devices. This application does not limit this. The one or more cameras may include cameras 111-113. Some or all of cameras 111-113 may be cameras dedicated to implementing the electronic rearview mirror function. Some or all of cameras 111-113 may also be part of sensors 103. In addition to implementing the electronic rearview mirror function, they can also be used to provide image data to the domain controller for vehicle control to achieve intelligent driving functions. This application does not limit this. The one or more display devices may include display devices 121-123. Some or all of the display devices 121-123 may be dedicated to implementing the electronic rearview mirror function, or some or all of the display devices 121-123 may be used to implement other intelligent driving functions in addition to the electronic rearview mirror function. This application does not limit this. The cameras 111-113 may be deployed outside the vehicle and connected to the electronic rearview mirror control unit 101 to capture images of the rear of the vehicle, obtain first image data, and send it to the electronic rearview mirror control unit 101. The electronic rearview mirror control unit 101 may be dedicated to implementing the electronic rearview mirror function by combining the camera and display devices. It may be deployed separately on the ECU or CDC, or integrated with other vehicle-mounted domain controllers. It may receive the first image data collected by the cameras 111-113, process the first image data to obtain processed image data, and send it to the display devices 121-123 for display. It may also send the first image data to the domain controller 102 deployed on the vehicle. Domain controller 102 can be connected to electronic rearview mirror control unit 101 to receive first image data sent by electronic rearview mirror control unit 101. Domain controller 102 can be a domain controller used to implement intelligent driving related functions, such as an in-vehicle computing device. Display devices 121-123 can be screens, projectors, etc., and can be deployed inside the vehicle. They can be connected to electronic rearview mirror control unit 101 to receive and display processed image data sent by electronic rearview mirror control unit 101.

[0095] In one possible implementation, other sensors are also deployed on the vehicle, as shown in the figure, such as sensor 103, which can be connected to domain controller 102. Sensor 103 may include one or more of the following: vehicle-mounted radar (such as millimeter-wave radar, lidar, ultrasonic radar, etc.), rain sensor, camera, vehicle attitude sensor (such as gyroscope), inertial measurement unit (IMU), global navigation satellite system (GNSS), etc. The camera may be a camera other than cameras 111-113, or some or all of the cameras 111-113. In this embodiment, sensor 103 can be used to enhance the function of the electronic rearview mirror. For example, after the electronic rearview mirror control unit 101 sends the first image data to the domain controller 102, the domain controller 102 can determine new image data based on the first image data and the sensing data collected by sensor 103 to achieve better electronic rearview mirror function.

[0096] In one possible implementation, the image processing method of this application can be used in intelligent driving scenarios. For example, in an autonomous driving scenario, if the domain controller 102 fails, the driver can take over vehicle control. At this time, the electronic rearview mirror control unit 101 can also work normally, realizing the function of the electronic rearview mirror, that is, the driver can drive safely based on the surrounding conditions indicated by the electronic rearview mirror. It should be noted that the image processing method of this application can also be used in other vehicle driving scenarios, such as in non-intelligent driving scenarios to assist the driver in driving, and this application does not limit this application.

[0097] The following is passed Figures 3-7 ,exist Figure 2 Based on the system architecture shown, the image processing method of the embodiments of this application will be described in detail.

[0098] Figure 3 A flowchart illustrating an image processing method according to an embodiment of this application is shown. Figure 3 As shown, the method includes: Step S201: The electronic rearview mirror control unit acquires first image data captured by one or more cameras.

[0099] One or more cameras may include a first camera (such as camera 111 described above) for capturing images of the left rear of the vehicle, a second camera (such as camera 112 described above) for capturing images of the right rear of the vehicle, and a third camera (such as camera 113 described above) for capturing images of the direct rear of the vehicle. The first image data may include image data of the left rear, right rear, and direct rear of the vehicle collected by cameras 111-113, respectively. The cameras may include only some of the cameras described above, or they may include more than those described above; this application does not impose any limitations on this.

[0100] The electronic rearview mirror control unit may include a deserializer and a processing unit. The processing unit may be a system on chip (SoC). The deserializer may be connected to one or more cameras (such as cameras 111-113 mentioned above). The deserializer acquires the first image data captured by the camera and sends the first image data to the processing unit, which then executes step S204.

[0101] In one possible implementation, one or more of the aforementioned cameras can send first image data to the deserializer via low-voltage differential signaling (LVDS).

[0102] In step S202, the electronic rearview mirror control unit sends the first image data to the domain controller.

[0103] In order to enable the electronic rearview mirror control unit to send the first image data to the domain controller to realize intelligent driving, and in order to meet the functional safety requirements of intelligent driving, the automotive safety integration level (ASIL) of the electronic rearview mirror control unit can be ASIL B or higher.

[0104] The electronic rearview mirror control unit can send the first image data to the domain controller via a Low Voltage Differential Signaling (LVDS) interface. (See also...) Figure 4 , Figure 4 This diagram illustrates a transmission method of first image data according to an embodiment of this application. Figure 4 As shown, the domain controller may include a deserializer and a processing unit, which may be a system-on-a-chip (SoC) (e.g., SoC2 in the figure).

[0105] In one possible implementation, in step S202, the electronic rearview mirror control unit can send the first image data to the domain controller via a deserializer. See also... Figure 4 (a) and Figure 4 (b) shows two ways in which the electronic rearview mirror control unit sends the first image data to the domain controller via a deserializer.

[0106] In the first method, such as Figure 4 As shown in (a), after receiving the first image data sent by the camera as parallel data, the deserializer of the electronic rearview mirror control unit can choose not to convert the first image data from parallel data to serial data, and instead send the first image data to the deserializer of the domain controller through the LVDS interface. The deserializer of the domain controller processes the data, converts the first image data from parallel data to serial data, and sends the processed image data to the SoC2 through the mobile industry processor interface (MIPI), so that the subsequent processing unit of the domain controller can process the image data to realize the relevant intelligent driving functions.

[0107] In the second method, such as Figure 4 As shown in (b), the electronic rearview mirror control unit may further include a serializer. The deserializer of the electronic rearview mirror control unit can process the first image data, converting it from parallel data to serial data, and send it to the serializer via MIPI. The serializer can convert the first image data from serial data to parallel data and send it to the deserializer of the domain controller via the LVDS interface. The deserializer of the domain controller can process the data, converting the first image data from parallel data to serial data, and send the processed image data to the SoC2 via the mobile industry processor interface (MIPI), so that the subsequent processing unit of the domain controller can process the image data to realize relevant intelligent driving functions.

[0108] In one possible implementation, in step S202, the electronic rearview mirror control unit can send the first image data to the domain controller via its processing unit. See also... Figure 4 (c) illustrates one method by which the electronic rearview mirror control unit sends first image data to the domain controller via a processing unit. The processing unit of the electronic rearview mirror control unit can be a system-on-a-chip (SoC) (e.g., SoC1 in the figure). like Figure 4As shown in (c), the electronic rearview mirror control unit may further include a serializer. SoC1 can receive first image data, which is processed by the deserializer and sent as serial data, via MIPI, and send it to the serializer via MIPI. The serializer can convert the first image data from serial data to parallel data and send the first image data to the deserializer of the domain controller via the LVDS interface. The deserializer of the domain controller can process the data, converting the first image data from parallel data back to serial data, and send the processed image data to SoC2 via MIPI, so that the subsequent processing unit of the domain controller can process the image data to realize relevant intelligent driving functions.

[0109] In step S203, the domain controller receives the first image data sent by the electronic rearview mirror control unit.

[0110] The domain controller can receive the first image data sent by the electronic rearview mirror control unit through the Low Voltage Differential Signaling (LVDS) interface.

[0111] In one possible implementation, in step S203, the domain controller may receive the first image data sent by the deserializer of the electronic rearview mirror control unit, and the domain controller may also receive the first image data sent by the processing unit of the electronic rearview mirror control unit. The above process corresponds to that in step S202, and the relevant description in step S202 can be found therein.

[0112] In step S204, the electronic rearview mirror control unit processes the first image data to obtain processed image data, and outputs the processed image data to the display device corresponding to the electronic rearview mirror control unit.

[0113] Step S204 can be executed by the processing unit of the electronic rearview mirror control unit.

[0114] For example, the processing unit can convert the color format of the first image data, such as converting the YUV format to the RGB format.

[0115] The electronic rearview mirror control unit may also include a serializer. The processing unit can send the processed image data to the serializer via MIPI for processing, converting the image data from serial data to parallel data, and then sending it to the corresponding display device for display via the LVDS interface.

[0116] The display device can be a display device dedicated to realizing the function of electronic rearview mirror, or it can be a display device that simultaneously realizes the function of electronic rearview mirror and other intelligent driving functions, such as the display screen of the center console. It can also be used to realize other functions such as global positioning system (GPS) navigation. This application does not limit this.

[0117] In one possible implementation, the display device may include: a first display device corresponding to the first camera (such as display device 121 described above), which may be located on the left front side of the vehicle interior and is used to display the electronic rearview mirror image of the left rear of the vehicle captured by the first camera; a second display device corresponding to the second camera (such as display device 122 described above), which may be located on the right front side of the vehicle interior and is used to display the electronic rearview mirror image of the right rear of the vehicle captured by the second camera; and a third display device corresponding to the third camera (such as display device 123 described above), which may be located above the center console of the vehicle interior and is used to display the electronic rearview mirror image of the vehicle directly behind the third camera captured by the third camera. The display device may also include only some of the display devices described above or include more display devices than described above, and this application does not impose any limitations on this.

[0118] According to the embodiments of this application, the electronic rearview mirror control unit acquires image data collected by the camera and outputs the processed image data to the display device. This allows the camera to be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without relying on the domain controller. Even if the domain controller fails, the electronic rearview mirror function can still operate normally, ensuring the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Sending the first image data to the domain controller through the electronic rearview mirror control unit also enables the electronic rearview mirror and the domain controller to share the image data collected by the camera, saving costs and deployment space.

[0119] In one possible implementation, based on the above process, the domain controller can further process the first image data using sensor data to enhance the functionality of the electronic rearview mirror, thereby improving safety and comfort during driving. Two methods for enhancing the functionality of the electronic rearview mirror are described below. Figure 5 and Figure 6 .

[0120] Figure 5 A flowchart illustrating an image processing method according to an embodiment of this application is shown. Figure 5 As shown, the method also includes: Step S401: The domain controller acquires sensing data collected by one or more sensors.

[0121] The sensor data can be used to realize related functions of intelligent driving. For example, it can be radar data collected by vehicle radar, image data collected by cameras (including some or all of the cameras 111-113, or other cameras other than cameras 111-113, such as 360-degree surround view cameras, etc.), etc. This application does not limit it.

[0122] In step S402, the domain controller determines the second image data based on the first image data and / or the sensing data collected by the one or more sensors.

[0123] For example, the sensing data may include image data captured by a camera in the sensor. This image data may include image data of obscured portions of the first image data (i.e., blind spots in the first image data, such as portions obscured by pillars on either side of the windshield of a vehicle), and may also include other portions not captured in the first image data. Based on this image data, the first image data can be supplemented to determine second image data, which may have a larger field of view than the first image data. The second image data may be determined by the processing unit of the domain controller.

[0124] In one possible implementation, the field of view of the second image data can include a 360-degree panoramic environment around the vehicle. This results in a wider field of vision while driving, making driving safer and more comfortable.

[0125] In one possible implementation, the field of view of the second image data can correspond to the vehicle driving scene.

[0126] For example, a domain controller can determine the current driving state of a vehicle, such as driving straight, reversing, turning left, turning right, or changing lanes, based on sensor data collected by vehicle posture sensors (such as gyroscopes). It can then determine the field of view (including different orientations and angles) of the second image data according to different scenarios. For instance, if the vehicle is driving straight, the second image data could be images of the left and right sides of the vehicle (including portions obscured by the pillars on either side of the windshield); if the vehicle is changing lanes to the left, the second image data could be images of the left and rear sides of the vehicle from the first image data. This second image data can be determined by supplementing the obscured portions of the first image data with image data collected by cameras in the sensors. The processing unit within the domain controller can then determine the second image data.

[0127] This allows for more refined image information and a better driving experience.

[0128] In step S403, the domain controller sends the second image data to the electronic rearview mirror control unit.

[0129] In one possible implementation, the interface used by the domain controller to send the second image data to the electronic rearview mirror control unit can be found in [reference needed]. Figure 4 In this context, the interface used to transmit the first image data is merely the same as the transmission direction when the domain controller sends the second image data to the electronic rearview mirror control unit. Figure 4 The transmission direction when sending the first image data is reversed, which will not be elaborated here.

[0130] In one possible implementation, the domain controller can also send second image data to the electronic rearview mirror control unit via a controller area network (CAN) or Ethernet (ETH) interface.

[0131] Figure 6 This diagram illustrates a transmission method for second image data according to an embodiment of this application. Figure 6 As shown, the processing unit of the domain controller (Soc2 in the figure) can send the second image data to the processing unit of the electronic rearview mirror control unit (SoC1 in the figure) via the CAN or ETH interface.

[0132] This allows for higher transmission efficiency when sending the second image data.

[0133] In step S404, the electronic rearview mirror control unit receives the second image data sent by the domain controller.

[0134] In one possible implementation, the electronic rearview mirror control unit receives the second image data from the domain controller via a Controller Area Network (CAN) interface or an Ethernet (ETH) interface. This process is described above. Figure 6 This will not be elaborated upon here.

[0135] In step S405, the electronic rearview mirror control unit processes the second image data to obtain processed image data, and outputs the processed image data to the display device.

[0136] Step S405 can be executed by the processing unit of the electronic rearview mirror control unit. The processing unit of the electronic rearview mirror control unit can process the second image data in the same way as the processing of the first image data in step S204, for example, by converting the color format of the second image data.

[0137] For example, if the field of view of the second image data is a 360-degree panoramic environment around the vehicle, the 360-degree panoramic environment can be displayed in split screen on all display devices. For example, if there are three display devices, the environment around the vehicle can be displayed in 120-degree increments after the second image data has been processed on each device.

[0138] For example, if the field of view of the second image data differs under different vehicle driving scenarios, the corresponding image can be displayed on the corresponding display device. For instance, if there are three display devices, the second image data sent by the domain controller can include image data of the left, right, and rear sides of the vehicle, corresponding to the left, right, and center console display devices inside the vehicle, respectively. When the domain controller determines that the current driving state of the vehicle is straight, the second image data sent by the domain controller to the electronic rearview mirror control unit can include image data of the left and right sides of the vehicle, while the image data corresponding to the rear side of the vehicle can be empty. After receiving the second image data, the electronic rearview mirror control unit can display the processed image data corresponding to the left and right sides of the vehicle on the left and right display devices inside the vehicle, respectively, instead of displaying the empty image data on the center console display device.

[0139] It should be noted that the display device may display image data in other ways, which will not be shown in this application.

[0140] According to the embodiments of this application, the first image data is sent to the domain controller for processing by the electronic rearview mirror control unit to obtain the second image data. This allows the domain controller to use its computing power to perform calculations, thereby enhancing the function of the electronic rearview mirror and further improving safety and comfort during driving.

[0141] Figure 7 A flowchart illustrating an image processing method according to an embodiment of this application is shown. Figure 7 As shown, the method also includes: Step S601: The domain controller acquires sensing data collected by one or more sensors.

[0142] This process can be found in the relevant description in step S401.

[0143] In step S602, the domain controller determines indication information based on the first image data and / or the sensing data collected by the one or more sensors.

[0144] The indication information can be used to indicate the conditions around the vehicle in a 360-degree range.

[0145] For example, the sensing data may include data collected by vehicle-mounted radar, which may indicate the distance of the vehicle to relevant obstacles. The domain controller can use the data collected by the vehicle-mounted radar (e.g., lidar) to determine whether there are obstacles too close to the vehicle (which can be determined as too close if the distance is less than a preset threshold), and to determine the position of the obstacle, which may be the relative position of the obstacle to the vehicle. After determining the position of the obstacle, it can also locate the obstacle in the first image data based on the obstacle's position to determine the type of obstacle (e.g., other vehicles, pedestrians, etc.), thereby determining the indication information. In one possible implementation, the domain controller can also determine the second image data based on the first image data and / or the sensing data collected by the one or more sensors, referring to the method in step S402. After determining the position of the obstacle, it can also locate the obstacle in the second image data based on the obstacle's position to determine the type of obstacle, thereby determining the indication information. The indication information may include information such as the location and type of obstacles that are too close to the vehicle, for example, image data indicating the location of the corresponding obstacle on the first or second image data, or information indicating the distance between the obstacle and the vehicle, etc. This application does not impose any limitations on this. The indication information may be determined by a processing unit in the domain controller.

[0146] In step S603, the domain controller sends the instruction information to the electronic rearview mirror control unit.

[0147] The method by which the domain controller sends instruction information to the electronic rearview mirror control unit can be referred to in step S403, which is the method by which the domain controller sends the second image data to the electronic rearview mirror control unit, and will not be repeated here.

[0148] In step S604, the electronic rearview mirror control unit receives the instruction information sent by the domain controller.

[0149] The method by which the electronic rearview mirror control unit receives the instruction information can be referred to in step S404, which describes the method by which the electronic rearview mirror control unit receives the second image data. It will not be repeated here.

[0150] In step S605, the electronic rearview mirror control unit outputs corresponding display information to the display device according to the instruction information.

[0151] Step S605 can be executed by the processing unit of the electronic rearview mirror control unit.

[0152] The information can be displayed on the display device in the form of text, images, video, etc., and this application does not impose any restrictions on this.

[0153] For example, the displayed information could include the location and type of the obstacle. For instance, if the obstacle is a pedestrian on the left side of the vehicle, the displayed information could include an image of the left side of the vehicle, highlighting the pedestrian as an obstacle, and also including text indicating the distance of the obstacle from the vehicle. After being output to a display device, the image of the left side of the vehicle can be displayed on the display device on the left side inside the vehicle, highlighting the pedestrian as an obstacle and displaying the pedestrian's current distance from the vehicle in text form.

[0154] According to the embodiments of this application, the first image data is sent to the domain controller for processing by the electronic rearview mirror control unit to obtain instruction information. This enables the domain controller to use its computing power to perform calculations, thereby enhancing the function of the electronic rearview mirror and further improving safety and comfort during driving.

[0155] Figure 8 This diagram illustrates a structural diagram of an image processing apparatus according to an embodiment of the present application. The apparatus is used in an electronic rearview mirror control unit, such as… Figure 8 As shown, the device includes: The first acquisition module 701 is used to acquire first image data captured by one or more cameras; The first sending module 702 is used to send the first image data to the domain controller DC; The first processing output module 703 is used to process the first image data to obtain processed image data, and output the processed image data to the display device corresponding to the electronic rearview mirror control unit.

[0156] According to the embodiments of this application, the electronic rearview mirror control unit acquires image data collected by the camera and outputs the processed image data to the display device. This allows the camera to be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without relying on the domain controller. Even when the domain controller fails, the electronic rearview mirror function can still operate normally, ensuring the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Simultaneously, by sending the first image data to the domain controller through the electronic rearview mirror control unit, the electronic rearview mirror and the domain controller can share the image data collected by the camera, saving costs and deployment space.

[0157] In one possible implementation, the electronic rearview mirror control unit includes a deserializer and a processing unit. The deserializer is used to connect with the one or more cameras, acquire the first image data, and send the first image data to the processing unit. The processing unit is used to process the first image data to obtain processed image data and output the processed image data to the display device corresponding to the electronic rearview mirror control unit.

[0158] Therefore, the electronic rearview mirror function can be realized independently.

[0159] In one possible implementation, the device further includes: a first receiving module for receiving second image data sent by the domain controller, the second image data being determined by the domain controller based on the first image data and / or sensor data collected by one or more sensors; and a second processing output module for processing the second image data to obtain processed image data and outputting the processed image data to the display device.

[0160] According to the embodiments of this application, the first image data is sent to the domain controller for processing by the electronic rearview mirror control unit to obtain the second image data. This allows the domain controller to use its computing power to perform calculations, thereby enhancing the function of the electronic rearview mirror and further improving safety and comfort during driving.

[0161] In one possible implementation, the first receiving module is configured to: receive the second image data sent by the domain controller via the Controller Area Network (CAN) interface or the Ethernet (ETH) interface.

[0162] This allows for higher transmission efficiency when the electronic rearview mirror control unit receives the second image data.

[0163] In one possible implementation, the field of view of the second image data corresponds to the vehicle driving scene.

[0164] This allows for more refined image information and a better driving experience.

[0165] In one possible implementation, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

[0166] This allows for a wider field of view in the image, resulting in a better driving experience.

[0167] In one possible implementation, the device further includes: a second receiving module for receiving indication information sent by the domain controller, the indication information being determined by the domain controller based on the first image data and / or sensing data collected by one or more sensors; and a third processing output module for outputting corresponding display information to the display device based on the indication information.

[0168] According to the embodiments of this application, the first image data is sent to the domain controller for processing by the electronic rearview mirror control unit to obtain instruction information. This enables the domain controller to use its computing power to perform calculations, thereby enhancing the function of the electronic rearview mirror and further improving safety and comfort during driving.

[0169] In one possible implementation, the first sending module is configured to: send the first image data to the domain controller via the deserializer.

[0170] This enables an electronic rearview mirror control unit to send first image data to a domain controller, allowing the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0171] In one possible implementation, the first sending module is configured to: send the first image data to the domain controller by the processing unit.

[0172] This provides an alternative way for the electronic rearview mirror control unit to send the first image data to the domain controller, enabling the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0173] In one possible implementation, the first transmitting module is configured to: transmit the first image data to the domain controller via a Low Voltage Differential Signaling (LVDS) interface.

[0174] This allows the domain controller to utilize its computing power to perform calculations that enhance the functionality of the electronic rearview mirror and improve the user experience.

[0175] In one possible implementation, the one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera for displaying the electronic rearview mirror image captured by the first camera; a second display device corresponding to the second camera for displaying the electronic rearview mirror image captured by the second camera; and a third display device corresponding to the third camera for displaying the electronic rearview mirror image captured by the third camera.

[0176] Therefore, by utilizing multiple cameras and display devices, the functions of electronic rearview mirrors can be better realized, enhancing the user experience.

[0177] Figure 9 This diagram illustrates a structural diagram of an image processing apparatus according to an embodiment of the present application. The apparatus is used in a domain controller (DC), such as... Figure 9 As shown, the device includes: The third receiving module 801 is used to receive first image data sent by the electronic rearview mirror control unit. The first image data is acquired by the electronic rearview mirror control unit from one or more cameras. The first image data is processed by the electronic rearview mirror control unit to obtain processed image data. The processed image data is output by the electronic rearview mirror control unit to the display device corresponding to the electronic rearview mirror control unit.

[0178] According to the embodiments of this application, by receiving first image data, which is image data acquired by the camera and obtained by the electronic rearview mirror control unit, and processing and outputting this image data to a display device, the camera can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without relying on a domain controller. Even if the domain controller fails, the electronic rearview mirror function can still be implemented normally, ensuring the safety requirements of intelligent driving. Furthermore, in this case, the electronic rearview mirror starts up quickly with low latency, improving the user experience. Simultaneously, by receiving the first image data sent by the electronic rearview mirror control unit, it is also possible to share the image data acquired by the camera with the electronic rearview mirror, saving costs and deployment space.

[0179] In one possible implementation, the device further includes: a second acquisition module for acquiring sensing data collected by one or more sensors; a first determination module for determining second image data based on the first image data and / or the sensing data collected by the one or more sensors; and a second transmission module for transmitting the second image data to the electronic rearview mirror control unit, wherein the second image data is processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is output by the electronic rearview mirror control unit to the display device.

[0180] According to the embodiments of this application, by acquiring sensor data through a domain controller and using the first image data and / or the sensor data to determine the second image data, the domain controller can use its computing power to perform calculations to enhance the function of the electronic rearview mirror and further improve safety and comfort during driving.

[0181] In one possible implementation, the second transmitting module is used to transmit the second image data to the electronic rearview mirror control unit via a controller area network (CAN) interface or an Ethernet (ETH) interface.

[0182] This allows for higher transmission efficiency when the domain controller sends the second image data.

[0183] In one possible implementation, the field of view of the second image data corresponds to the vehicle driving scene.

[0184] This allows for more refined image information and a better driving experience.

[0185] In one possible implementation, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

[0186] This allows for a wider field of view in the image, resulting in a better driving experience.

[0187] In one possible implementation, the device further includes: a third acquisition module for acquiring sensing data collected by one or more sensors; a second determination module for determining indication information based on the first image data and / or the sensing data collected by the one or more sensors; and a third transmission module for sending the indication information to the electronic rearview mirror control unit, wherein the indication information is used by the electronic rearview mirror control unit to output corresponding display information to the display device.

[0188] According to the embodiments of this application, by acquiring sensor data through a domain controller and using the first image data and / or the sensor data to determine indication information, the domain controller can use its computing power to perform calculations to enhance the function of the electronic rearview mirror and further improve safety and comfort during driving.

[0189] In one possible implementation, the electronic rearview mirror control unit includes a deserializer and a third receiving module for receiving the first image data sent by the deserializer.

[0190] This enables a way for a domain controller to receive first image data, allowing the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0191] In one possible implementation, the electronic rearview mirror control unit includes a processing unit and a third receiving module, configured to: receive the first image data sent by the processing unit.

[0192] This allows for an alternative way for the domain controller to receive the first image data, enabling the domain controller to utilize its computing power to perform calculations, thereby enhancing the functionality of the electronic rearview mirror and improving the user experience.

[0193] In one possible implementation, the third receiving module is configured to: receive the first image data sent by the electronic rearview mirror control unit via the Low Voltage Differential Signaling (LVDS) interface.

[0194] This allows the domain controller to utilize its computing power to perform calculations that enhance the functionality of the electronic rearview mirror and improve the user experience.

[0195] In one possible implementation, the one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera for displaying the electronic rearview mirror image captured by the first camera; a second display device corresponding to the second camera for displaying the electronic rearview mirror image captured by the second camera; and a third display device corresponding to the third camera for displaying the electronic rearview mirror image captured by the third camera.

[0196] Therefore, by utilizing multiple cameras and display devices, the functions of electronic rearview mirrors can be better realized, enhancing the user experience.

[0197] Figure 10 A structural diagram of an electronic device 100 according to an embodiment of this application is shown. Figure 10 As shown, the electronic device 100 can be the aforementioned electronic rearview mirror control unit 101 or the aforementioned domain controller 102, performing the aforementioned... Figure 2 Figure 7 The electronic device 100 includes the functions of an electronic rearview mirror control unit or domain controller in any of the image processing methods shown. The electronic device 100 includes at least one processor 1801, at least one memory 1802, and at least one communication interface 1803. Furthermore, the electronic device may also include general-purpose components such as antennas, which will not be detailed here.

[0198] The following is combined with Figure 10 The various components of the electronic device 100 will be described in detail.

[0199] Processor 1801 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme. Processor 1801 may include one or more processing units, such as: processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0200] The communication interface 1803 is used to communicate with other electronic devices or communication networks, such as Ethernet, Radio Access Network (RAN), core network, Wireless Local Area Networks (WLAN), etc. For example, see reference... Figure 2 For example, electronic device 100 may be an electronic rearview mirror control unit 101, and communication interface 1803 may be used to communicate with domain controller 102. Alternatively, electronic device 100 may be a domain controller 102, and communication interface 1803 may be used to communicate with electronic rearview mirror control unit 101.

[0201] The memory 1802 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0202] The memory 1802 stores the application code for executing the above scheme, and its execution is controlled by the processor 1801. The processor 1801 executes the application code stored in the memory 1802.

[0203] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0204] An embodiment of this application provides an image processing apparatus, including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions.

[0205] Embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0206] An embodiment of this application provides a terminal device that can perform the above-described method.

[0207] Embodiments of this application provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0208] Embodiments of this application provide an electronic rearview mirror system, including: one or more cameras; a display device corresponding to the one or more cameras; and an electronic rearview mirror control unit, which can be used to execute the aforementioned related methods.

[0209] Embodiments of this application provide a vehicle including one or more cameras; a display device corresponding to the one or more cameras; an electronic rearview mirror control unit, which can be used to execute the aforementioned related methods; and a domain controller (DC), which can be used to execute the aforementioned related methods.

[0210] An embodiment of this application provides a vehicle, including: the electronic rearview mirror system described above; and a domain controller DC, which can be used to execute the related methods described above.

[0211] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital video disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing.

[0212] The computer-readable program instructions or code described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0213] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of this application.

[0214] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0215] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0216] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0217] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0218] It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using hardware (such as circuits or ASICs (Application Specific Integrated Circuits)) that performs the corresponding function or action, or using a combination of hardware and software, such as firmware.

[0219] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0220] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that, The method is used in an electronic rearview mirror control unit, and the method includes: Acquire first image data from one or more cameras; The first image data is processed to obtain processed image data, and the processed image data is output to the display device corresponding to the electronic rearview mirror control unit.

2. The method according to claim 1, characterized in that, The step of processing the first image data to obtain processed image data and outputting the processed image data to the display device corresponding to the electronic rearview mirror control unit includes: In the event of a failure of the domain controller connected to the electronic rearview mirror control unit, the first image data is processed to obtain processed image data, and the processed image data is output to the display device corresponding to the electronic rearview mirror control unit.

3. The method according to claim 2, further comprising: If the domain controller connected to the electronic rearview mirror control unit is not malfunctioning, the first image data is sent to the domain controller; The domain controller receives second image data, which is determined by the domain controller based on the first image data and / or sensor data collected by one or more sensors. The second image data is processed to obtain processed image data, and the processed image data is output to the display device.

4. The method according to claim 3, characterized in that, Receiving the second image data sent by the domain controller includes: Receive the second image data sent by the domain controller via the Controller Area Network (CAN) interface or the Ethernet (ETH) interface.

5. The method according to claim 3 or 4, characterized in that, The field of view of the second image data corresponds to the vehicle driving scene.

6. The method according to any one of claims 3-5, characterized in that, The field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

7. The method according to any one of claims 2-6, characterized in that, The method further includes: If the domain controller connected to the electronic rearview mirror control unit is not malfunctioning, the system receives an instruction message sent by the domain controller, the instruction message being determined by the domain controller based on the first image data and / or sensor data collected by one or more sensors; The corresponding display information is output to the display device according to the instruction information.

8. The method according to any one of claims 1-7, characterized in that, The electronic rearview mirror control unit includes a deserializer and a processing unit. The deserializer is used to connect with the one or more cameras, acquire the first image data, and send the first image data to the processing unit. The processing unit is used to process the first image data to obtain processed image data, and output the processed image data to the display device corresponding to the electronic rearview mirror control unit.

9. The method according to claim 8, characterized in that, The deserializer is also used to send the first image data to a domain controller connected to the electronic rearview mirror control unit.

10. The method according to claim 8, characterized in that, The The processing unit is also used to send the first image data to a domain controller connected to the electronic rearview mirror control unit.

11. The method according to any one of claims 3-10, characterized in that, Sending the first image data to the domain controller DC includes: The first image data is sent to the domain controller via the Low Voltage Differential Signaling (LVDS) interface.

12. The method according to any one of claims 1-11, characterized in that, The one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera, used to display the electronic rearview mirror image captured by the first camera; A second display device corresponding to the second camera is used to display the electronic rearview mirror image captured by the second camera, and a third display device corresponding to the third camera is used to display the electronic rearview mirror image captured by the third camera.

13. The method according to any one of claims 2-12, characterized in that, The domain controller is used to implement at least the functions of intelligent driving.

14. An image processing method, characterized in that, The method is used for a domain controller (DC), and the method includes: If the domain controller DC is not faulty, the electronic rearview mirror control unit receives first image data sent by the electronic rearview mirror control unit. The first image data is acquired by the electronic rearview mirror control unit from one or more cameras. The first image data is processed by the electronic rearview mirror control unit to obtain processed image data. The processed image data is output by the electronic rearview mirror control unit to the display device corresponding to the electronic rearview mirror control unit.

15. The method according to claim 14, characterized in that, The method further includes: Acquire sensing data collected by one or more sensors; The second image data is determined based on the first image data and / or the sensing data collected by the one or more sensors; The second image data is sent to the electronic rearview mirror control unit. The second image data is processed by the electronic rearview mirror control unit to obtain processed image data, which is then output to the display device by the electronic rearview mirror control unit.

16. The method according to claim 15, characterized in that, Sending the second image data to the electronic rearview mirror control unit includes: The second image data is sent to the electronic rearview mirror control unit via the Controller Area Network (CAN) interface or the Ethernet (ETH) interface.

17. The method according to claim 15 or 16, characterized in that, The field of view of the second image data corresponds to the vehicle driving scene.

18. The method according to any one of claims 15-17, characterized in that, The field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

19. The method according to any one of claims 14-18, characterized in that, The method further includes: Acquire sensing data collected by one or more sensors; Based on the first image data, and / or the sensing data collected by the one or more sensors, determine the indication information; The instruction information is sent to the electronic rearview mirror control unit, and the instruction information is used by the electronic rearview mirror control unit to output corresponding display information to the display device according to the instruction information.

20. The method according to any one of claims 14-19, characterized in that, The electronic rearview mirror control unit includes a deserializer, and the receipt of the first image data sent by the electronic rearview mirror control unit includes: Receive the first image data sent by the deserializer.

21. The method according to any one of claims 14-19, characterized in that, The electronic rearview mirror control unit includes a processing unit, and the receiving of the first image data sent by the electronic rearview mirror control unit includes: The first image data sent by the processing unit is received.

22. The method according to any one of claims 14-21, characterized in that, The first image data received from the electronic rearview mirror control unit includes: The system receives the first image data sent by the electronic rearview mirror control unit via the Low Voltage Differential Signaling (LVDS) interface.

23. The method according to any one of claims 14-22, characterized in that, The one or more cameras include a first camera for capturing images of the left rear of the vehicle, a second camera for capturing images of the right rear of the vehicle, and a third camera for capturing images of the direct rear of the vehicle. The display device includes a first display device corresponding to the first camera, used to display the electronic rearview mirror image captured by the first camera; A second display device corresponding to the second camera is used to display the electronic rearview mirror image captured by the second camera, and a third display device corresponding to the third camera is used to display the electronic rearview mirror image captured by the third camera.

24. An image processing apparatus, characterized in that, The device is used in an electronic rearview mirror control unit, and the device includes: The first acquisition module is used to acquire first image data captured by one or more cameras; The first processing output module is used to process the first image data to obtain processed image data, and output the processed image data to the display device corresponding to the electronic rearview mirror control unit.

25. An image processing apparatus, characterized in that, The device is used for a domain controller (DC), and the device includes: The third receiving module is used to receive first image data sent by the electronic rearview mirror control unit when the domain controller DC is not malfunctioning. The first image data is acquired by the electronic rearview mirror control unit from one or more cameras. The first image data is processed by the electronic rearview mirror control unit to obtain processed image data. The processed image data is output by the electronic rearview mirror control unit to the display device corresponding to the electronic rearview mirror control unit.

26. An image processing apparatus, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1-13, or the method of any one of claims 14-23, when executing the instructions.

27. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-13, or the method described in any one of claims 14-23.

28. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, wherein when the computer-readable code is executed in an electronic device, a processor in the electronic device performs the method of any one of claims 1-13, or performs the method of any one of claims 14-23.

29. An electronic rearview mirror system, comprising: One or more cameras; Display devices corresponding to the one or more cameras; And an electronic rearview mirror control unit, the electronic rearview mirror control unit being used to perform the method according to any one of claims 1-13.

30. A vehicle, characterized in that, include: One or more cameras; Display devices corresponding to the one or more cameras; An electronic rearview mirror control unit, configured to perform the method according to any one of claims 1-13; A domain controller (DC) is used to perform the method according to any one of claims 14-23.

Images