Vehicle-mounted cabin driving integrated image processing system and vehicle

Abstract

The present disclosure provides a vehicle-mounted cabin driving integrated image processing system and a vehicle. The system comprises a driving domain image signal input module, a cabin domain image signal input module, a driving domain image signal processing module, a cabin domain image signal processing module and a memory. The driving domain image signal input module is used for storing driving domain image signals to the memory. The cabin domain image signal input module is used for storing cabin domain image signals to the memory. The driving domain image signal processing module is used for reading driving domain image signals from the memory and processing to obtain driving domain images, and storing the driving domain images to the memory. The cabin domain image signal processing module is used for reading cabin domain image signals from the memory and processing to obtain cabin domain images, and storing the cabin domain images to the memory. The system occupies less space, has shorter data transmission delay, lower cost, avoids data transmission interference caused by too long lines, and data transmission errors caused by line faults.

Description

Technical Field

[0001] This disclosure relates to intelligent vehicle technology, and in particular to an in-vehicle cockpit integrated image processing system and vehicle. Background Technology

[0002] Currently, intelligent vehicles typically separate image processing systems for the driving domain and the cockpit domain. These two systems are located in different spaces within the vehicle and connected by wiring to facilitate the transmission of image data between the driving and cockpit domains. This architecture increases costs due to the need for additional hardware (such as multiple ECUs and wiring harnesses), resulting in higher manufacturing and maintenance costs. Cross-domain data exchange via a bus leads to significant data transmission latency, impacting real-time performance. Furthermore, the two image processing systems occupy interior space, hindering lightweight vehicle design. Summary of the Invention

[0003] To address the aforementioned technical issues, this disclosure provides an in-vehicle cockpit-driver integrated image processing system and vehicle, thereby resolving the problems of large space occupation, high data transmission latency, and high cost caused by separating the intelligent driving domain image processing system and the cockpit domain image processing system.

[0004] A first aspect of this disclosure provides an in-vehicle cockpit-driver integrated image processing system, comprising: a driver domain image signal input module, a cockpit domain image signal input module, a driver domain image signal processing module, a cockpit domain image signal processing module, and a memory; the output terminals of the driver domain image signal input module and the cockpit domain image signal input module are respectively connected to the memory; the input terminals of the driver domain image signal processing module and the cockpit domain image signal processing module are respectively connected to the memory; the driver domain image signal input module is used to store driver domain image signals acquired by a driver domain camera into the memory; the cockpit domain image signal input module is used to store cockpit domain image signals acquired by a cockpit domain camera into the memory; the driver domain image signal processing module is used to read driver domain image signals from the memory, process the driver domain image signals to obtain a driver domain image, and store the driver domain image into the memory; the cockpit domain image signal processing module is used to read cockpit domain image signals from the memory, process the cockpit domain image signals to obtain a cockpit domain image, and store the cockpit domain image into the memory.

[0005] A second aspect of this disclosure provides a vehicle, including: at least one driver-side camera, at least one cabin-side camera, and the aforementioned vehicle-mounted integrated driver-cabin image processing system, wherein the at least one driver-side camera is disposed outside the vehicle's cabin, and the at least one cabin-side camera is disposed inside the vehicle's cabin; the at least one driver-side camera is connected to the input terminal of the driver-side image signal input module in the vehicle-mounted integrated driver-cabin image processing system, and the at least one cabin-side camera is connected to the input terminal of the cabin-side image signal input module in the vehicle-mounted integrated driver-cabin image processing system.

[0006] Based on the vehicle-mounted cockpit-driver integrated image processing system and vehicle provided in this disclosure, the system utilizes a driver-domain image signal input module to store driver-domain image signals acquired by a driver-domain camera into a memory, and a cockpit-domain image signal input module to store cockpit-domain image signals acquired by a cockpit-domain camera into a memory. The driver-domain image signal processing module and the cockpit-domain image signal processing module respectively read the driver-domain image signals and cockpit-domain image signals from the memory, process them, and obtain driver-domain images and cockpit-domain images. The driver-domain images are then stored in the memory. This integrates cockpit-domain image processing and driver-domain image processing functions into the same hardware system, resulting in shorter cross-domain data exchange transmission distances, smaller system footprint, shorter data transmission latency, and lower costs. In this system, cockpit-domain images and driver-domain images are processed by different hardware modules, without interference, reducing the incidence of data transmission errors. Furthermore, the cockpit-domain and driver-domain images share a single memory, thus avoiding data transmission interference caused by excessively long lines and data transmission errors caused by line faults during cross-domain data transmission. Attached Figure Description

[0007] Figure 1 This is a structural diagram of an integrated vehicle-mounted cockpit image processing system provided in an exemplary embodiment of this disclosure; Figure 2 This is a structural diagram of an integrated vehicle-mounted cockpit image processing system provided in another exemplary embodiment of this disclosure; Figure 3 This is a structural diagram of an integrated vehicle-mounted cockpit image processing system provided in yet another exemplary embodiment of this disclosure; Figure 4 This is a structural diagram of an integrated vehicle-mounted cockpit image processing system provided in yet another exemplary embodiment of this disclosure; Figure 5 This is a structural diagram of an integrated vehicle-mounted cockpit image processing system provided in yet another exemplary embodiment of this disclosure; Figure 6 This is a structural diagram of an integrated vehicle-mounted cockpit image processing system provided in yet another exemplary embodiment of this disclosure; Figure 7This is a structural diagram of a vehicle provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0008] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0009] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0010] Application Overview Currently, intelligent vehicles typically separate image processing systems for the driving domain and the cockpit domain. These two systems are located in different spaces within the vehicle and connected by wiring to facilitate the transmission of image data between the driving and cockpit domains. This architecture increases costs due to the need for additional hardware (such as multiple ECUs and wiring harnesses), resulting in higher manufacturing and maintenance costs. Cross-domain data exchange via a bus leads to significant data transmission latency, impacting real-time performance. Furthermore, the two image processing systems occupy interior space, hindering lightweight vehicle design.

[0011] To address the aforementioned issues, the vehicle-mounted cockpit-driver integrated image processing system provided in this disclosure integrates the cockpit-domain image processing system and the driver-domain image processing system into a single system. These systems are hardware-isolated from each other and store data in the same memory. Cross-domain data exchange does not require long-distance buses, resulting in a smaller system footprint, shorter data transmission latency, and lower cost. In this system, cockpit-domain and driver-domain images are processed by different hardware modules, ensuring they do not interfere with each other and reducing the incidence of data transmission errors.

[0012] Exemplary System Figure 1 A structural diagram of an in-vehicle cockpit integrated image processing system to which embodiments of the present disclosure can be applied is shown.

[0013] like Figure 1 As shown, the vehicle-mounted cockpit-driver integrated image processing system 100 may include: a driver-domain image signal input module 101, a cockpit-domain image signal input module 102, a driver-domain image signal processing module 103, a cockpit-domain image signal processing module 104, and a memory 105. This system can be housed within an integrated circuit; for example, the modules can be housed within a System-on-Chip (SoC). Optionally, the system can also be mounted on a circuit board, with the modules being discrete components connected by wires.

[0014] In this embodiment, the driver domain image signal input module 101 and the cabin domain image signal input module 102 are hardware isolated from each other, and the driver domain image signal processing module 103 and the cabin domain image signal processing module 104 are also hardware isolated from each other, thereby avoiding mutual interference between cabin domain signals and driver domain signals.

[0015] The output terminals of the driving area image signal input module 101 and the cabin area image signal input module 102 are respectively connected to the memory 105; the input terminals of the driving area image signal processing module 103 and the cabin area image signal processing module 104 are respectively connected to the memory 105.

[0016] The driving area image signal input module 101 is used to store the driving area image signal collected by the driving area camera into the memory 105; the cabin area image signal input module 102 is used to store the cabin area image signal collected by the cabin area camera into the memory 105.

[0017] The aforementioned driving domain image signal input module 101 can directly store the raw signal acquired by the driving domain camera as a driving domain image signal in the storage; alternatively, it can process the raw signal according to a preset processing method (e.g., sequentially processing it through a serial decoder, a MIPI data transmission interface, and a preprocessing unit) to obtain a driving domain image signal, and then store it in the memory 105. The cabin domain image signal input module 102 can directly store the raw signal acquired by the driving domain camera as a driving domain image signal in the storage; alternatively, it can process the raw signal according to a preset processing method (e.g., sequentially processing it through a serial decoder, a MIPI data transmission interface, and a preprocessing unit) to obtain a driving domain image signal, and then store it in the memory 105.

[0018] The aforementioned memory 105 can be of various types, such as DDR (Double Data Rate Synchronous Dynamic Random Access Memory) memory 105, or other types of dynamic random access memory 105. This memory 105 is shared by the driving area image signal input module 101, the cabin area image signal input module 102, the driving area image signal processing module 103, and the cabin area image signal processing module 104.

[0019] The driving domain image signal processing module 103 is used to read the driving domain image signal from the memory 105, process the driving domain image signal to obtain the driving domain image, and store the driving domain image in the memory 105.

[0020] The cabin area image signal processing module 104 is used to read cabin area image signals from memory 105, process cabin area image signals to obtain cabin area images, and store cabin area images in memory 105.

[0021] The aforementioned driver domain image signal processing module 103 and cabin domain image signal processing module 104 can each be a dedicated hardware module for performing image processing tasks. For example, driver domain image signal processing module 103 and cabin domain image signal processing module 104 can be image signal processors (ISPs).

[0022] As an example, the driving domain image signal and cabin domain image signal stored in memory 105 can be RAW images captured by a camera. The driving domain image signal processing module 103 and the cabin domain image signal processing module 104 can perform a series of operations and optimizations on the RAW images respectively, and finally output image data that conforms to the characteristics of human vision or specific application requirements. For example, the driving domain image and cabin domain image generated by the driving domain image signal processing module 103 and the cabin domain image signal processing module 104 can be YUV format images. The vehicle-mounted cockpit-driver integrated image processing system provided in this embodiment integrates cockpit-driver image processing and driver-driver image processing functions into the same hardware system. This results in shorter cross-domain data exchange transmission distances, smaller system footprint, shorter data transmission latency, and lower cost. The system utilizes a driver-domain image signal input module 101 to store driver-domain image signals captured by a driver-domain camera into a memory 105, and a cockpit-domain image signal input module 102 to store cockpit-domain image signals captured by a cockpit-domain camera into the memory 105. In this system, the cabin domain image and the driving domain image are processed by different hardware modules, which do not interfere with each other, reducing the incidence of data transmission errors. Furthermore, the cabin domain and the driving domain share a single memory 105, thus avoiding data transmission interference caused by excessively long lines and data transmission errors caused by line faults during cross-domain data transmission.

[0023] In some alternative implementations, such as Figure 2 As shown, the driving domain image signal input module 101 includes a driving domain image signal interface 1011 and a driving domain image signal preprocessing unit 1012; the cabin domain image signal input module 102 includes a cabin domain image signal interface 1021 and a cabin domain image signal preprocessing unit 1022.

[0024] The output of the driving domain image signal interface 1011 is connected to the driving domain image signal preprocessing unit 1012, and the output of the driving domain image signal preprocessing unit 1012 is connected to the memory 105; the output of the cabin domain image signal interface 1021 is connected to the cabin domain image signal preprocessing unit 1022, and the output of the cabin domain image signal preprocessing unit 1022 is connected to the memory 105.

[0025] The driving domain image signal interface 1011 is used to receive the original driving domain image signal captured by the driving domain camera and send the original driving domain image signal to the driving domain image signal preprocessing unit 1012.

[0026] The driving domain image signal preprocessing unit 1012 is used to preprocess the original driving domain image signal to obtain the driving domain image signal.

[0027] The cabin area image signal interface 1021 is used to receive the original cabin area image signal collected by the cabin area camera and send the original cabin area image signal to the cabin area image signal preprocessing unit 1022.

[0028] The cabin area image signal preprocessing unit 1022 is used to preprocess the original cabin area image signal to obtain the cabin area image signal.

[0029] The aforementioned driver area image signal interface 1011 and cabin area image signal interface 1021 may include various interfaces for transmitting image signals. For example, Figure 2 The driving domain image signal interface 1011 and cabin domain image signal interface 1021 shown may respectively include SerDes (serializer / deserializer) and MIPI (Mobile Industry Processor Interface) data transmission interfaces.

[0030] The aforementioned driver-area image signal preprocessing unit 1012 and cabin-area image signal preprocessing unit 1022 can perform preliminary processing on the raw image signals acquired by the camera before they are transmitted to the corresponding image signal processing modules, aiming to improve the quality and efficiency of subsequent image processing. For example, the driver-area image signal preprocessing unit 1012 and cabin-area image signal preprocessing unit 1022 can each be a PreISP (Image Signal Preprocessor), which can perform processing on the input raw image such as black level correction, bad pixel correction, lens shading correction, automatic white balance, automatic exposure control, and preliminary noise reduction.

[0031] This embodiment achieves complete hardware isolation between the driver domain image signal input module 101 and the cabin domain image signal preprocessing unit 1022 by setting a driver domain image signal interface 1011 and a cabin domain image signal preprocessing unit 1022 in the driver domain image signal input module 101, and a cabin domain image signal interface 1021 and a cabin domain image signal preprocessing unit 1022 in the cabin domain image signal input module 102. The image signal transmission and preprocessing are performed by their own image signal interfaces and image signal preprocessing units, respectively, thereby avoiding interference between cabin domain image signals and driver domain image signals and improving the accuracy of image processing.

[0032] In some alternative implementations, such as Figure 3 As shown, the system also includes a driver-specific image signal processing module 106. The type of the driver-specific image signal processing module 106 can be the same as that of the driver-specific image signal processing module 103 described above. The driver-specific image signal processing module 106 can be a module separately installed in the system, for example, it can form a cockpit-driver integrated chip with other modules.

[0033] The input terminal of the dedicated driving domain image signal processing module 106 is connected to the output terminal of the driving domain image signal input module 101. The dedicated driving domain image signal processing module 106 is used to process the driving domain image signal acquired by the first target driving domain camera to obtain a dedicated driving domain image.

[0034] The aforementioned first target driving domain camera can be a pre-set camera used to collect images of the vehicle's driving environment in real time, such as a camera located in the center above the vehicle's windshield.

[0035] The obtained driving domain-specific image can be stored in the aforementioned memory 105, or it can be directly sent to the driving domain processor used to process the driving domain-specific image. For example... Figure 3 As shown, the input terminal of the driving domain dedicated image signal processing module 106 is connected to the memory 105. The generated driving domain dedicated image is stored in the memory 105, and the driving domain processor can obtain the driving domain image from the memory 105.

[0036] The dedicated image signal processing module 106 for the driving domain can directly receive driving domain image signals from the driving domain image signal input module 101 without needing to read driving domain image signals from the memory 105. Therefore, this dedicated image signal processing module 106 can be applied in scenarios with high real-time requirements for image processing. For example, when the vehicle is in motion, the dedicated image signal processing module 106 enters online image processing mode, receiving driving domain image signals in real time to provide images for intelligent driving functions. Correspondingly, the aforementioned driving domain image signal processing module 103 needs to read driving domain image signals from the memory 105. Therefore, the driving domain image signal processing module 103 can be applied in scenarios with lower real-time requirements for image processing; that is, the driving domain image signal processing module 103 can perform image processing functions offline.

[0037] This embodiment enables online image processing by setting up a dedicated image signal processing module 106 for the driving domain, while other image signal processing modules 103 for the driving domain can perform offline image processing. This allows for the execution of corresponding image processing operations for both online and offline states, meeting the needs of the vehicle driving domain for online real-time image acquisition and improving vehicle driving safety.

[0038] In some alternative implementations, such as Figure 3 As shown, the system also includes a cabin-specific image signal processing module 107. The type of cabin-specific image signal processing module 107 can be the same as that of the cabin-specific image signal processing module 104 described above. This cabin-specific image signal processing module 107 can be a module separately installed in the system, for example, it can form a cabin-riding integrated chip with other modules.

[0039] The input terminal of the cabin-specific image signal processing module 107 is connected to the output terminal of the cabin-specific image signal input module 102; the cabin-specific image signal processing module 107 is used to process the cabin-specific image signal acquired by the target cabin-specific camera to obtain a cabin-specific image.

[0040] The obtained cabin-specific images can be stored in the aforementioned memory 105, or they can be directly sent to the cabin-specific processor used to process the cabin-specific images. For example... Figure 3 As shown, the input terminal of the cabin-specific image signal processing module 107 is connected to the memory 105. The generated cabin-specific image is stored in the memory 105, and the cabin processor can obtain the cabin image from the memory 105.

[0041] Similar to the driver-specific image signal processing module 106 described above, the cabin-specific image signal processing module 107 can directly receive cabin image signals from the cabin image signal input module 102 without needing to read cabin image signals from the memory 105. Therefore, the cabin-specific image signal processing module 107 can be applied in scenarios with high real-time requirements for image processing. For example, in scenarios requiring interaction with occupants, the cabin-specific image signal processing module 107 enters online image processing mode, receiving cabin image signals in real time to support in-cabin perception, interaction, safety, entertainment, and other functions. Correspondingly, the cabin image signal processing module 104, which needs to read cabin image signals from the memory 105, can be applied in scenarios with lower real-time requirements for image processing; that is, the cabin image signal processing module 104 can perform image processing functions offline.

[0042] This embodiment enables online image processing by setting up a dedicated cabin area image signal processing module 107, while other cabin area image signal processing modules 104 can perform offline image processing. This allows for the execution of corresponding image processing operations for both online and offline states, meeting the vehicle cabin area's need for online real-time image acquisition and improving vehicle driving safety.

[0043] In some alternative implementations, such as Figure 4 As shown, the system also includes a driver domain processor 108 and a cabin domain processor 109; the driver domain processor 108 and the cabin domain processor 109 are connected to the memory 105.

[0044] In this embodiment, the driving domain processor 108 and the cabin domain processor 109 can also be connected to communicate, so that communication can be performed between the driving domain processor 108 and the cabin domain processor 109 when the image signal is processed across domains.

[0045] The driving domain processor 108 is used to read driving domain images from memory 105 and perform driving operations based on the driving domain images. The driving operations may include various operations to support vehicle driving, such as environmental perception, target recognition, high-precision positioning, path planning, vehicle control command output, etc.

[0046] The cabin area processor 109 is used to read cabin area images from the memory 105 and, upon receiving a cabin area image display command, send the cabin area images to the display device for display. This cabin area processor 109 can provide interactive functions with users inside the cabin; when a user needs to view images captured by the cabin area camera, the cabin area controller can retrieve and display the desired image.

[0047] The aforementioned driver domain processor 108 and cabin domain processor 109 are integrated into the same hardware device as other modules. For example, the driver domain processor 108 and cabin domain processor 109 can be integrated into a single integrated driver-cabin chip with other modules.

[0048] This embodiment improves system integration by setting up a driver domain processor 108 and a cabin domain processor 109 in the system. The driver domain processor 108 and the cabin domain processor 109 are hardware isolated, that is, they only retrieve the corresponding images from the memory 105, avoiding mutual interference.

[0049] In some alternative implementations, the cabin area processor 109 is also used for: In response to receiving a driving domain image display command, the system reads the driving domain image to be displayed corresponding to the driving domain image display command from the memory 105 and sends the driving domain image to be displayed to the display device for display.

[0050] The driving domain image to be displayed is the driving domain image corresponding to the second target driving domain camera. This second target driving domain camera may be the same as or different from the first target driving domain camera. For example, the second target driving domain camera may be a pre-set camera used for capturing panoramic images.

[0051] Typically, in some scenarios, there is a need to display images of the vehicle's external environment. The driving domain images captured by the aforementioned second target driving domain camera can be reused in the cabin domain. After receiving the driving domain image display instruction, the cabin domain processor 109 can directly read the corresponding image from the memory 105 for display.

[0052] This embodiment enables the reuse of driving domain images from the second target driving domain camera for display in the cabin domain. Data transmission is not required via a bus; the cabin domain processor 109 can simply read the data from the memory 105, thereby further reducing the risk of data transmission errors.

[0053] In some alternative implementations, the cabin area processor 109 is also used for: If the cabin area image signal processing module 104 does not currently meet the first preset image processing conditions, determine whether the driving area image signal processing module 103 currently meets the second preset image processing conditions.

[0054] The first preset image processing condition is a trigger condition used to reuse the cabin image signal processing module 103 to process the cabin image signal. For example, if the cabin image signal processing module 104 is currently in a non-idle state, or if the current data processing bandwidth of the cabin image signal processing module 104 is insufficient, resulting in data congestion, then it can be determined that the cabin image signal processing module 104 does not meet the first preset image processing condition.

[0055] The aforementioned second preset image processing condition is a condition set to enable the driving domain image signal processing module 103 to process cabin domain image signals. For example, if the driving domain image signal processing module 103 is currently in an idle state, it is determined that the driving domain image signal processing module 103 meets the second preset image processing condition.

[0056] If the driving domain image signal processing module 103 meets the second preset image processing conditions, a cabin domain image signal processing instruction is sent to the driving domain image signal processing module 103.

[0057] When the driving domain image signal processing module 103 meets the second preset image processing conditions, the driving domain image signal processing module 103 can perform processing of the cabin domain image signal. At this time, the cabin domain processor 109 can send cabin domain image signal processing instructions to the driving domain image signal processing module 103 through the cabin domain processor 109.

[0058] It should be noted that when this system includes the driving domain dedicated image signal processing module 106 used in the online state as described in the above embodiments, this embodiment requires the driving domain image processing module in the offline state to process the cabin domain image signal to ensure the real-time signal processing capability of the driving domain dedicated image signal processing module 106 in the online state.

[0059] This embodiment enables the driving domain image signal processing module 103 to be reused to process cabin domain image signals under certain conditions, thereby further improving the utilization rate of the driving domain image signal processing module 103, and helping to reduce the risk of image signal congestion and improve the speed of image signal processing.

[0060] In some optional implementations, based on the driving domain image signal processing module 103 reused in the above embodiments, the driving domain image signal processing module 103 is further used for: In response to receiving a cabin area image signal processing instruction, the target cabin area image signal indicated by the cabin area image signal processing instruction is read from the memory 105, and the target cabin area image signal is processed to obtain the target cabin area image to be displayed.

[0061] The cabin area image signal processing module 103, based on the cabin area image signal processing instructions, can determine which cabin area camera's captured cabin area image needs to be displayed in the cabin area, and thus determine the corresponding target cabin area image signal. The target cabin area image obtained after processing the target cabin area image signal can be stored in the memory 105. The cabin area processor 109 can retrieve the target cabin area image from the memory 105 and display it.

[0062] Typically, because the image processing function performed by the driver domain image signal processing module 103 needs to meet driving safety requirements, it can process the cabin domain image signal in a manner that meets these requirements. However, the cabin domain image signal processing module 104 does not have driving safety requirements; therefore, the driver domain image signal cannot be processed by the cabin domain image signal processing module 104, which lacks driving safety support. As an example, the driver domain image signal processing module 103 can process the driver image signal captured by the cabin domain camera, and then the driver domain processor 108 can further perform fatigue driving recognition on the driver image to improve driving safety.

[0063] This embodiment realizes that the driver domain image signal processing module 103 processes the target cabin domain image signal based on the cabin domain image signal processing instructions issued by the cabin domain processor 109, thereby realizing more flexible and efficient data transmission between the cabin domain and the driver domain, and improving the image processing efficiency of the cabin-driver integrated image processing system.

[0064] In some alternative implementations, the cabin area image signal processing module 104 is further configured to: In response to receiving a driving domain image signal processing instruction sent by the driving domain processor 108, the target driving domain image signal indicated by the driving domain image signal processing instruction is extracted from the memory 105, and the target driving domain image signal is processed to obtain the target driving domain image to be displayed.

[0065] Specifically, when it is necessary to display the driving domain image captured by the driving domain camera, the driving domain processor 108 can send a driving domain image signal processing instruction to the cabin domain image signal processing module 104 through the cabin domain processor 109. The cabin domain image signal processing module 104 retrieves the corresponding target driving domain image signal from the memory 105 for processing, so that the processed target driving domain image can be displayed in the cabin.

[0066] This embodiment enables the cabin image signal processing module 104 to extract the corresponding target driving domain image signal from the memory 105 for processing when the driving domain image needs to be displayed. This eliminates the need to transmit image signals between the driving domain and the cabin domain via a bus, thereby avoiding interference with the driving domain image and improving the efficiency of driving domain image display.

[0067] In some alternative implementations, the cabin area image signal input module 102 is also used for: It receives sensor data collected by a sensor of a preset type and stores the sensor data in the memory 105 in a transparent manner.

[0068] Transmission via pass-through means that sensor data is transmitted only through the cabin image signal input module 102 without any processing, and is directly stored in the memory 105. In this embodiment, at least one sensor data input interface can be provided in the cabin image signal input module 102 to receive various types of sensor data. For example, sensor data may include depth data acquired from a depth camera, point cloud data acquired from a lidar, etc.

[0069] like Figure 5 As shown, the cabin area image signal input module 102 can be connected to the sensor. The cabin area image signal input module 102 does not process the sensor data and directly outputs the sensor data to the memory 105.

[0070] This embodiment further enriches the types of data transmitted by the system and expands the application scenarios of the system by setting the data pass-through function of the cabin image signal input module 102.

[0071] In summary, the above embodiments, Figure 6 Another system architecture diagram of the vehicle-mounted cockpit integrated image processing system provided in an embodiment of this disclosure is shown. For example... Figure 6 As shown, the system comprises a driver domain section and a cabin domain section. The driver domain section includes three driver domain image signal input modules and three driver domain image signal processing modules (ISP0-ISP2). Each driver domain image signal input module includes a SerDes (serializer / deserializer), a MIPI SCI RX interface, and a PreISP (image signal preprocessing unit). Each driver domain image signal input module receives driver domain image signals acquired by four driver domain cameras. In the diagram, CAM0-CAM11 represent the driver domain cameras, and CAM0 RAW-CAM11 RAW in the DDR memory represent the driver domain image signals.

[0072] Among them, CAM0-2 consists of three forward-facing cameras. The signals acquired by CAM0 and CAM1 are input through MIPI RX0 and PreISP0, while the signals acquired by CAM2 are input through MIPI RX2 and PreISP2, and all are output to DDR memory. The signals acquired by CAM0 are transmitted to ISP0 in real time while online, outputting YUV format driving-domain images for subsequent driving-domain processing. ISP0 is the dedicated image signal processing module for the driving-domain, and CAM0 is the first target driving-domain camera.

[0073] The signal acquired by CAM1 is output to the DDR memory by PreISP1, and processed by ISP1 in offline mode, outputting a YUV format driving domain image for subsequent driving domain processing; the signal acquired by CAM2 is output to the DDR memory by PreISP2, and processed by ISP2 in offline mode, outputting a YUV format driving domain image for subsequent driving domain processing.

[0074] The signals acquired by CAM3 are input through MIPI RX2 and PreISP2 and output to DDR memory. Data is read from DDR memory and sent to ISP3 in the cabin area for processing, outputting a YUV format driving-area image for subsequent driving-area processing; simultaneously, the driving-area image corresponding to CAM3 is multiplexed for display in the cabin area. CAM3 is the second target driving-area camera.

[0075] The signals acquired by CAM4-7 are input through MIPI RX1 and PreISP1 and output to DDR memory. The driving domain image signals of CAM4-5 are processed by ISP1, which outputs YUV format driving domain images for subsequent driving domain processing; the driving domain image signals of CAM6-7 are processed by ISP2, which outputs YUV format driving domain images for subsequent driving domain processing.

[0076] The signals acquired by CAM8-9 are connected to MIPI RX0 and PreISP0, while the signals acquired by CAM10-11 are connected to MIPI RX2 and PreISP2, and output to DDR memory. The driving domain image signals from CAM8-11 are sent to ISP2 for processing, and output YUV format driving domain images for subsequent driving domain processing.

[0077] The cabin area section includes two cabin area image signal input modules and one cabin area image signal processing module (ISP3). Each cabin area image signal input module includes a SerDes, a MIPI SCI RX interface, and a PreISP. In the diagram, CAM12-CAM14 are cabin area cameras. SENSOR0 and SENSOR1 are sensors. CAM12RAW-CAM14RAW in the DDR memory contain cabin area image signals, and S0 and S1 contain sensor data.

[0078] The signals acquired by CAM12 are input from MIPI RX3 and PREISP3 and output to DDR memory. ISP3 processes the signals and outputs YUV format cabin area images for subsequent driving area processing. At the same time, the images can be read from DDR memory and displayed in the cabin area.

[0079] The signals acquired by CAM13 are input from MIPI RX3 and PreISP3, and additional DPC and RGBIrremosaic processing are performed in PreISP3. RAW data or IR mono data is output to DDR memory, which is then processed by ISP3 to output YUV format cabin area images or IR mono images for use in cabin video telephone, display and other applications.

[0080] The signals acquired by CAM14 are input to MIPI RX3 and PreISP3, output to DDR memory, processed by ISP3, and output in YUV format as cabin area images for display.

[0081] The data output by SENSOR0 is connected to MIPI RX3 and PREISP3, and is output to DDR memory without processing.

[0082] SENSOR1: The output data is connected to MIPI RX4 and PREISP4, and is output to DDR memory without processing.

[0083] Figure 7 A schematic diagram of the structure of a vehicle 700 provided in an embodiment of this disclosure is shown. The vehicle includes: at least one driver-side camera 701, at least one cabin-side camera 702, and the aforementioned vehicle-mounted integrated driver-cabin image processing system 100.

[0084] In this vehicle, at least one driver-centric camera 701 is installed outside the vehicle's cabin, and at least one cabin-centric camera 702 is installed inside the vehicle's cabin. The driver-centric camera can capture images of the external environment of the vehicle, and the resulting driver-centric images can be used for intelligent driving. The cabin-centric camera can capture images of the vehicle's interior, and the resulting cabin-centric images can be used for tasks such as facial recognition and motion recognition.

[0085] At least one driver-field camera 701 is connected to the input terminal of the driver-field image signal input module in the vehicle-mounted cockpit integrated image processing system 100, and at least one cabin-field camera 702 is connected to the input terminal of the cabin-field image signal input module in the vehicle-mounted cockpit integrated image processing system 100.

[0086] The vehicle provided in this embodiment integrates cabin-domain image processing and driver-domain image processing functions into the same hardware system by applying the above-mentioned vehicle-mounted cabin-driver integrated image processing system. This results in shorter transmission distances for cross-domain data exchange, smaller system footprint, shorter data transmission latency, and lower cost. Furthermore, since the cabin and driver domains share a single memory, data transmission interference caused by excessively long lines and data transmission errors caused by line faults can be avoided during cross-domain data transmission.

[0087] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0088] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A vehicle-mounted cockpit integrated image processing system, comprising: The system includes a driver's area image signal input module, a cabin area image signal input module, a driver's area image signal processing module, a cabin area image signal processing module, and a memory. The output terminals of the driving area image signal input module and the cabin area image signal input module are respectively connected to the memory. The input terminals of the driving area image signal processing module and the cabin area image signal processing module are respectively connected to the memory; The driving domain image signal input module is used to store the driving domain image signal captured by the driving domain camera into the memory; The cabin area image signal input module is used to store cabin area image signals captured by the cabin area camera into the memory; The driving domain image signal processing module is used to read the driving domain image signal from the memory, process the driving domain image signal to obtain a driving domain image, and store the driving domain image in the memory; The cabin area image signal processing module is used to read the cabin area image signal from the memory, process the cabin area image signal to obtain the cabin area image, and store the cabin area image in the memory.

2. The system according to claim 1, wherein, The driving domain image signal input module includes a driving domain image signal interface and a driving domain image signal preprocessing unit; the cabin domain image signal input module includes a cabin domain image signal interface and a cabin domain image signal preprocessing unit; The output terminal of the driving domain image signal interface is connected to the driving domain image signal preprocessing unit, and the output terminal of the driving domain image signal preprocessing unit is connected to the memory; The output terminal of the cabin area image signal interface is connected to the cabin area image signal preprocessing unit, and the output terminal of the cabin area image signal preprocessing unit is connected to the memory; The driving domain image signal interface is used to receive the original driving domain image signal captured by the driving domain camera, and send the original driving domain image signal to the driving domain image signal preprocessing unit; The driving domain image signal preprocessing unit is used to preprocess the original driving domain image signal to obtain the driving domain image signal; The cabin area image signal interface is used to receive the original cabin area image signal captured by the cabin area camera, and send the original cabin area image signal to the cabin area image signal preprocessing unit; The cabin area image signal preprocessing unit is used to preprocess the original cabin area image signal to obtain the cabin area image signal.

3. The system according to claim 1, wherein, The system also includes a dedicated image signal processing module for the driving domain; The input terminal of the driving domain dedicated image signal processing module is connected to the output terminal of the driving domain image signal input module; The dedicated driving domain image signal processing module is used to process the driving domain image signal acquired by the first target driving domain camera to obtain a dedicated driving domain image.

4. The system according to claim 1, wherein, The system also includes a cabin-specific image signal processing module; The input terminal of the cabin-specific image signal processing module is connected to the output terminal of the cabin image signal input module; The cabin-specific image signal processing module is used to process the cabin image signals acquired by the target cabin camera to obtain cabin-specific images.

5. The system according to claim 1, wherein, The system also includes a driving domain processor and a cabin domain processor; the driving domain processor and the cabin domain processor are connected to the memory. The driving domain processor is used to read the driving domain image from the memory and perform driving operations based on the driving domain image; The cabin area processor is used to read the cabin area image from the memory, and upon receiving a cabin area image display instruction, send the cabin area image to the display device for display.

6. The system according to claim 5, wherein, The cabin area processor is also used for: In response to receiving a driving domain image display command, the system reads the driving domain image to be displayed corresponding to the driving domain image display command from the memory and sends the driving domain image to be displayed to the display device for display, wherein the driving domain image to be displayed is a driving domain image corresponding to the second target driving domain camera.

7. The system according to claim 5, wherein, The cabin area processor is also used for: If the cabin area image signal processing module does not currently meet the first preset image processing conditions, determine whether the driving area image signal processing module currently meets the second preset image processing conditions. If the driving domain image signal processing module meets the second preset image processing conditions, a cabin domain image signal processing instruction is sent to the driving domain image signal processing module.

8. The system according to claim 7, wherein, The driving domain image signal processing module is also used for: In response to receiving a cabin area image signal processing instruction, the system reads the target cabin area image signal indicated by the instruction from the memory and processes the target cabin area image signal to obtain the target cabin area image to be displayed.

9. The system according to claim 5, wherein, The cabin area image signal processing module is also used for: In response to receiving a driving domain image signal processing instruction sent by the driving domain processor, the target driving domain image signal indicated by the driving domain image signal processing instruction is retrieved from the memory, and the target driving domain image signal is processed to obtain a target driving domain image to be displayed.

10. The system according to any one of claims 1-9, wherein, The cabin area image signal input module is also used for: Receive sensor data collected by a sensor of a preset type and store the sensor data in the memory in a transparent manner.

11. A vehicle comprising: At least one driver-side camera, at least one cabin-side camera, and the vehicle-mounted integrated driver-cabin image processing system according to any one of claims 1-10, wherein the at least one driver-side camera is disposed outside the vehicle's cabin, and the at least one cabin-side camera is disposed inside the vehicle's cabin. The at least one driving-domain camera is connected to the input terminal of the driving-domain image signal input module in the vehicle-mounted cockpit integrated image processing system, and the at least one cabin-domain camera is connected to the input terminal of the cabin-domain image signal input module in the vehicle-mounted cockpit integrated image processing system.

Images