Image processing method and apparatus, and vehicle

Abstract

The present application is applicable to the field of intelligent cockpits. Provided are an image processing method and apparatus, and a vehicle. The method comprises: acquiring an image and / or sensing data; on the basis of the image and / or the sensing data, determining a weight vector, wherein N weights in the weight vector correspond to N lookup tables (LUTs) in a color LUT group on a one-to-one basis, and N is an integer greater than 1; on the basis of the weight vector and the N LUTs in the LUT group, determining a target LUT; on the basis of the target LUT, processing the image, so as to obtain a processed image; and controlling a display apparatus to display the processed image. The present application is applicable to intelligent vehicles or electric vehicles, and is conducive to improving users' image viewing experience.

Description

Image processing methods, apparatus and vehicles Technical Field

[0001] This application relates to the field of smart cockpits, and more specifically, to an image processing method, apparatus, and vehicle. Background Technology

[0002] With the rapid development of the automotive industry, cars are no longer just a means of transportation; they are increasingly emphasizing the driving and riding experience, becoming more and more like a "second home," and in-car viewing features are becoming increasingly popular with consumers. Currently, most vehicles are equipped with a large central control screen, and a large number of models also feature multiple screens such as a passenger-side screen, rear-seat screens, and a dedicated movie screen, demonstrating the importance that consumers and manufacturers place on viewing needs. Currently, to address the changing environment, most automakers use ambient light sensors inside and outside the vehicle to automatically control screen brightness, adjusting the display effect according to different environments. However, this adjustment method is too simplistic and cannot achieve a truly optimal viewing experience. Summary of the Invention

[0003] This application provides an image processing method, apparatus, and vehicle that help improve the viewing experience for users when viewing images.

[0004] In a first aspect, this application provides an image processing method, the method comprising: acquiring first information, the first information including a first image and / or first sensing data; determining a first weight vector based on the first information, wherein N weights in the first weight vector correspond one-to-one with N LUTs in a first look-up table (LUT) group, and N is an integer greater than 1; determining a first LUT based on the first weight vector and the N LUTs in the first LUT group; processing the first image based on the first LUT to obtain a second image; and controlling a display device to display the second image.

[0005] Based on the above technical solution, multiple weights can be obtained from image and / or sensor data, thereby determining a first LUT based on these weights and multiple LUTs. Processing the first image using the first LUT yields a second image. This allows for the creation of a target LUT that matches the original image and / or sensor data. Further image enhancement processing of the original image based on the target LUT helps improve the visual effect of the image, making the processed image more suitable for the current viewing scenario and enhancing the user's viewing experience.

[0006] In some possible implementations, the first image is a frame from a video.

[0007] In some possible implementations, determining the first LUT based on the first weight vector and the N LUTs in the first LUT group includes: weighting and fusing the N LUTs in the first LUT group according to the N weights to obtain the first LUT.

[0008] In some possible implementations, the first sensing data can be data collected by the vehicle's sensors. Examples of such sensors include sunlight and rain sensors, cameras, etc.

[0009] In some possible implementations, the first LUT group can be a preset LUT group.

[0010] In some possible implementations, the first LUT can be a 3-Dimension look-up table (3DLUT).

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring second information, the second information including one or more of the data collected by the vehicle's sensors, vehicle control information, and information related to the first image; and determining a first LUT group from multiple LUT groups based on the second information.

[0012] Based on the above technical solution, a first LUT group can be selected from multiple LUT groups according to at least one of the data collected by sensors, vehicle control information, and information related to the first image. This allows the determined first LUT group to match the current viewing scenario, making the processed image more suitable for the current viewing scenario, thus improving the visual effect of the image and enhancing the user's viewing experience.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the second information includes data collected by a light sensor, wherein when the data collected by the light sensor indicates that the light intensity inside the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by the LUT groups other than the first LUT group; and / or, when the data collected by the light sensor indicates that the light intensity inside the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group.

[0014] Based on the above technical solution, when the lighting conditions inside the cabin are good, the image processed by the first LUT group determined from multiple LUT groups has higher brightness and contrast, which helps to improve the viewing experience for users inside the cabin when the lighting conditions are good.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is sunny outside the cockpit, the hue of the image processed by the first LUT group is warmer than the hue of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is sunny outside the cockpit, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0016] Based on the above technical solution, when the environmental information outside the cockpit indicates that it is sunny, the image processed by the first LUT group determined from multiple LUT groups has a warmer tone and higher saturation, which helps to improve the viewing experience for users inside the cockpit when the weather is sunny.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is raining or snowing outside the cockpit, the hue of the image processed by the first LUT group is cooler than the hue of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0018] Based on the above technical solution, when the environmental information outside the cockpit indicates that it is raining or snowing, the image processed by the first LUT group determined from multiple LUT groups has a cooler tone and lower saturation, which helps to improve the viewing experience for users inside the cockpit when the weather is rainy or snowy.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the second information includes vehicle control information. When the vehicle control information indicates that the vehicle speed is greater than or equal to a preset speed, the image processed by the first LUT group has a lower degree of sharpness compared to the image processed by multiple LUT groups other than the first LUT group.

[0020] Based on the above technical solution, when the vehicle speed is high, the image processed by the first LUT group determined from multiple LUT groups is smoother, which helps to improve the viewing experience for users in the cabin when the vehicle is at a high speed.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the second information includes information related to the first image. When the information related to the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by LUT groups other than the first LUT group among multiple LUT groups; or, when the information related to the first image indicates that the first image is an image of a second style, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by LUT groups other than the first LUT group among multiple LUT groups, and / or, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0022] Based on the above technical solution, when the first image is a first-style image (e.g., a science fiction film), the image processed by the first LUT group determined from multiple LUT groups is deeper and more textured, making it more suitable for first-style images. When the first image is a second-style image (e.g., a comedy film), the image processed by the first LUT group determined from multiple LUT groups will exhibit more prominent colors, higher contrast, and higher saturation, making it more suitable for second-style images.

[0023] In conjunction with the first aspect, in some implementations of the first aspect, determining a first LUT group from multiple LUT groups based on the second information includes: when determining a second LUT group from multiple LUT groups based on data collected by sensors and determining a third LUT group from multiple LUT groups based on vehicle control information, the second LUT group is determined as the first LUT group; or, when determining a second LUT group from multiple LUT groups based on data collected by sensors, determining a third LUT group from multiple LUT groups based on vehicle control information and determining a fourth LUT group from multiple LUT groups based on information related to the first image, the fourth LUT group is determined as the first LUT group.

[0024] Based on the above technical solution, after obtaining different LUT groups through different dimensions, a target LUT group can be selected based on the priority information of each dimension. For example, after selecting different LUT groups through data collected by sensors and vehicle control information respectively, the LUT group determined by the data collected by sensors can be determined as the target LUT group. As another example, after selecting different LUT groups through data collected by sensors, vehicle control information, and information related to the first image respectively, the LUT group determined by the information related to the first image can be determined as the target LUT group.

[0025] In some possible implementations, the priority of the LUT group determined by information related to the first image is higher than the priority of the LUT group based on data acquired by the sensor, and the priority of the LUT group based on data acquired by the sensor is higher than the priority of the LUT group determined by vehicle control information.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, determining the first LUT group from multiple LUT groups based on the second information includes: determining the first LUT group from multiple LUT groups based on the second information and the first mapping relationship, wherein the first mapping relationship includes the correspondence between vehicle control information, light intensity, video content and 3DLUT groups.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the first mapping relationship includes the correspondence between vehicle control information, environmental and weather conditions, light intensity, video content, and 3DLUT groups.

[0028] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: controlling the display device to display multiple style options, each style option corresponding to a LUT group; in response to receiving input from a user selecting a first style option from the multiple style options, obtaining a first LUT group, the first style option corresponding to the first LUT group.

[0029] Based on the above technical solution, multiple style options can be displayed on the display device, and these style options can correspond to different LUT groups. The first LUT group can be determined from among the multiple LUT groups based on the user's selection, thus making the final image processing result more in line with the user's expectations.

[0030] In some possible implementations, the method further includes: determining a first LUT group from a plurality of LUT groups based on second information; and controlling the display device to recommend a first style option to the user, the first style option corresponding to the first LUT group.

[0031] Based on the above technical solution, the second information can be used to obtain a LUT group that matches the current scene, thereby recommending style options corresponding to that LUT group to the user. In this way, even when the user lacks relevant image processing knowledge, style options matching the current scene can be recommended, helping the user identify the style option that best suits the current scene from multiple options, avoiding the process of the user viewing multiple style options.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: in response to receiving input from a user selecting a second style option from multiple style options, obtaining a fifth LUT group; determining a second LUT based on a first weight vector and N LUTs in the fifth LUT group; processing the first image based on the second LUT to obtain a third image; and controlling the display device to switch from displaying the second image to displaying the third image.

[0033] Based on the above technical solution, when the user's input of an updated style option is detected, a LUT group corresponding to the updated style option can be selected to determine the second LUT. The second LUT is then used to process the image to obtain the updated image. The user can switch LUT groups after switching style options. Even when the image remains unchanged, the first weight vector can continue to be used, enabling rapid switching of image styles.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, the first image is the Lth frame image in a multi-frame image, where L is a positive integer. The method further includes: acquiring the Sth frame image in the multi-frame image, where the Sth frame is the frame image after the Lth frame, and S is a positive integer; determining a first variation coefficient based on the pixel average value and / or pixel variance of the Lth frame image and the Sth frame image; if the first variation coefficient is less than or equal to a preset variation coefficient, processing the Sth frame image according to the first LUT to obtain a fourth image; and controlling the display device to display the fourth image.

[0035] Based on the above technical solution, when the coefficient of change between the current frame image and historical frame images is less than or equal to a preset coefficient of change, the LUT determined by the historical frames can be used to enhance the current frame image. This ensures that image enhancement is performed using the LUT determined by the historical frames while maintaining minimal changes between the current and historical frames, thus improving the user's viewing experience. Simultaneously, it saves the computational overhead of switching weight vectors.

[0036] In conjunction with the first aspect, in some implementations of the first aspect, the first image is the Pth frame image in a multi-frame image, where P is a positive integer. The method further includes: within a preset time period starting from when the control display device displays the second image, processing the images after the Pth frame image in the multi-frame image according to the first LUT to obtain the fifth image; and controlling the control display device to display the fifth image.

[0037] Based on the above technical solution, within a preset time period starting from the display of the second image, the first LUT can continue to process images after the Pth frame, thus saving the computational overhead of switching weight vectors. For devices with low computing power, this can prevent system lag.

[0038] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: at the end of a preset time period, acquiring third information, the third information including the Q-th frame image and / or second sensor data from multiple frames of images, where Q is a positive integer; determining a second weight vector based on the third information, wherein the N weights in the second weight vector correspond one-to-one with the N LUTs in the sixth LUT group; determining a third LUT based on the second weight vector and the N LUTs in the sixth LUT group; processing the Q-th frame image based on the third LUT to obtain the sixth image; and controlling the display device to display the sixth image.

[0039] Based on the above technical solution, after the preset time period ends, the weight vector can be reacquired based on the Q-frame image and / or the second sensor data, thereby enabling the calculation of a new LUT. By keeping the weight vector unchanged within the preset time period, the computational overhead of switching weight vectors can be saved. For devices with low computing power, system lag can be avoided.

[0040] In conjunction with the first aspect, in some implementations of the first aspect, the first image is a partial frame image in a multi-frame image, and the method further includes: processing other images in the multi-frame image except for the partial frame images according to the first LUT to obtain a seventh image; and controlling the display device to display the seventh image.

[0041] Based on the above technical solution, a first LUT can be obtained by using some frames from multiple frames, and then the first LUT can be used to process other frames from the multiple frames.

[0042] In some possible implementations, this frame image can be the first frame, middle frame, or cover of a multi-frame image.

[0043] In conjunction with the first aspect, in some implementations of the first aspect, before processing the images other than some frames in the multi-frame images according to the first LUT, the method further includes: determining that the playback duration of the multi-frame images is less than or equal to a preset duration.

[0044] Based on the above technical solution, since the pixel change rate in multi-frame images with short playback durations may not be significant, a first LUT can be obtained from a subset of the multi-frame images. This first LUT can then be used to process the remaining frames in the multi-frame images. This approach ensures image enhancement across the multi-frame images while avoiding the computational overhead of switching weight vectors.

[0045] In conjunction with the first aspect, in some implementations of the first aspect, determining the first weight vector based on the first information includes: inputting the first information into the prediction model to obtain the first weight vector.

[0046] In some possible implementations, the first information includes a first image, and the prediction model includes three parts: the first part is a preprocessing module, which is used to uniformly scale the first image to a smaller size to facilitate subsequent network processing and reduce the amount of computation; the second part is a multi-layer convolutional neural network, which is used to extract deep features in the image; and the third part is a fully connected network, which is used to output N weights and normalize the weight range of the weights through a post-processing layer.

[0047] Secondly, this application provides an image processing apparatus, comprising: an acquisition unit for acquiring first information, the first information including a first image and / or first sensing data; a determination unit for determining a first weight vector based on the first information, wherein N weights in the first weight vector correspond one-to-one with N LUTs in a first color lookup table (LUT) group, and N is an integer greater than 1; the determination unit is further configured to determine a first LUT based on the first weight vector and the N LUTs in the first LUT group; an image processing unit for processing the first image based on the first LUT to obtain a second image; and a control unit for controlling a display device to display the second image.

[0048] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to acquire second information, which includes one or more of the following: data collected by the vehicle's sensors, vehicle control information, and information related to the first image; and the determination unit is configured to determine a first LUT group from multiple LUT groups based on the second information.

[0049] In conjunction with the second aspect, in some implementations of the second aspect, the second information includes data collected by a light sensor, wherein when the data collected by the light sensor indicates that the light intensity inside the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by the LUT groups other than the first LUT group; and / or, when the data collected by the light sensor indicates that the light intensity inside the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group.

[0050] In conjunction with the second aspect, in some implementations of the second aspect, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is sunny outside the cockpit, the hue of the image processed by the first LUT group is warmer than the hue of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is sunny outside the cockpit, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0051] In conjunction with the second aspect, in some implementations of the second aspect, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is raining or snowing outside the cockpit, the hue of the image processed by the first LUT group is cooler than the hue of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0052] In conjunction with the second aspect, in some implementations of the second aspect, the second information includes vehicle control information. When the vehicle control information indicates that the vehicle speed is greater than or equal to a preset speed, the image processed by the first LUT group has a lower degree of sharpness compared to the image processed by multiple LUT groups other than the first LUT group.

[0053] In conjunction with the second aspect, in some implementations of the second aspect, the second information includes information related to the first image. When the information related to the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by LUT groups other than the first LUT group among multiple LUT groups; or, when the information related to the first image indicates that the first image is an image of a second style, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by LUT groups other than the first LUT group among multiple LUT groups, and / or, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0054] In conjunction with the second aspect, in some implementations of the second aspect, the determining unit is configured to: determine the second LUT group as the first LUT group when determining the second LUT group from multiple LUT groups based on data collected by the sensor and determining the third LUT group from multiple LUT groups based on vehicle control information; or, when determining the second LUT group from multiple LUT groups based on data collected by the sensor, determining the third LUT group from multiple LUT groups based on vehicle control information and determining the fourth LUT group from multiple LUT groups based on information related to the first image, determine the fourth LUT group as the first LUT group.

[0055] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is further configured to control the display device to display multiple style options, each style option corresponding to a LUT group; the acquisition unit is further configured to acquire a first LUT group in response to acquiring input from the user selecting a first style option from the multiple style options, the first style option corresponding to the first LUT group.

[0056] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to acquire a fifth LUT group in response to acquiring input from the user selecting a second style option from multiple style options; the determination unit is further configured to determine a second LUT based on the first weight vector and N LUTs in the fifth LUT group; the image processing unit is further configured to process the first image based on the second LUT to obtain a third image; and the control unit is further configured to control the display device to switch from displaying the second image to displaying the third image.

[0057] In conjunction with the second aspect, in some implementations of the second aspect, the first image is the Lth frame image in a multi-frame image, where L is a positive integer; the acquisition unit is further configured to acquire the Sth frame image in the multi-frame image, where the Sth frame is the frame image following the Lth frame, and S is a positive integer; the determination unit is further configured to determine a first change coefficient based on the pixel average value and / or pixel variance of the Lth frame image and the Sth frame image; the image processing unit is further configured to process the Sth frame image according to the first LUT when the first change coefficient is less than or equal to a preset change coefficient, to obtain a fourth image; and the control unit is further configured to control the display device to display the fourth image.

[0058] In conjunction with the second aspect, in some implementations of the second aspect, the first image is the Pth frame image in a multi-frame image, where P is a positive integer. The image processing unit is further configured to process the images after the Pth frame image in the multi-frame image according to the first LUT within a preset time period after the control unit controls the display device to display the second image, thereby obtaining the fifth image. The control unit is further configured to control the display device to display the fifth image.

[0059] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to acquire third information at the end of a preset time period, the third information including the Q-th frame image in the multi-frame images and / or the second sensor data, where Q is a positive integer; the determination unit is further configured to determine a second weight vector based on the third information, wherein the N weights in the second weight vector correspond one-to-one with the N LUTs in the sixth LUT group; the determination unit is further configured to determine a third LUT based on the second weight vector and the N LUTs in the sixth LUT group; the image processing unit is further configured to process the Q-th frame image based on the third LUT to obtain the sixth image; and the control unit is further configured to control the display device to display the sixth image.

[0060] In conjunction with the second aspect, in some implementations of the second aspect, the first image is a partial frame image in a multi-frame image, the image processing unit is further configured to process other images in the multi-frame image besides the partial frame image according to the first LUT to obtain the seventh image; the control unit is further configured to control the display device to display the seventh image.

[0061] In conjunction with the second aspect, in some implementations of the second aspect, the determining unit is further configured to determine that the playback duration of the multi-frame images is less than or equal to a preset duration before the image processing unit processes other images in the multi-frame images, excluding some frame images.

[0062] In conjunction with the second aspect, in some implementations of the second aspect, a unit is defined for: inputting the first information into the prediction model to obtain the first weight vector.

[0063] This application can be applied to in-vehicle viewing scenarios as well as other fields, such as smart home viewing scenarios. The solution may also be used in smart screen audio-visual systems, for example, by acquiring ambient light information through sensors such as light intensity, and combining this information with video information to select a suitable LUT group. Additionally, the solution may also be applied in video recording scenarios using mobile phones and other photography devices. By using sensors attached to the mobile phone to perceive the external environment, applying different LUT groups, and then improving the video recording effect through the recorded content, the solution can be optimized.

[0064] Thirdly, this application provides an image processing apparatus including a processor and a memory, wherein the memory is used to store instructions, and the processor executes the instructions stored in the memory to cause the apparatus to perform any of the possible methods in the first aspect.

[0065] Fourthly, this application provides an image processing system, which includes a computing platform and a display screen, wherein the computing platform includes any of the possible devices in the second or third aspect.

[0066] In some possible implementations, the image processing system also includes a perception system.

[0067] Fifthly, this application provides a vehicle that includes any of the possible devices of the second or third aspect, or includes the system described in the fourth aspect.

[0068] In a sixth aspect, this application provides a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the possible methods described in the first aspect above.

[0069] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or packaged separately from the processor. This application embodiment does not specifically limit this.

[0070] In a seventh aspect, this application provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform any of the possible methods described in the first aspect above.

[0071] Eighthly, this application provides a chip system including a processor for calling a computer program or computer instructions stored in a memory to cause the processor to perform any of the possible methods in the first aspect above.

[0072] In conjunction with the eighth aspect, in one possible implementation, the processor is coupled to the memory via an interface.

[0073] In conjunction with the eighth aspect, in one possible implementation, the chip system also includes a memory in which computer programs or computer instructions are stored.

[0074] Ninthly, this application provides a chip system including circuitry for performing any of the possible methods described in the first aspect above. Attached Figure Description

[0075] Figure 1 is a functional block diagram of the vehicle provided in an embodiment of this application.

[0076] Figure 2 is a schematic diagram of a vehicle cabin scene provided in an embodiment of this application.

[0077] Figure 3 is a schematic flowchart of the image processing method provided in the embodiments of this application.

[0078] Figure 4 is a human-machine interface (HMI) provided in an embodiment of this application.

[0079] Figure 5 is a schematic block diagram of the image processing system provided in an embodiment of this application.

[0080] Figure 6 is another schematic block diagram of the image processing system provided in an embodiment of this application.

[0081] Figure 7 is another schematic block diagram of the image processing system provided in an embodiment of this application.

[0082] Figure 8 is another schematic block diagram of the image processing system provided in an embodiment of this application.

[0083] Figure 9 is another schematic block diagram of the image processing system provided in an embodiment of this application.

[0084] Figure 10 is a schematic block diagram of an image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0085] Figure 1 is a functional block diagram of a vehicle provided in an embodiment of this application. As shown in Figure 1, the vehicle 100 may include a display device 130 and a computing platform 150. The display device 130 in the cabin is mainly divided into two categories: the first category is an in-vehicle display screen; the second category is a projection display screen, such as a head-up display (HUD). An in-vehicle display screen is a physical display screen and is an important component of the in-vehicle infotainment system. Multiple display screens can be installed in the cabin, such as a digital instrument display screen, a central control screen, a display screen in front of the passenger in the front passenger seat (also known as the front passenger), a display screen in front of the left rear passenger, a display screen in front of the right rear passenger, and even the car window can be used as a display screen.

[0086] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n. A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory to implement the corresponding functions.

[0087] Optionally, the structure of the vehicle 100 described above is merely illustrative. In actual applications, various components of the vehicle 100 may be added or removed as needed.

[0088] Figure 2 is a schematic diagram of a vehicle cockpit scenario provided in an embodiment of this application. The smart cockpit is equipped with one or more in-vehicle displays (or in-vehicle screens), including but not limited to display screen 201 (or central control screen), display screen 202 (or passenger entertainment screen), display screen 203 (or driver's headrest rear screen), display screen 204 (or passenger headrest rear screen), display screen 205 (or second-row entertainment screen) mounted on the cockpit ceiling, and an instrument panel. Further, displays 201 to 205 can display a graphical user interface (GUI), which may include icons for one or more applications and / or one or more cards. Exemplarily, the display device 130 shown in Figure 1 can be one or more of displays 201 to 205. In some possible implementations, display screen 201 can also be a long, continuous screen extending to the passenger area. Additionally, display screen 205 can also be a projection screen associated with a projector, which can be associated with a desktop launcher to manage applications projected onto the projection screen.

[0089] As shown in Figure 2, one or more cameras can also be installed in the cockpit to capture images inside or outside the cockpit. These could include cameras from a driver monitor system (DMS), a cabin monitor system (CMS), or a dashcam. The cameras used to capture images inside and outside the cockpit can be the same camera or different cameras. In addition, one or more pressure sensors and acoustic sensors are installed in the cockpit to monitor the presence and location of users.

[0090] It should be understood that the control display method in the following embodiments is illustrated using a 5-seat vehicle as shown in Figure 2 as an example, and the embodiments of this application are not limited to this. For example, for a 7-seat sport / suburban utility vehicle (SUV), the cabin may include a central control screen, a passenger entertainment screen, a screen behind the driver's headrest, a screen behind the passenger's headrest, entertainment screens in the left area of ​​the third row, and entertainment screens in the right area of ​​the third row. As another example, for a bus, the cabin may include front and rear entertainment screens; or, the cabin may include a display screen in the driver's area and an entertainment screen in the passenger area. Furthermore, the following embodiments use a left-hand drive vehicle (i.e., the driver is located on the left side of the vehicle) as an example for illustration; in actual implementation, the vehicle may also be a right-hand drive vehicle (i.e., the driver is located on the right side of the vehicle).

[0091] As mentioned earlier, with the rapid development of the automotive industry, cars are no longer just a means of transportation; they are increasingly emphasizing the driving and riding experience, becoming more and more like a "second home," and in-car viewing features are becoming increasingly popular with consumers. Currently, most vehicles are equipped with a large central control screen, and a large number of models also feature multiple screens such as a passenger-side screen, rear-seat screens, and a dedicated movie screen, demonstrating the importance that consumers and manufacturers place on viewing needs. Currently, to address the changing environment, most automakers use ambient light sensors to automatically control screen brightness, adjusting the display effect in different environments. However, this adjustment method is too simplistic and cannot achieve a truly optimal viewing experience.

[0092] This application provides an image processing method, apparatus, and vehicle. By obtaining a target LUT that matches the original image and / or sensor data through images and / or sensor data, and then performing image enhancement processing on the original image based on the target LUT, the processed image can be more in line with the current viewing scenario, which helps to improve the visual effect of the image and also helps to improve the viewing experience for users.

[0093] Figure 3 shows a schematic flowchart of an image processing method 300 provided in an embodiment of this application. The method 300 includes:

[0094] S310, acquire first information, the first information including a first image and / or first sensing data.

[0095] For example, the first image may be a single frame or multiple frames from the video source to be played.

[0096] For example, taking method 300 as performed by a vehicle, the first sensing data includes data collected by the vehicle's sensors. For instance, the first sensing data may include one or more of the following: data collected by a light sensor inside the cabin, data collected by a light sensor outside the cabin, and data collected by a sunlight and rain sensor. This data can be used to determine the light intensity of the environment in which the vehicle is currently located.

[0097] For example, the first sensing data includes data collected by sensors inside the cockpit (e.g., cameras, light sensors) and / or data collected by sensors outside the cockpit (e.g., cameras, light sensors). This data can be used to indicate one or more of the following: ambient light intensity, light source type (e.g., direct sunlight, natural light, artificial light source, etc.), light direction (e.g., backlight, backlight, etc.), and environment type (e.g., indoor garage, outdoor garage, mountainous countryside, highway, etc.).

[0098] For example, the first sensing data may include vehicle control information, which may be data collected by vehicle control sensors (e.g., gear position sensor, vehicle speed sensor, etc.). Alternatively, the vehicle control information may be used to indicate the current driving status of the vehicle, for example, the vehicle control information may be used to indicate the current gear and / or speed of the vehicle.

[0099] For example, the first sensing data may include information related to the first image, such as one or more of the following: the type of the first image (e.g., movie, live stream, short video, etc.), the content style (e.g., comedy, ancient style, science fiction, etc.), and the playback duration.

[0100] Optionally, determining the first weight vector based on the first information includes: inputting the first information into the prediction model to obtain the first weight vector.

[0101] S320, Based on the first information, determine the first weight vector. The N weights in the first weight vector correspond one-to-one with the N LUTs in the first LUT group, where N is an integer greater than 1.

[0102] Optionally, determining the first weight vector based on the first information includes: inputting the first information into the prediction model to obtain the first weight vector.

[0103] For example, the first information includes a first image, and the prediction model includes three parts: the first part is a preprocessing module, which is used to uniformly scale the first image to a smaller size (e.g., 256*256 pixels) to facilitate subsequent network processing and reduce the amount of computation; the second part is a multi-layer convolutional neural network, which is used to extract deep features in the image; and the third part is a fully connected network, which is used to output N weights and normalize the weight range of the weights through a post-processing layer.

[0104] Optionally, before executing method 300, M 3DLUT groups can be generated through training, where M is an integer greater than 1. For example, each 3DLUT group is suitable for different external environment scenarios, source types, and source styles.

[0105] For example, when playing a movie in a dimly lit underground parking garage, the image processed by the 3DLUT group will appear deeper and more textured. Or, when driving outdoors and playing an animated film, the image processed by the 3DLUT group will have higher contrast and saturation. M can be determined based on the style being represented. For example, M can be 3, representing three styles: high contrast, medium contrast, and low contrast. For example, color temperature styles can also be added, such as cool tones or warm tones. M can be increased according to the desired style's hue, color temperature, and other dimensions.

[0106] Optionally, each 3DLUT group curve has N 3DLUTs for real-time video content recognition, dynamically adjusting the 3DLUTs ultimately used for each frame of the image. For example, N can be 3, representing the adjustment strength for highlight, medium brightness, and low brightness areas of the video, respectively. A larger N value allows for more flexible real-time adjustments.

[0107] For example, the above M 3DLUT groups and prediction models can be obtained through training. The training steps include, but are not limited to, the following steps (1)-(3):

[0108] (1) Prepare high-quality video source data and optimize it under different environments to obtain reference data.

[0109] (2) Prepare low-quality video data corresponding to the content and perform degradation processing according to different parameters (for example, reduce the contrast of the image or downsample it to reduce the video quality) to obtain another reference data.

[0110] (3) Train the data obtained in steps (1) and (2) in the network to obtain the prediction model and M 3DLUT groups.

[0111] The training of the prediction model and the training of the multiple 3DLUT groups mentioned above can be carried out simultaneously. Therefore, the weight vector output by the prediction model is matched with the multiple 3DLUT groups.

[0112] The above training methods are merely illustrative and are not limited to the embodiments described in this application. For example, 3DLUTs can also be manually adjusted and generated by professional engineers, and during training, existing 3DLUT sets can be used and fixed to train only the prediction model.

[0113] Optionally, method 300 further includes: acquiring second information, the second information including one or more of the data collected by the vehicle's sensors, vehicle control information, and information related to the first image; and determining a first LUT group from multiple LUT groups based on the second information.

[0114] The data collected by the sensors of the above vehicles can be referred to the description of the data collected by the sensors inside the cabin and / or the data collected by the sensors outside the cabin. The vehicle control information and the information related to the first image can be referred to the description in the above embodiments, and will not be repeated here.

[0115] Optionally, the second information includes data collected by a light sensor, wherein when the data collected by the light sensor indicates that the light intensity inside the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by the LUT groups other than the first LUT group; and / or, when the data collected by the light sensor indicates that the light intensity inside the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group.

[0116] For example, Table 1 shows the correspondence between the light intensity and the 3DLUT group provided in the embodiments of this application.

[0117] Table 1

[0118] For example, images processed by 3DLUT Group 1 have higher brightness and contrast than images processed by 3DLUT Group 2 or 3DLUT Group 3, making them more suitable for viewing in bright light.

[0119] Optionally, the light intensity can be directly determined using data collected by a light sensor. Furthermore, combining this data with data from cameras inside or outside the cabin to sense the direction of light and the viewer's position allows for a more accurate assessment. For example, when the light intensity is strong in the front row and weak in the rear row, 3DLUT group 1 can be used to process the video content played on the central control screen or the passenger-side screen, and 3DLUT group 3 can be used to process the video content played on both screens. This ensures a better viewing experience for both front and rear passengers.

[0120] Optionally, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is sunny outside the cockpit, the color tone of the image processed by the first LUT group is warmer than the color tone of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is sunny outside the cockpit, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0121] Optionally, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is raining or snowing outside the cockpit, the color tone of the image processed by the first LUT group is cooler than the color tone of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0122] For example, Table 2 shows the correspondence between the environmental and weather conditions provided in the embodiments of this application and the 3DLUT group.

[0123] Table 2

[0124] For example, images processed by 3DLUT group 4 have a warmer tone and higher saturation than images processed by other 3DLUT groups, making them more suitable for viewing in sunny scenes; images processed by 3DLUT group 7 have a cooler tone and lower saturation compared to images processed by other 3DLUT groups, making them more suitable for viewing in rainy weather.

[0125] Optionally, the second information includes vehicle control information. When the vehicle control information indicates that the vehicle speed is greater than or equal to a preset speed, the image processed by the first LUT group has a lower degree of sharpness compared to the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0126] For example, Table 3 shows the correspondence between the vehicle driving state and the 3DLUT provided in the embodiments of this application.

[0127] Table 3

[0128] For example, the image processed by 3DLUT group 10 has a lower degree of sharpening compared to the images processed by 3DLUT group 8 and DLUT group 9. This makes the image processed by 3DLUT group 10 softer than the images processed by 3DLUT group 8 and DLUT group 9, making it suitable for viewing in driving scenarios.

[0129] Optionally, the second information includes information related to the first image. When the information related to the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by LUT groups other than the first LUT group among multiple LUT groups; or, when the information related to the first image indicates that the first image is an image of a second style, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by LUT groups other than the first LUT group among multiple LUT groups, and / or, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0130] Optionally, when information related to the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than that of the image processed by LUT groups other than the first LUT group, the saturation of a certain hue in the image processed by the first LUT group is lower than that of the hue in the image processed by LUT groups other than the first LUT group, and the contrast of the image processed by the first LUT group is higher than that of the image processed by LUT groups other than the first LUT group, which can make the image processed by the first LUT group deeper than the image processed by LUT groups other than the first LUT group.

[0131] For example, images of the first style include science fiction films, and images of the second style include comedy films.

[0132] For example, Table 4 shows the correspondence between the video content provided in the embodiments of this application and the 3DLUT group.

[0133] Table 4

[0134] For example, images processed by 3DLUT group 11 will have a fresh and natural style, making them more suitable for films featuring people or natural scenery. Images processed by 3DLUT group 12 will have a bright and vivid style, making them more suitable for films with a traditional Chinese style. Images processed by 3DLUT group 13 will have lower brightness, lower saturation of certain hues, and higher contrast, resulting in a deeper and more textured film style, suitable for science fiction films. Images processed by 3DLUT group 14 will have more prominent colors and higher contrast and saturation, making them suitable for comedy films.

[0135] Optionally, the method 300 includes: determining the video content based on one or more of the following: the application (app) type, video duration, and tags corresponding to the video source to be played.

[0136] Optionally, determining a first LUT group from multiple LUT groups based on the second information includes: when determining a second LUT group from multiple LUT groups based on sensor-collected data and a third LUT group from multiple LUT groups based on vehicle control information, the second LUT group is determined as the first LUT group; or, when determining a second LUT group from multiple LUT groups based on sensor-collected data, a third LUT group from multiple LUT groups based on vehicle control information, and a fourth LUT group from multiple LUT groups based on information related to the first image, the fourth LUT group is determined as the first LUT group.

[0137] For example, referring to Tables 1-4 above, after obtaining different LUT groups through different dimensions, a target LUT group can be selected based on the priority information of each dimension. For instance, after selecting different LUT groups through sensor-collected data and vehicle control information respectively, the LUT group determined through sensor-collected data can be determined as the target LUT group. As another example, after selecting different LUT groups through sensor-collected data, vehicle control information, and information related to the first image respectively, the LUT group determined through information related to the first image can be determined as the target LUT group.

[0138] For example, the priority of the LUT group determined by information related to the first image is higher than the priority of the LUT group determined by data acquired by the sensor, and the priority of the LUT group determined by data acquired by the sensor is higher than the priority of the LUT group determined by vehicle control information.

[0139] The above dimensions are merely examples to illustrate the effects of selecting a 3DLUT group under a single dimension. In actual implementations, the 3DLUT group to be used will be determined based on one or more of these dimensions simultaneously.

[0140] Optionally, determining a first LUT group from multiple LUT groups based on the second information includes: determining a first LUT group from multiple LUT groups based on the second information and a first mapping relationship, wherein the first mapping relationship includes the correspondence between vehicle control information, light intensity, video content and 3DLUT.

[0141] For example, Table 5 shows the correspondence between vehicle driving status, light intensity, video content and 3DLUT group provided in the embodiments of this application.

[0142] Table 5

[0143] For example, the image processed by 3DLUT group 15 has higher brightness and contrast than the image processed by 3DLUT group 16.

[0144] For example, the image processed by 3DLUT group 23 is softer than the image processed by 3DLUT group 19, making it suitable for viewing in driving scenarios.

[0145] Optionally, the first mapping relationship includes the correspondence between vehicle control information, environmental and weather conditions, light intensity, video content, and 3DLUT.

[0146] Optionally, method 300 further includes: controlling the display device to display a plurality of style options, each style option corresponding to a LUT group; in response to receiving input from the user selecting a first style option from the plurality of style options, obtaining a first LUT group, the first style option corresponding to the first LUT group.

[0147] For example, Figure 4 illustrates a human machine interface (HMI) provided in an embodiment of this application.

[0148] As shown in Figure 4, during video playback on the central control screen, the vehicle can display multiple style options, such as warm color option 401, neutral color option 402, and cool color option 403. Users can switch between different style options by clicking on the style option on the central control screen or dragging the style option onto the video source. Each style option can correspond to a 3DLUT group.

[0149] Optionally, after determining the first 3DLUT group through the second information, the user can be prompted with a style option that matches the current scene through the central control screen. This style option is the style option corresponding to the first 3DLUT group.

[0150] S330, determine the first LUT based on the first weight vector and the N LUTs in the first LUT group.

[0151] Optionally, the N LUTs in the first LUT group are weighted and fused according to the first weight vector to obtain the first LUT.

[0152] For example, the first 3DLUT group contains N 3DLUTs, i.e., LG = (L1, L2, ..., L...). N The first weight vector includes the weights (w1, w2, ..., w) for each 3DLUT. N Through weights (w1, w2, ..., w) NThe first 3DLUT group is weighted and fused to obtain the final 3DLUT used for the current frame image. For example, the weighted fusion can be performed using the following formula (1) to obtain the final 3DLUT used, i.e., L. f :

[0153] Optionally, the values ​​stored in the 3DLUT may not be all possible RGB values; they can be a subset of sampled points. For example, the size of the 3DLUT may be 9, 13, 17, or 33. Trilinear interpolation or tetrahedral interpolation can be used to obtain the lookup value of the input RGB values. This saves storage space on the 3DLUT.

[0154] Optionally, method 300 further includes: in response to receiving input from a user selecting a second style option from multiple style options, obtaining a fifth LUT group; determining a second LUT based on a first weight vector and N LUTs in the fifth LUT group; processing the first image based on the second LUT to obtain a third image; and controlling the display device to switch from displaying the second image to displaying the third image.

[0155] For example, taking method 300 executed by a vehicle, the vehicle's infotainment system has M pre-set 3DLUT groups, each corresponding to M different style options. After obtaining the second information, the style option suitable for the current environment can be determined, and its corresponding first 3DLUT group can be obtained.

[0156] When a video is played for the first time or during a video transition, a newly calculated style option can be output. During video playback, the 3DLUT group is updated at a low frequency (e.g., every 5 minutes to avoid frequent transitions). A new style option will be obtained when the style option transitions due to scene changes or video source changes. The vehicle can then transition between the two style options based on the recorded transition time. For example, the 3DLUT group used before, during, and after the transition can be as shown in formula (2):

[0157] LG1 and LG2 represent the 3DLUT groups corresponding to style option 1 and style option 2, respectively. t0 represents the moment of switching from style option 1 to style option 2, and T represents the transition duration. For example, T can be set to 5 seconds to provide a better user experience.

[0158] When a user manually switches the style options on the interface, the set style options will be updated to the configuration of the current environment state, forming a user-defined style configuration. When the user is in the same or a similar environment state again, the updated main style will be applied to the user.

[0159] S340, based on the first LUT, process the first image to obtain the second image.

[0160] For example, by traversing the first image and inputting each RGB value in the first image into the first LUT, the corresponding RGB values ​​can be output, thereby obtaining the second image after video enhancement.

[0161] S350, the control display device displays the second image.

[0162] Optionally, determining a first weight vector based on the first information includes: when the first information indicates that the difference between the light intensity of a first area and the light intensity of a second area within the vehicle cabin is greater than or equal to a preset difference, determining a weight vector 1 based on the light intensity of the first area and a weight vector 2 based on the light intensity of the second area. The N weights in weight vector 1 correspond one-to-one with the N LUTs in LUT group a, and the X weights in weight vector 2 correspond one-to-one with the X LUTs in LUT group b. For example, the first area can be the front row area within the cabin, and the second area can be the rear row area within the cabin.

[0163] Optionally, determining a first LUT group from multiple LUT groups based on the second information includes: determining LUT group a from multiple LUT groups based on the illumination intensity of the first region and determining LUT group b from multiple LUT groups based on the illumination intensity of the second region.

[0164] Optionally, the first LUT is determined based on the first weight vector and N LUTs in the first LUT group, including: determining LUT1 based on weight vector 1 and LUT group a, and determining LUT2 based on weight vector 2 and LUT group b.

[0165] Optionally, processing the first image according to the first LUT to obtain the second image includes: processing the first image according to LUT1 to obtain image 1 and processing the first image according to LUT2 to obtain image 2.

[0166] Optionally, controlling the display device to display the second image includes: controlling the display device in the first area to display image 1 and controlling the display device in the second area to display image 2.

[0167] For example, while the vehicle is in motion, even with high outside light intensity, opening the privacy curtain in the rear seats can still result in a significant difference in light intensity between the front and rear areas inside the cabin. Similarly, even with high outside light intensity, opening the rear curtain can also lead to a significant difference in light intensity between the front and rear areas inside the cabin. In this case, weight vector 1 can be determined based on the light intensity of the front area, and weight vector 2 can be determined based on the light intensity of the rear area. Alternatively, weight vector 1 can be determined based on the light intensity of the front area, vehicle control information, and information related to the first image, and weight vector 2 can be determined based on the light intensity of the rear area, vehicle control information, and information related to the first image.

[0168] Optionally, determining a first weight vector based on the first information includes: when the first information indicates that the difference between the light intensity of the first area and the light intensity of the second area in the vehicle cabin is less than a preset difference, determining a weight vector 3 based on information related to the image displayed by the display device in the first area, and determining a weight vector 4 based on information related to the image displayed by the display device in the second area, wherein the N weights in the weight vector 3 correspond one-to-one with the N LUTs in LUT group c, and the X weights in the weight vector 4 correspond one-to-one with the X LUTs in LUT group d.

[0169] Optionally, determining a first LUT group from multiple LUT groups based on the second information includes: determining LUT group c from multiple LUT groups based on information related to the image displayed on the display device of the first area, and determining LUT group d from multiple LUT groups based on information related to the image displayed on the display device of the second area.

[0170] Optionally, the first LUT is determined based on the first weight vector and N LUTs in the first LUT group, including: determining LUT3 based on weight vector 3 and LUT group c, and determining LUT4 based on weight vector 4 and LUT group d.

[0171] Optionally, processing the first image according to the first LUT to obtain the second image includes: processing the image displayed on the display device of the first area according to LUT3 to obtain image 3, and processing the image displayed on the display device of the second area according to LUT4 to obtain image 4.

[0172] Optionally, controlling the display device to display the second image includes: controlling the display device in the first area to display image 3 and controlling the display device in the second area to display image 4.

[0173] For example, during vehicle operation, the video content displayed in the front and rear areas differs. In this case, weight vector 3 can be determined based on information related to the images displayed in the front area (e.g., the style of the video content on the central control screen), and weight vector 4 can be determined based on information related to the images displayed in the rear area (e.g., the style of the video content on the rear screen). Alternatively, weight vector 3 can be determined based on the light intensity in the front area, vehicle control information, and information related to the images displayed in the front area, and weight vector 4 can be determined based on the light intensity in the rear area, vehicle control information, and information related to the images displayed in the rear area.

[0174] Figure 5 shows a schematic block diagram of an image processing system 500 provided in an embodiment of this application. The image processing system 500 includes a cockpit multimedia system 510 and a display screen 520. The cockpit multimedia system 510 includes a video encoding / decoding (MediaCodec) module 511, a display engine 512, and a graphics synthesis and display module 513. The display screen 520 may include one or more of a central control screen, a passenger screen, or a rear screen.

[0175] In the smart cockpit (or on mobile phones, tablets, smart screen TVs, etc.), video apps transmit the video source data to the video encoding / decoding module 511. The video encoding / decoding module 511 decodes the video data and transmits each frame as an image to the display engine 512. The display engine 512 processes the image based on one or more of the information from the perception system, vehicle control system, operating system, and user parameters, controlling the image display effect. The processed graphic data is then fused with other graphics (such as bullet screen text, UI interface, etc.) by the operating system before being displayed on various screens within the cockpit.

[0176] In this embodiment, image enhancement is primarily performed in the display engine 512 to achieve a better viewing experience. In the technical solution provided in this embodiment, the display engine 512, in addition to interacting with the upstream video encoding / decoding module 511 and the downstream graphics synthesis and display module 513 to exchange video stream data, also receives user configuration parameters, basic video information, in-vehicle perception system data, and external vehicle perception system data, and performs image enhancement based on this data.

[0177] Figure 6 shows another schematic block diagram of the image processing system 500 provided in an embodiment of this application. The display engine 512 includes a style control module 5121, an image detection module 5122, and an image enhancement module 5123. The multiple 3DLUT groups and prediction models obtained through pre-training can be referred to the description in the above embodiments. In addition to receiving data from the perception system, the style control module 5121 can also receive data from the vehicle control system and the operating system.

[0178] For example, the image detection module 5122 can be used to perform the above-described step S320. The image enhancement module 5123 can be used to perform the above-described steps S330 and S340.

[0179] Optionally, the style control module 5121 can operate at a low frequency. For example, the style control module 5121 can receive data from the perception system, vehicle control system, and operating system every 5 minutes. As another example, the style control module 5121 can update the 3DLUT group after receiving a user's instruction to switch style options.

[0180] Optionally, the image detection module 5122 and the image enhancement module 5123 can operate at high frequency. For example, for each frame of the video, the image detection module 5122 can calculate a weight vector and the image enhancement module 5123 can calculate the target LUT based on the weight vector and the 3DLUT set.

[0181] Optionally, the first image is the Lth frame image in a multi-frame image, where L is a positive integer. Method 300 further includes: acquiring the Sth frame image in the multi-frame image, where the Sth frame is the frame image after the Lth frame, and S is a positive integer; determining a first variation coefficient based on the pixel average value and / or pixel variance of the Lth frame image and the Sth frame image; if the first variation coefficient is less than or equal to a preset variation coefficient, processing the Sth frame image according to a first LUT to obtain a fourth image; and controlling the display device to display the fourth image.

[0182] For example, FIG7 shows another schematic block diagram of an image processing system 500 provided in an embodiment of this application. The display engine 512 may include a content change detection module 5124.

[0183] For example, taking method 300 executed by a vehicle, due to cockpit system performance issues, it may be impossible to calculate the corresponding weight vector for every frame of video image. A content change detection module 5124 can be added. By inputting the current frame image and historical frame images into the content change detection module 5124, the content change detection module 5124 can output a change coefficient. The content change detection module 5124 can send this change coefficient to the image detection module 5122. The image detection module 5122 can determine whether to perform the operation of outputting the weight vector based on the value of the change coefficient. This solution can significantly reduce the operating frequency of the image detection module 5122 while also achieving better video enhancement effects.

[0184] For example, a content change detection module 5124 is added as a preprocessing module for the image detection module 5122. The definition and calculation steps of the change coefficient are as follows:

[0185] S1: The current frame image is received at time t=N, and the current frame image is downsampled.

[0186] For example, downsample the current frame image to a size of 256*256.

[0187] S2: The image is divided into W*H blocks.

[0188] For example, it can be divided into 16*16 blocks, each block being 16*16 pixels in size.

[0189] S3: Calculate the pixel mean and variance in each block. and

[0190] S4: Compare the data from the images at time N-1 and calculate the coefficient of change D. N The formulas are shown in (3) and (4):

[0191] Among them, T E For example, T is the mean error threshold. E 3.0 is acceptable. S For example, T is the variance error threshold. S 2.0 is acceptable. The parameter selection can be determined based on the operating frequency that the actual product performance can support.

[0192] S5: Save the current time. Used for calculating the change coefficient in the next frame.

[0193] After the change coefficient is input to the image detection module 5122, a threshold judgment is performed. If the change coefficient is less than the threshold, the image detection module 5122 no longer needs to perform the output weight vector operation and directly uses the cached historical weight vector. If the coefficient change is greater than or equal to the threshold, the image detection module 5122 can perform the output weight vector operation based on the current frame image and / or the current sensing data to update the weight vector. For example, the threshold can be 0.1.

[0194] Optionally, the first image is the Pth frame image in the multi-frame image, where P is a positive integer. The method 300 further includes: within a preset time period starting from when the control display device displays the second image, processing the images after the Pth frame image in the multi-frame image according to the first LUT to obtain the fifth image; and controlling the control display device to display the fifth image.

[0195] Here, a minimum interval time limit can be added to the image detection module 5122 to avoid frequent execution of the output weight vector operation, which could cause operational lag. For example, the minimum interval time can be 1 second. The specific parameter selection can be determined based on the operating frequency that the actual product performance can support.

[0196] Optionally, method 300 further includes: at the end of a preset time period, acquiring third information, the third information including the Q-th frame image and / or second sensor data in a multi-frame image, where Q is a positive integer; determining a second weight vector based on the third information, wherein the N weights in the second weight vector correspond one-to-one with the N LUTs in the sixth LUT group; determining a third LUT based on the second weight vector and the N LUTs in the sixth LUT group; processing the Q-th frame image based on the third LUT to obtain the sixth image; and controlling the display device to display the sixth image.

[0197] Optionally, the first image is a portion of the frames in the multi-frame image, and the method further includes: processing the other images in the multi-frame image except for the portion of the frames according to the first LUT to obtain a seventh image; and controlling the display device to display the seventh image.

[0198] In some embodiments, due to cockpit system performance issues, step S320 cannot be run frequently. Therefore, it can be further simplified to achieve a better video enhancement effect.

[0199] For example, FIG8 shows another schematic block diagram of an image processing system 500 provided in an embodiment of this application. The display engine 512 may include a frame extraction module 5125.

[0200] The image detection module 5122 no longer runs in real time. Instead, when the video starts playing, it extracts a portion of the video frames (such as the first frame, cover frame, and middle frames) using the frame extraction module 5125. The frame extraction module 5125 sends the extracted frame images to the image detection module 5122. The image detection module 5122 can determine the weight vector of the video based on these frame images. When switching videos, it uses the data from the new video to re-extract frames from the new video, thereby updating the output weight vector.

[0201] Optionally, before processing the images other than some of the frames in the multi-frame images according to the first LUT, method 300 further includes: determining that the playback duration of the multi-frame images is less than or equal to a preset duration.

[0202] For example, Figure 9 shows another schematic block diagram of the image processing system 500 provided in an embodiment of this application. Compared to Figure 6, the image processing system 500 shown in Figure 9 does not include an image detection module 5122. After acquiring data from the perception system, vehicle control execution, and operating system inputs, the style control module 5121 can select a target LUT from multiple 3DLUTs. The image enhancement module 5123 can process the image based on the target LUT to obtain a processed image. This method still has a better effect on video viewing experience compared to the current solution that adjusts screen brightness only by ambient light intensity.

[0203] The above method 300 can be executed by the vehicle 100; or by the computing platform 120; or by the processor, chip or circuit in the computing platform 120; or by the image processing system 500.

[0204] Figure 10 shows a schematic block diagram of an image processing apparatus 1000 provided in an embodiment of this application. The apparatus 1000 includes: an acquisition unit 1010, configured to acquire first information, the first information including a first image and / or first sensor data; a determination unit 1020, configured to determine a first weight vector based on the first information, wherein N weights in the first weight vector correspond one-to-one with N LUTs in a first color lookup table (LUT) group, and N is an integer greater than 1; the determination unit 1020 is further configured to determine a first LUT based on the first weight vector and the N LUTs in the first LUT group; an image processing unit 1030, configured to process the first image according to the first LUT to obtain a second image; and a control unit 1040, configured to control a display device to display the second image.

[0205] Optionally, the acquisition unit 1010 is further configured to acquire second information, which includes one or more of the following: data collected by the vehicle's sensors, vehicle control information, and information related to the first image; the determination unit 1020 is configured to determine a first LUT group from multiple LUT groups based on the second information.

[0206] Optionally, the second information includes data collected by a light sensor, wherein when the data collected by the light sensor indicates that the light intensity inside the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by the LUT groups other than the first LUT group; and / or, when the data collected by the light sensor indicates that the light intensity inside the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group.

[0207] Optionally, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is sunny outside the cockpit, the color tone of the image processed by the first LUT group is warmer than the color tone of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is sunny outside the cockpit, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0208] Optionally, the second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is raining or snowing outside the cockpit, the color tone of the image processed by the first LUT group is cooler than the color tone of the image processed by LUT groups other than the first LUT group among multiple LUT groups; and / or, when the environmental information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0209] Optionally, the second information includes vehicle control information. When the vehicle control information indicates that the vehicle speed is greater than or equal to a preset speed, the image processed by the first LUT group has a lower degree of sharpness compared to the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0210] Optionally, the second information includes information related to the first image. When the information related to the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by LUT groups other than the first LUT group among multiple LUT groups; or, when the information related to the first image indicates that the first image is an image of a second style, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by LUT groups other than the first LUT group among multiple LUT groups, and / or, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group among multiple LUT groups.

[0211] Optionally, the determining unit is configured to: determine the second LUT group as the first LUT group when determining the second LUT group from multiple LUT groups based on data collected by the sensor and determining the third LUT group from multiple LUT groups based on vehicle control information; or, determine the fourth LUT group as the first LUT group when determining the second LUT group from multiple LUT groups based on data collected by the sensor, determining the third LUT group from multiple LUT groups based on vehicle control information and determining the fourth LUT group from multiple LUT groups based on information related to the first image.

[0212] Optionally, the control unit is further configured to control the display device to display multiple style options, each style option corresponding to a LUT group; the acquisition unit is further configured to acquire a first LUT group in response to acquiring input from the user selecting a first style option from the multiple style options, the first style option corresponding to the first LUT group.

[0213] Optionally, the acquisition unit 1010 is further configured to acquire a fifth LUT group in response to acquiring input from the user selecting a second style option from multiple style options; the determination unit 1020 is further configured to determine a second LUT based on the first weight vector and N LUTs in the fifth LUT group; the image processing unit 1030 is further configured to process the first image based on the second LUT to obtain a third image; and the control unit 1040 is further configured to control the display device to switch from displaying the second image to displaying the third image.

[0214] Optionally, the first image is the Lth frame image in a multi-frame image, where L is a positive integer; the acquisition unit 1010 is further configured to acquire the Sth frame image in the multi-frame image, where the Sth frame is the frame image after the Lth frame, and S is a positive integer; the determination unit 1020 is further configured to determine a first variation coefficient based on the pixel average value and / or pixel variance of the Lth frame image and the Sth frame image; the image processing unit 1030 is further configured to process the Sth frame image according to the first LUT when the first variation coefficient is less than or equal to a preset variation coefficient, to obtain a fourth image; and the control unit 1040 is further configured to control the display device to display the fourth image.

[0215] Optionally, the first image is the Pth frame image in the multi-frame image, where P is a positive integer. The image processing unit 1030 is further configured to process the images after the Pth frame image in the multi-frame image according to the first LUT within a preset time period after the control unit controls the display device to display the second image, so as to obtain the fifth image. The control unit 1040 is further configured to control the display device to display the fifth image.

[0216] Optionally, the acquisition unit 1010 is further configured to acquire third information at the end of a preset time period, the third information including the Q-th frame image and / or second sensor data in the multi-frame image, where Q is a positive integer; the determination unit 1020 is further configured to determine a second weight vector based on the third information, wherein the N weights in the second weight vector correspond one-to-one with the N LUTs in the sixth LUT group; the determination unit 1020 is further configured to determine a third LUT based on the second weight vector and the N LUTs in the sixth LUT group; the image processing unit 1030 is further configured to process the Q-th frame image based on the third LUT to obtain the sixth image; and the control unit 1040 is further configured to control the display device to display the sixth image.

[0217] Optionally, the first image is a partial frame image in a multi-frame image. The image processing unit 1030 is further configured to process other images in the multi-frame image except for the partial frame image according to the first LUT to obtain the seventh image. The control unit 1040 is further configured to control the display device to display the seventh image.

[0218] Optionally, the determining unit 1020 is further configured to determine that the playback duration of the multi-frame images is less than or equal to a preset duration before the image processing unit processes other images in the multi-frame images, excluding some frame images.

[0219] Optionally, the determining unit 1020 is used to: input the first information into the prediction model to obtain the first weight vector.

[0220] It should be understood that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0221] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0222] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0223] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented as a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.

[0224] This application also provides an image processing apparatus, which includes a processing unit and a storage unit. The storage unit stores instructions, and the processing unit executes the instructions stored in the storage unit to enable the apparatus to perform the methods or steps described in the above embodiments.

[0225] Alternatively, if the image processing device is located in a vehicle, the processing unit may be the processor 151-15n shown in FIG1.

[0226] This application also provides an image processing system, which may include a computing platform and a display screen, and the computing platform may include the image processing device described above.

[0227] This application also provides a vehicle that may include the image processing device or image processing system described above.

[0228] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0229] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0230] This application also provides a chip, which includes a circuit for performing the methods described in the above embodiments.

[0231] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0232] It should be understood that in the embodiments of this application, the memory may include read-only memory and random access memory, and provides instructions and data to the processor.

[0233] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0234] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0235] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0236] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0237] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0238] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0239] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0240] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, include: Acquire first information, which includes a first image and / or first sensor data; Based on the first information, a first weight vector is determined, wherein the N weights in the first weight vector correspond one-to-one with the N LUTs in the first color lookup table LUT group, and N is an integer greater than 1. The first LUT is determined based on the first weight vector and the N LUTs in the first LUT group; Based on the first LUT, the first image is processed to obtain the second image; The control display device displays the second image.

2. The method according to claim 1, characterized in that, The method further includes: Obtain second information, which includes one or more of the following: data collected by the vehicle's sensors, vehicle control information, and information related to the first image; Based on the second information, the first LUT group is determined from multiple LUT groups.

3. The method according to claim 2, characterized in that, The second information includes data collected by the light sensor. When the data collected by the light sensor indicates that the light intensity inside the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by any of the plurality of LUT groups other than the first LUT group; and / or, When the data collected by the light sensor indicates that the light intensity inside the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group.

4. The method according to claim 2, characterized in that, The second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is sunny outside the cockpit, the image processed by the first LUT group has a warmer color tone compared to the image processed by LUT groups other than the first LUT group; and / or, When the environmental information indicates that it is a sunny day outside the cockpit, the image processed by the first LUT group has a higher saturation than the image processed by LUT groups other than the first LUT group.

5. The method according to claim 2, characterized in that, The second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is raining or snowing outside the cockpit, the image processed by the first LUT group has a cooler color tone compared to the image processed by LUT groups other than the first LUT group; and / or, When the environmental information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by LUT groups other than the first LUT group.

6. The method according to claim 2, characterized in that, The second information includes the vehicle control information. When the vehicle control information indicates that the vehicle's speed is greater than or equal to a preset speed, the image processed by the first LUT group has a lower degree of sharpness compared to the image processed by LUT groups other than the first LUT group.

7. The method according to claim 2, characterized in that, The second information includes information related to the first image. When information associated with the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by LUT groups other than the first LUT group; or... When information associated with the first image indicates that the first image is a second-style image, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by LUT groups other than the first LUT group, and / or the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group.

8. The method according to any one of claims 2 to 7, characterized in that, The step of determining the first LUT group from multiple LUT groups based on the second information includes: When determining a second LUT group from multiple LUT groups based on data collected by the sensors, and determining a third LUT group from the multiple LUT groups based on vehicle control information, the second LUT group is designated as the first LUT group; or, When determining a second LUT group from multiple LUT groups based on data collected by sensors, determining a third LUT group from the multiple LUT groups based on vehicle control information, and determining a fourth LUT group from the multiple LUT groups based on information related to the first image, the fourth LUT group is determined as the first LUT group.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: The display device is controlled to display multiple style options, each style option corresponding to a LUT group; In response to receiving input from the user selecting a first style option from multiple style options, the first LUT group is obtained, and the first style option corresponds to the first LUT group.

10. The method according to claim 9, characterized in that, The method further includes: In response to receiving input from the user that they have selected a second style option from the plurality of style options, the fifth LUT group is obtained; The second LUT is determined based on the first weight vector and the N LUTs in the fifth LUT group; Based on the second LUT, the first image is processed to obtain the third image; Control the display device to switch from displaying the second image to displaying the third image.

11. The method according to any one of claims 1 to 10, characterized in that, The first image is the Lth frame in a multi-frame image set, where L is a positive integer. The method further includes: Obtain the S-th frame image from the multi-frame images, where the S-th frame is the frame image following the L-th frame, and S is a positive integer; The first variation coefficient is determined based on the average pixel value and / or pixel variance of the Lth frame image and the Sth frame image; If the first change coefficient is less than or equal to the preset change coefficient, the S-frame image is processed according to the first LUT to obtain the fourth image; Control the display device to display the fourth image.

12. The method according to any one of claims 1 to 10, characterized in that, The first image is the P-th frame image in a multi-frame image, where P is a positive integer. The method further includes: Within a preset time period starting from when the display device displays the second image, the images after the Pth frame in the multi-frame images are processed according to the first LUT to obtain the fifth image; Control the display device to display the fifth image.

13. The method according to claim 12, characterized in that, The method further includes: When the preset duration ends, third information is obtained, which includes the Q-th frame image and / or second sensor data in the multi-frame images, where Q is a positive integer; Based on the third information, a second weight vector is determined, and the N weights in the second weight vector correspond one-to-one with the N LUTs in the sixth LUT group; The third LUT is determined based on the second weight vector and the N LUTs in the sixth LUT group; Based on the third LUT, the Q-th frame image is processed to obtain the sixth image; Control the display device to display the sixth image.

14. The method according to any one of claims 1 to 10, characterized in that, The first image is a subset of frames from a multi-frame image set, and the method further includes: Based on the first LUT, the other images in the multi-frame images, excluding the partial frame images, are processed to obtain the seventh image; Control the display device to display the seventh image.

15. The method according to claim 14, characterized in that, Before processing the images other than the partial frame images in the multi-frame images according to the first LUT, the method further includes: The playback duration of the multi-frame images is determined to be less than or equal to a preset duration.

16. The method according to any one of claims 1 to 15, characterized in that, Determining the first weight vector based on the first information includes: The first information is input into the prediction model to obtain the first weight vector.

17. An image processing apparatus, characterized in that, include: An acquisition unit is configured to acquire first information, the first information including a first image and / or first sensing data; The determining unit is used to determine a first weight vector based on the first information, wherein the N weights in the first weight vector correspond one-to-one with the N LUTs in the first color lookup table LUT group, and N is an integer greater than 1. The determining unit is further configured to determine the first LUT based on the first weight vector and the N LUTs in the first LUT group; An image processing unit is configured to process the first image according to the first LUT to obtain a second image; A control unit is used to control the display device to display the second image.

18. The apparatus according to claim 17, characterized in that, The acquisition unit is further configured to acquire second information, the second information including one or more of the following: data collected by the vehicle's sensors, vehicle control information, and information related to the first image; The determining unit is configured to determine the first LUT group from multiple LUT groups based on the second information.

19. The apparatus according to claim 18, characterized in that, The second information includes data collected by the light sensor. When the data collected by the light sensor indicates that the light intensity inside the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by any of the plurality of LUT groups other than the first LUT group; and / or, When the data collected by the light sensor indicates that the light intensity inside the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group.

20. The apparatus according to claim 18, characterized in that, The second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is sunny outside the cockpit, the image processed by the first LUT group has a warmer color tone compared to the image processed by LUT groups other than the first LUT group; and / or, When the environmental information indicates that it is a sunny day outside the cockpit, the image processed by the first LUT group has a higher saturation than the image processed by LUT groups other than the first LUT group.

21. The apparatus according to claim 18, characterized in that, The second information includes environmental information collected by sensors outside the cockpit. When the environmental information indicates that it is raining or snowing outside the cockpit, the image processed by the first LUT group has a cooler color tone compared to the image processed by LUT groups other than the first LUT group; and / or, When the environmental information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by LUT groups other than the first LUT group.

22. The apparatus according to claim 18, characterized in that, The second information includes the vehicle control information. When the vehicle control information indicates that the vehicle's speed is greater than or equal to a preset speed, the image processed by the first LUT group has a lower degree of sharpness compared to the image processed by LUT groups other than the first LUT group.

23. The apparatus according to claim 18, characterized in that, The second information includes information related to the first image. When information associated with the first image indicates that the first image is an image of a first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by LUT groups other than the first LUT group; or... When information associated with the first image indicates that the first image is a second-style image, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by LUT groups other than the first LUT group, and / or the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by LUT groups other than the first LUT group.

24. The apparatus according to any one of claims 18 to 23, characterized in that, The determining unit is used for: When determining a second LUT group from multiple LUT groups based on data collected by the sensor and determining a third LUT group from the multiple LUT groups based on vehicle control information, the second LUT group is determined as the first LUT group; or, When determining a second LUT group from multiple LUT groups based on data collected by sensors, determining a third LUT group from the multiple LUT groups based on vehicle control information, and determining a fourth LUT group from the multiple LUT groups based on information related to the first image, the fourth LUT group is determined as the first LUT group.

25. The apparatus according to any one of claims 17 to 24, characterized in that, The control unit is also used to control the display device to display multiple style options, each of which corresponds to a LUT group; The acquisition unit is further configured to acquire the first LUT group in response to acquiring input from the user selecting a first style option from multiple style options, wherein the first style option corresponds to the first LUT group.

26. The apparatus according to claim 25, characterized in that, The acquisition unit is further configured to acquire a fifth LUT group in response to receiving input from the user selecting a second style option from the plurality of style options; The determining unit is further configured to determine a second LUT based on the first weight vector and the N LUTs in the fifth LUT group; The image processing unit is further configured to process the first image according to the second LUT to obtain a third image; The control unit is also used to control the display device to switch from displaying the second image to displaying the third image.

27. The apparatus according to any one of claims 17 to 26, characterized in that, The first image is the Lth frame in a multi-frame image set, where L is a positive integer. The acquisition unit is further configured to acquire the S-th frame image in the multi-frame image, wherein the S-th frame is the frame image after the L-th frame, and S is a positive integer; The determining unit is further configured to determine a first variation coefficient based on the pixel average value and / or pixel variance of the Lth frame image and the Sth frame image; The image processing unit is further configured to process the S-frame image according to the first LUT to obtain a fourth image when the first change coefficient is less than or equal to a preset change coefficient. The control unit is also used to control the display device to display the fourth image.

28. The apparatus according to any one of claims 17 to 26, characterized in that, The first image is the Pth frame in a multi-frame image set, where P is a positive integer. The image processing unit is further configured to process the images after the Pth frame in the multi-frame images according to the first LUT within a preset time period after the display device is controlled by the control unit to display the second image, so as to obtain the fifth image; The control unit is also used to control the display device to display the fifth image.

29. The apparatus according to claim 28, characterized in that, The acquisition unit is further configured to acquire third information when the preset duration ends, the third information including the Q-th frame image and / or second sensor data in the multi-frame images, where Q is a positive integer; The determining unit is further configured to determine a second weight vector based on the third information, wherein the N weights in the second weight vector correspond one-to-one with the N LUTs in the sixth LUT group; The determining unit is further configured to determine a third LUT based on the second weight vector and the N LUTs in the sixth LUT group; The image processing unit is further configured to process the Q-frame image according to the third LUT to obtain the sixth image; The control unit is also used to control the display device to display the sixth image.

30. The apparatus according to any one of claims 17 to 26, characterized in that, The first image is a subset of frames from a multi-frame image set. The image processing unit is further configured to process other images in the multi-frame images, excluding the partial frame images, according to the first LUT, to obtain a seventh image; The control unit is also used to control the display device to display the seventh image.

31. The apparatus according to claim 30, characterized in that, The determining unit is further configured to determine, before the image processing unit processes other images in the multi-frame images besides the partial frame images, that the playback duration of the multi-frame images is less than or equal to a preset duration.

32. The apparatus according to any one of claims 17 to 31, characterized in that, The determining unit is used for: The first information is input into the prediction model to obtain the first weight vector.

33. An image processing apparatus, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to cause the apparatus to perform the method as described in any one of claims 1 to 16.

34. An image processing system, characterized in that, It includes a display screen and a computing platform, the computing platform including the apparatus as described in any one of claims 17 to 33.

35. A vehicle, characterized in that, Includes the apparatus as described in any one of claims 17 to 33, or the system as described in claim 34.

36. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, cause the processor to implement the method as described in any one of claims 1 to 16.

37. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 16.

38. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 16.

Images