Image processing method and device and vehicle

Abstract

The invention provides an image processing method and device and a vehicle, which can be applied to the field of intelligent cabins. Acquiring an image and / or sensing data; according to the image and / or sensing data, determining a weight vector, N weights in the weight vector being in one-to-one correspondence with N LUTs in a color lookup table LUT group, N being an integer greater than 1; determining a target LUT according to the weight vector and N LUTs in the LUT group; according to the target LUT, processing the image to obtain a processed image; and controlling the display device to display the processed image. The method and the device can be applied to an intelligent automobile or an electric automobile, and are beneficial to improving the watching effect when a user watches an image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent cockpit, and more particularly, to an image processing method and device and vehicle. BACKGROUND

[0002] With the rapid development of the automobile industry, the automobile is not only a means of transportation, but also more and more focuses on the driving experience. The property of the automobile as the second home is becoming stronger and stronger, and the in-vehicle viewing feature is also more and more popular with consumers. At present, the vehicle is basically equipped with a central control large screen, and a large part of the vehicle models are equipped with a co-pilot screen, a rear screen, a viewing curtain and other multiple screens. It can be seen that consumers and manufacturers attach great importance to the viewing demand. At present, in view of the changeable environment, most automobile companies use in-vehicle and out-of-vehicle ambient light sensors to automatically control the screen brightness to realize the adjustment of the display effect under different environments. However, such adjustment method is too simple and cannot achieve better viewing effect. SUMMARY

[0003] The present application provides an image processing method, device and vehicle, which helps to improve the viewing effect of users when watching images.

[0004] In a first aspect, the present application provides an image processing method, which comprises: acquiring first information, the first information comprising a first image and / or first sensing data; determining a first weight vector according to the first information, N weights in the first weight vector corresponding to N first color look up tables (LUTs) in a first LUT group one by one, N being an integer greater than 1; determining a first LUT according to the first weight vector and the N first LUTs; processing the first image according to the first LUT to obtain a second image; and controlling a display device to display the second image.

[0005] Based on the above technical solution, through the image and / or sensing data, multiple weights can be obtained, so that the first LUT can be determined based on the multiple weights and the multiple LUTs. By processing the first image through the first LUT, the second image can be obtained. In this way, the target LUT matched with the original image and / or sensing data can be obtained, and the original image is processed based on the target LUT for image enhancement, which helps to improve the visual effect of the image, and the processed image can be more consistent with the current viewing scene, and helps to improve the viewing effect of users when watching images.

[0006] In some possible implementation manners, the first image is one frame of image in a video.

[0007] In some possible implementation manners, the first LUT is determined according to the first weight vector and the N LUTs in the first LUT group, including: the N LUTs in the first LUT group are fused according to the N weights to obtain the first LUT.

[0008] In some possible implementation manners, the first sensing data can be data collected by a sensor of the vehicle. For example, the sensor can include a sunshine and rain sensor, a camera, or the like.

[0009] In some possible implementation manners, the first LUT group can be a preset LUT group.

[0010] In some possible implementation manners, the first LUT can be a three-dimension lookup table (3-Dimension lookup table, 3DLUT).

[0011] With reference to the first aspect, in some implementation manners of the first aspect, the method further includes: obtaining second information, the second information including one or more of data collected by a sensor of the vehicle, vehicle control information, and information related to the first image; and determining the first LUT group from a plurality of LUT groups according to the second information.

[0012] Based on the above technical solution, the first LUT group can be selected from the plurality of LUT groups according to at least one of the data collected by the sensor, the vehicle control information, and the information related to the first image. In this way, the determined first LUT group can be matched with the current viewing scene, so that the processed image can be more consistent with the current viewing scene, which helps to improve the visual effect of the image and also helps to improve the viewing effect of the user when viewing the image.

[0013] With reference to the first aspect, in some implementation manners of the first aspect, the second information includes data collected by a light sensor, and when the data collected by the light sensor indicates that the light intensity in the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or when the data collected by the light sensor indicates that the light intensity in the vehicle cabin is greater than or equal to the preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by LUT groups other than the first LUT group in the plurality of LUT groups.

[0014] Based on the above technical solution, when the light condition in the vehicle cabin is good, the brightness and the contrast of the image processed by the first LUT group determined from the plurality of LUT groups are higher, which helps to improve the viewing effect of the user in the vehicle cabin when the light condition is good.

[0015] In some implementations of the first aspect, the second information includes environment information collected by a sensor outside the cockpit, when the environment information indicates that it is sunny outside the cockpit, the color tone of the image processed by the first LUT group is warmer than the color tone of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups, and / or when the environment information indicates that it is sunny outside the cockpit, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups.

[0016] Based on the above technical solution, when the environment information outside the cockpit indicates that it is sunny, the color tone of the image processed by the first LUT group determined from the plurality of LUT groups is warmer and the saturation is higher, which helps to improve the viewing effect of the user in the cockpit when it is sunny.

[0017] In some implementations of the first aspect, the second information includes environment information collected by a sensor outside the cockpit, when the environment information indicates that it is raining or snowing outside the cockpit, the color tone of the image processed by the first LUT group is cooler than the color tone of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups, and / or when the environment information indicates that it is raining or snowing outside the cockpit, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups.

[0018] Based on the above technical solution, when the environment information outside the cockpit indicates that it is raining or snowing, the color tone of the image processed by the first LUT group determined from the plurality of LUT groups is cooler and the saturation is lower, which helps to improve the viewing effect of the user in the cockpit when it is raining or snowing.

[0019] In some implementations of the first aspect, the second information includes vehicle control information, when the vehicle control information indicates that the speed of the vehicle is greater than or equal to a preset speed, the sharpening degree of the image processed by the first LUT group is lower than the sharpening degree of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups.

[0020] Based on the above technical solution, when the speed of the vehicle is high, the image processed by the first LUT group determined from the plurality of LUT groups is softer, which helps to improve the viewing effect of the user in the cockpit when the vehicle is at a high speed.

[0021] With reference to the first aspect, in some implementations of the first aspect, the second information comprises information related to the first image, when the information related to the first image indicates that the first image is an image of the first style, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups, or when the information related to the first image indicates that the first image is an image of the second style, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups, and / or the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by the LUT group other than the first LUT group in the plurality of LUT groups.

[0022] Based on the above technical solution, when the first image is an image of the first style (for example, a science fiction film), the image processed by the first LUT group determined from the plurality of LUT groups is darker and more textured, which is more suitable for the image of the first style. When the first image is an image of the second style (for example, a comedy film), the image processed by the first LUT group determined from the plurality of LUT groups will present the effect of color highlighting, higher contrast and saturation, which is more suitable for the image of the second style.

[0023] With reference to the first aspect, in some implementations of the first aspect, the first LUT group is determined from the plurality of LUT groups according to the second information, comprising: when the second LUT group is determined from the plurality of LUT groups according to the data collected by the sensor and the third LUT group is determined from the plurality of LUT groups according to the vehicle control information, the second LUT group is determined as the first LUT group; or when the second LUT group is determined from the plurality of LUT groups according to the data collected by the sensor, the third LUT group is determined from the plurality of LUT groups according to the vehicle control information, and the fourth LUT group is determined from the plurality of LUT groups according to the information related to the first image, the fourth LUT group is determined as the first LUT group.

[0024] Based on the above technical solution, after different LUT groups are obtained through different dimensions, the target LUT group can be selected based on the priority information of each dimension. For example, after different LUT groups are selected through the data collected by the sensor and the vehicle control information respectively, the LUT group determined through the data collected by the sensor can be determined as the target LUT group. For another example, after different LUT groups are selected through the data collected by the sensor, the vehicle control information and the information related to the first image respectively, the LUT group determined through the information related to the first image can be determined as the target LUT group.

[0025] In some possible implementation manners, the priority of the LUT group determined by the information related to the first image is higher than the priority of the LUT group of the data collected by the sensor, and the priority of the LUT group of the data collected by the sensor is higher than the priority of the LUT group determined by the vehicle control information.

[0026] With reference to the first aspect, in some implementation manners of the first aspect, the first LUT group is determined from the plurality of LUT groups according to the second information, including: the first LUT group is determined from the plurality of LUT groups according to the second information and a first mapping relationship, the first mapping relationship including a corresponding relationship between the vehicle control information, the light intensity, the video content and the 3DLUT groups.

[0027] With reference to the first aspect, in some implementation manners of the first aspect, the first mapping relationship includes a corresponding relationship between the vehicle control information, the environment and weather condition, the light intensity, the video content and the 3DLUT groups.

[0028] With reference to the first aspect, in some implementation manners of the first aspect, the method further includes: controlling the display device to display a plurality of style options, each of the plurality of style options corresponding to a LUT group; in response to obtaining an input of a user selecting a first style option from the plurality of style options, obtaining the first LUT group, the first style option corresponding to the first LUT group.

[0029] Based on the above technical solution, the display device can display a plurality of style options, and the plurality of style options can correspond to different LUT groups. The first LUT group can be determined from the plurality of LUT groups based on the selection of the user, so that the final image processing result can be more in line with the expectation of the user.

[0030] In some possible implementation manners, the method further includes: determining the first LUT group from the plurality of LUT groups according to the second information; and controlling the display device to recommend the first style option to the user, the first style option corresponding to the first LUT group.

[0031] Based on the above technical solution, the LUT group matched with the current scene can be obtained through the second information, so that the style option corresponding to the LUT group can be recommended to the user. In this way, the style option matched with the current scene can be recommended to the user without the relevant image processing knowledge of the user, which helps the user to clearly determine the style option more in line with the current scene from the plurality of style options, and avoids the process of the user viewing the plurality of style options.

[0032] With reference to the first aspect, in some implementations of the first aspect, the method further includes: in response to obtaining the input of the user selecting the second style option from the plurality of style options, obtaining a fifth LUT set; determining a second LUT according to the first weight vector and the N LUTs in the fifth LUT set; processing the first image according to the second LUT to obtain a third image; and controlling the display device to switch from displaying the second image to displaying the third image.

[0033] Based on the above technical solution, when the input of the user updating the style option is detected, the LUT set corresponding to the updated style option can be selected to determine the second LUT. Thus, the second LUT is used to process the image to obtain the updated image. After the user switches the style option, the LUT set can be switched. When the image does not change, the first weight vector can still be used, and the quick switching of the image style can be realized.

[0034] With reference to the first aspect, in some implementations of the first aspect, the first image is an Lth image in a plurality of images, L being a positive integer, and the method further includes: obtaining an Sth image in the plurality of images, the Sth image being one image after the Lth image, S being a positive integer; determining a first change coefficient according to the pixel average value and / or the pixel variance of the Lth image and the Sth image; in a case where the first change coefficient is less than or equal to a preset change coefficient, processing the Sth image according to the first LUT to obtain a fourth image; and controlling the display device to display the fourth image.

[0035] Based on the above technical solution, when the change coefficient of the current frame image and the historical frame image is less than or equal to the preset change coefficient, the LUT determined by the historical frame can be used to perform image enhancement on the current frame image. In this way, in a case where the change between the current frame image and the historical frame image is small, the LUT determined by the historical frame image can be used for image enhancement, which helps to improve the viewing effect of the user. At the same time, the calculation overhead caused by switching the weight vector can be saved.

[0036] With reference to the first aspect, in some implementations of the first aspect, the first image is a Pth image in a plurality of images, P being a positive integer, and the method further includes: within a preset time length since the display device displays the second image, processing images after the Pth image in the plurality of images according to the first LUT to obtain a fifth image; and controlling the display device to display the fifth image.

[0037] Based on the above technical solution, within the preset time length since the second image is displayed, the first LUT can be used to process the images after the Pth image, which can save the calculation overhead caused by switching the weight vector. For a device with low computing power, system lag can be avoided.

[0038] With reference to the first aspect, in some implementations of the first aspect, the method further includes: obtaining third information at the end of the preset time length, the third information including a Qth frame of the multiple frames of images and / or the second sensing data, Q being a positive integer; determining a second weight vector according to the third information, the N weights in the second weight vector corresponding to the N LUTs in the sixth LUT group one by one; determining a third LUT according to the second weight vector and the N LUTs in the sixth LUT group; processing the Qth frame of images according to the third LUT to obtain a sixth image; and controlling the display device to display the sixth image.

[0039] Based on the above technical solution, after the end of the preset time length, the weight vector can be reacquired based on the Qth frame of images and / or the second sensing data, so that a new LUT can be calculated. By keeping the weight vector unchanged within the preset time length, the calculation overhead caused by switching the weight vector can be saved. For devices with lower computing power, system lag can be avoided.

[0040] With reference to the first aspect, in some implementations of the first aspect, the first image is a partial frame of image in the multiple frames of images, and the method further includes: processing other images in the multiple frames of images except the partial frame of image according to the first LUT to obtain a seventh image; and controlling the display device to display the seventh image.

[0041] Based on the above technical solution, the first LUT can be obtained using a partial frame of image in the multiple frames of images, and then other frames of image in the multiple frames of images can be processed using the first LUT.

[0042] In some possible implementations, the partial frame of image can be a first frame, a middle frame or a cover of the multiple frames of images.

[0043] With reference to the first aspect, in some implementations of the first aspect, before processing other images in the multiple frames of images except the partial frame of image according to the first LUT, the method further includes: determining that a playing time length of the multiple frames of images is less than or equal to the preset time length.

[0044] Based on the above technical solution, since the pixel change rate in the multiple frames of images with a short playing time length can not be too large, the first LUT can be obtained using a partial frame of image in the multiple frames of images, and then other frames of image in the multiple frames of images can be processed using the first LUT. In this way, the multiple frames of images can be guaranteed to be image-enhanced, and the calculation overhead caused by switching the weight vector can be avoided.

[0045] With reference to the first aspect, in some implementations of the first aspect, determining the first weight vector according to the first information includes: inputting the first information into a prediction model to obtain the first weight vector.

[0046] In some possible implementation manners, the first information comprises a first image, and the prediction model comprises three parts, a first part is a preprocessing module, used for uniformly scaling the first image to a smaller size, facilitating subsequent network processing and reducing a calculation amount; a second part is a multi-layer convolutional neural network, used for extracting deep features in the image; and a third part is a fully connected network, used for outputting N weights and normalizing a weight range of the weights through a post-processing layer.

[0047] In a second aspect, the present application provides an image processing apparatus, comprising: an acquisition unit, configured to acquire first information, the first information comprising a first image and / or first sensing data; a determination unit, configured to determine a first weight vector according to the first information, N weights in the first weight vector corresponding to N LUTs in a first LUT group one by one, N being an integer greater than 1; the determination unit is further configured to determine a first LUT according to the first weight vector and the N LUTs in the first LUT group; and an image processing unit, configured to process the first image according to the first LUT to obtain a second image; and a control unit, configured to control a display apparatus to display the second image.

[0048] In combination with the second aspect, in some implementation manners of the second aspect, the acquisition unit is further configured to acquire second information, the second information comprising one or more of data collected by a sensor of the vehicle, vehicle control information, and information related to the first image; and the determination unit is configured to determine the first LUT group from a plurality of LUT groups according to the second information.

[0049] In combination with the second aspect, in some implementation manners of the second aspect, the second information comprises data collected by an illumination sensor, and when the data collected by the illumination sensor indicates that an illumination intensity in a cabin of the vehicle is greater than or equal to a preset illumination intensity, brightness of an image processed by the first LUT group is greater than brightness of images processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or, when the data collected by the illumination sensor indicates that the illumination intensity in the cabin is greater than or equal to the preset illumination intensity, contrast of the image processed by the first LUT group is greater than contrast of the images processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

[0050] In combination with the second aspect, in some implementation manners of the second aspect, the second information comprises environmental information collected by a sensor outside the cabin, and when the environmental information indicates that it is sunny outside the cabin, a color tone of an image processed by the first LUT group is warmer than color tones of images processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or, when the environmental information indicates that it is sunny outside the cabin, saturation of the image processed by the first LUT group is higher than saturation of the images processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

[0051] With reference to the second aspect, in some implementations of the second aspect, the second information comprises environment information collected by a sensor outside the cabin, when the environment information indicates that it is raining or snowing outside the cabin, a hue of the image processed by the first LUT set is colder than a hue of an image processed by a LUT set other than the first LUT set among the plurality of LUT sets; and / or, when the environment information indicates that it is raining or snowing outside the cabin, a saturation of the image processed by the first LUT set is lower than a saturation of an image processed by a LUT set other than the first LUT set among the plurality of LUT sets.

[0052] With reference to the second aspect, in some implementations of the second aspect, the second information comprises vehicle control information, when the vehicle control information indicates that a speed of the vehicle is greater than or equal to a preset speed, a sharpening degree of the image processed by the first LUT set is lower than a sharpening degree of an image processed by a LUT set other than the first LUT set among the plurality of LUT sets.

[0053] With reference to the second aspect, in some implementations of the second aspect, the second information comprises information related to the first image, when the information related to the first image indicates that the first image is a first style of image, a brightness of the image processed by the first LUT set is lower than a brightness of an image processed by a LUT set other than the first LUT set among the plurality of LUT sets; or, when the information related to the first image indicates that the first image is a second style of image, a contrast of the image processed by the first LUT set is higher than a contrast of an image processed by a LUT set other than the first LUT set among the plurality of LUT sets, and / or, a saturation of the image processed by the first LUT set is higher than a saturation of an image processed by a LUT set other than the first LUT set among the plurality of LUT sets.

[0054] With reference to the second aspect, in some implementations of the second aspect, the determining unit is configured to: determine the second LUT set as the first LUT set when the second LUT set is determined from the plurality of LUT sets according to the data collected by the sensor and a third LUT set is determined from the plurality of LUT sets according to the vehicle control information; or determine a fourth LUT set as the first LUT set when the second LUT set is determined from the plurality of LUT sets according to the data collected by the sensor, the third LUT set is determined from the plurality of LUT sets according to the vehicle control information, and the fourth LUT set is determined from the plurality of LUT sets according to the information related to the first image.

[0055] With reference to the second aspect, in some implementations of the second aspect, the control unit is further configured to control the display device to display a plurality of style options, each of the plurality of style options corresponding to a LUT group; and the obtaining unit is further configured to, in response to obtaining an input of selection of a first style option from the plurality of style options by the user, obtain a first LUT group corresponding to the first style option.

[0056] With reference to the second aspect, in some implementations of the second aspect, the obtaining unit is further configured to, in response to obtaining an input of selection of a second style option from the plurality of style options by the user, obtain a fifth LUT group; the determining unit is further configured to determine a second LUT according to the first weight vector and N LUTs in the fifth LUT group; the image processing unit is further configured to process the first image according to the second LUT to obtain a third image; and the control unit is further configured to control the display device to switch from displaying the second image to displaying the third image.

[0057] With reference to the second aspect, in some implementations of the second aspect, the first image is an Lth image in a plurality of images, L being a positive integer, the obtaining unit is further configured to obtain an Sth image in the plurality of images, the Sth image being one image after the Lth image, S being a positive integer; the determining unit is further configured to determine a first variation coefficient according to a pixel mean value and / or a pixel variance of the Lth image and the Sth image; the image processing unit is further configured to, in a case where the first variation coefficient is less than or equal to a preset variation coefficient, process the Sth image according to the first LUT to obtain a fourth image; and the control unit is further configured to control the display device to display the fourth image.

[0058] With reference to the second aspect, in some implementations of the second aspect, the first image is a Pth image in a plurality of images, P being a positive integer, the image processing unit is further configured to, within a preset time duration since the control unit controls the display device to display the second image, process images after the Pth image in the plurality of images according to the first LUT to obtain a fifth image; and the control unit is further configured to control the display device to display the fifth image.

[0059] With reference to the second aspect, in some implementations of the second aspect, the obtaining unit is further configured to, at the end of the preset time duration, obtain third information including a Qth image in the plurality of images and / or second sensing data, Q being a positive integer; the determining unit is further configured to determine a second weight vector according to the third information, N weights in the second weight vector corresponding to N LUTs in a sixth LUT group in a one-to-one manner; the determining unit is further configured to determine a third LUT according to the second weight vector and the N LUTs in the sixth LUT group; the image processing unit is further configured to process the Qth image according to the third LUT to obtain a sixth image; and the control unit is further configured to control the display device to display the sixth image.

[0060] With reference to the second aspect, in some implementations of the second aspect, the first image is a partial frame image in a plurality of frame images, and the image processing unit is further configured to process other images in the plurality of frame images except the partial frame image according to the first LUT to obtain a seventh image; and the control unit is further configured to control the display device to display the seventh image.

[0061] With reference to the second aspect, in some implementations of the second aspect, the determining unit is further configured to determine that a playing time length of the plurality of frame images is less than or equal to a preset time length before the image processing unit processes the other images in the plurality of frame images except the partial frame image.

[0062] With reference to the second aspect, in some implementations of the second aspect, the determining unit is configured to: input the first information into the prediction model to obtain the first weight vector.

[0063] The present application can be applied to a vehicle-mounted viewing scene, and can also be applied to other fields, such as a smart home viewing scene. The present application can also be used in the following scenario: in a smart audio and video system of a smart screen, external environment light information is obtained through a light intensity sensor, and a suitable LUT group is selected in combination with video information. In addition, the present application can also be applied to a recording video scene of a mobile phone and other photographic devices. Through a sensor attached to the mobile phone, the external environment is sensed, different LUT groups are applied, and the recording content is recorded to improve the video recording effect.

[0064] In a third aspect, the present application provides an image processing device, which includes a processor and a memory. The memory is configured to store instructions, and the processor is configured to execute the instructions stored in the memory, so that the device executes any possible method in the first aspect.

[0065] In a fourth aspect, the present application provides an image processing system, which includes a computing platform and a display screen. The computing platform includes any possible device in the second aspect or the third aspect.

[0066] In some possible implementations, the image processing system further includes a sensing system.

[0067] In a fifth aspect, the present application provides a vehicle, which includes any possible device in the second aspect or the third aspect, or includes the system in the fourth aspect.

[0068] In a sixth aspect, the present application provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer program code causes the computer to execute any possible method in the first aspect.

[0069] It should be noted that the computer program codes described above can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or packaged separately from the processor, and the embodiments of the present application do not make a specific limitation in this regard.

[0070] In a seventh aspect, the present application provides a computer readable medium, which stores program codes, when the program codes are run on a computer, the computer is caused to execute any possible method in the first aspect.

[0071] In an eighth aspect, the present application provides a chip system, which comprises a processor, and is configured to invoke computer programs or computer instructions stored in a memory, so that the processor executes any possible method in the first aspect.

[0072] With reference to the eighth aspect, in a possible implementation, the processor is coupled with the memory through an interface.

[0073] With reference to the eighth aspect, in a possible implementation, the chip system further comprises a memory, and the memory stores computer programs or computer instructions.

[0074] In a ninth aspect, the present application provides a chip, which comprises a circuit, and the circuit is configured to execute any possible method in the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 is a functional block diagram of a vehicle provided by an embodiment of the present application.

[0076] Figure 2 is a schematic diagram of a vehicle cabin scene provided by an embodiment of the present application.

[0077] Figure 3 is a schematic flowchart of an image processing method provided by an embodiment of the present application.

[0078] Figure 4 is a human-machine interface (HMI) provided by an embodiment of the present application.

[0079] Figure 5 is a schematic block diagram of an image processing system provided by an embodiment of the present application.

[0080] Figure 6 is another schematic block diagram of an image processing system provided by an embodiment of the present application.

[0081] Figure 7 is another schematic block diagram of an image processing system provided by an embodiment of the present application.

[0082] Figure 8This is another schematic block diagram of the image processing system provided in the embodiments of this application.

[0083] Figure 9 This is another schematic block diagram of the image processing system provided in the embodiments of this application.

[0084] Figure 10 This is a schematic block diagram of the image processing apparatus provided in the embodiments of this application. Detailed Implementation

[0085] Figure 1 This is a functional block diagram of a vehicle provided in an embodiment of this application. For example... Figure 1 As shown, the vehicle 100 may include a display device 130 and a computing platform 150. The display device 130 in the cabin is mainly divided into two categories: the first is an in-vehicle display screen; the second is a projection display screen, such as a head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple display screens can be installed in the cabin, such as a digital instrument display screen, a central control screen, a display screen in front of the passenger in the front passenger seat (also known as the front row passenger), a display screen in front of the left rear passenger, a display screen in front of the right rear passenger, and even the windows can be used as display screens.

[0086] Part or all functions of the vehicle 100 can be controlled by the computing platform 150. The computing platform 150 can include processors 151-15n, which are circuits having a processing capability of signals. In one implementation, the processors can be circuits having an instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), a digital signal processor (DSP), and the like. In another implementation, the processors can be circuits having a certain function implemented by a logic relationship of hardware circuits, which is fixed or can be reconfigured, such as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) implemented hardware circuit, such as a field programmable gate array (FPGA). In the reconfigurable hardware circuit, the processor loads a configuration document to implement the hardware circuit configuration, which can be understood as the process of the processor loading instructions to implement the functions of the above part or all units. In addition, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), and the like. In addition, the computing platform 150 can also include a memory for storing instructions, and part or all of the processors 151-15n can call the instructions in the memory to implement corresponding functions.

[0087] Optionally, the above structure of the vehicle 100 is only schematic, and in actual application, each component in the vehicle 100 described above can be added or deleted according to actual needs.

[0088] Figure 2This is a schematic diagram of a vehicle cockpit scenario provided in an embodiment of this application. The smart cockpit includes one or more in-vehicle displays (or in-vehicle screens), including but not limited to display screen 201 (or central control screen), display screen 202 (or passenger entertainment screen), display screen 203 (or driver's headrest rear screen), display screen 204 (or passenger headrest rear screen), display screen 205 (or second-row entertainment screen) mounted on the cockpit ceiling, and an instrument panel. Further, displays 201 to 205 can display a graphical user interface (GUI), which may include icons for one or more applications, and / or one or more cards. For example, Figure 1 The display device 130 shown may be one or more of displays 201 to 205. In some possible implementations, display 201 may also be a long screen extending into the passenger area. In addition, display 205 may also be a projection screen associated with a projector, which may be associated with a desktop launcher, through which applications projected by the projector onto the projection screen can be managed.

[0089] Figure 2 The cockpit can also be equipped with one or more cameras to capture images inside or outside the cockpit, such as cameras from a driver monitor system (DMS), a cabin monitor system (CMS), and a dashcam. These cameras can be the same or different cameras. In addition, one or more pressure sensors and acoustic sensors are installed in the cockpit to monitor the presence and location of users.

[0090] It should be understood that the method for controlling the display in the following embodiments is based on... Figure 2 The embodiments shown are based on a 5-seater vehicle, but are not limited to this. For example, for a 7-seater sport / suburban utility vehicle (SUV), the cabin may include a central control screen, a passenger entertainment screen, a screen behind the driver's headrest, a screen behind the passenger's headrest, entertainment screens in the left-hand area of ​​the third row, and entertainment screens in the right-hand area of ​​the third row. As another example, for a bus, the cabin may include front and rear entertainment screens; or, the cabin may include a display screen in the driver's area and an entertainment screen in the passenger area. Furthermore, the following embodiments use a left-hand drive vehicle (i.e., the driver is on the left side of the vehicle) as an example; in actual implementation, the vehicle may also be a right-hand drive vehicle (i.e., the driver is on the right side of the vehicle).

[0091] As mentioned above, with the rapid development of the automobile industry, the automobile is not only a means of transportation, but also more and more focuses on the driving experience, and the property of the "second home" of the automobile is more and more intense, and the in-vehicle viewing feature is more and more popular with consumers. At present, the vehicle is basically equipped with a central control large screen, and a large part of the vehicle models are equipped with a co-pilot screen, a rear row screen, a viewing curtain and other multiple screens, which shows that consumers and manufacturers attach importance to the viewing demand. At present, in view of the changeable environment, most vehicle manufacturers use in-vehicle and out-of-vehicle ambient light sensors to automatically control the screen brightness to achieve the adjustment of the display effect under different environments. However, such adjustment method is too simple and cannot achieve better viewing effect.

[0092] The embodiment of the present application provides an image processing method, device and vehicle, which obtains a target LUT matched with original image and / or sensor data through image and / or sensor data, and then performs image enhancement processing on the original image based on the target LUT, so that the processed image is more consistent with the current viewing scene, which helps to improve the visual effect of the image and also helps to improve the viewing effect of the user when watching the image.

[0093] Figure 3 A schematic flowchart of an image processing method 300 provided by the embodiment of the present application is shown. The method 300 includes:

[0094] S310, obtaining first information, the first information including a first image and / or first sensor data.

[0095] Exemplarily, the first image can be a certain frame image or multiple frame images in a video source to be played.

[0096] Exemplarily, taking an example that the method 300 is executed by a vehicle, the first sensor data includes data collected by a sensor of the vehicle. For example, the first sensor data can include one or more of data collected by an illumination sensor in the cabin, data collected by an illumination sensor outside the cabin, and data collected by a sunshine and rainfall sensor, which can be used to determine the illumination intensity of the current environment of the vehicle.

[0097] For another example, the first sensor data includes data collected by a sensor (for example, a camera, an illumination sensor) in the cabin and / or data collected by a sensor (for example, a camera, an illumination sensor) outside the cabin, which can be used to indicate one or more of the current ambient light intensity, the light source type (for example, direct sunlight, natural light, artificial light source, etc.), the light direction (for example, backlight, backlight, etc.), and the environment type (for example, indoor garage, outdoor garage, mountainous area, highway, etc.).

[0098] Exemplarily, the first sensing data can include vehicle control information, which can be data collected by a vehicle control sensor (e.g., a gear sensor, a vehicle speed sensor, etc.). Alternatively, the vehicle control information can be used to indicate the current driving state of the vehicle, for example, the vehicle control information can be used to indicate the gear and / or the vehicle speed, etc. currently in which the vehicle is located.

[0099] Exemplarily, the first sensing data can include information related to the first image, for example, one or more of the type of the first image (e.g., a movie, a live broadcast, a short video, etc.), the content style (e.g., comedy, ancient style, science fiction, etc.), and the playing time length.

[0100] Optionally, according to the first information, the first weight vector is determined, including: inputting the first information into a prediction model to obtain the first weight vector.

[0101] S320, according to the first information, a first weight vector is determined, N weights in the first weight vector correspond to N LUTs in the first LUT group one by one, N is an integer greater than 1.

[0102] Optionally, according to the first information, the first weight vector is determined, including: inputting the first information into a prediction model to obtain the first weight vector.

[0103] Exemplarily, the first information includes the first image, and the prediction model includes three parts, the first part is a preprocessing module, which is used to uniformly scale the first image to a smaller size (e.g., 256*256 pixels), so as to facilitate subsequent network processing and reduce the amount of calculation; the second part is a multi-layer convolutional neural network, which is used to extract deep features in the image; the third part is a fully connected network, which is used to output N weights, and the weight range of the weights is normalized through a post-processing layer.

[0104] Optionally, before the method 300 is executed, M 3DLUT groups can be generated by training, M is an integer greater than 1. Exemplarily, each 3DLUT group is suitable for different external environment scenes, film source types, and film source styles.

[0105] For example, in a dimly lit basement, a movie is played, and the image processed by the 3DLUT group used is dark and has more texture. For another example, in outdoor driving, an animation is played, and the image processed by the 3DLUT group used has high contrast and higher saturation. M can depend on the actual performance style category. Exemplarily, M can be 3, divided into three styles of strong contrast, medium contrast, and weak contrast. Exemplarily, the style of color temperature can also be increased, such as cold color and warm color. M can be increased according to the required color tone, color temperature, etc.

[0106] Optionally, each 3DLUT group curve has N 3DLUTs, for real-time video content recognition, dynamically adjusting the 3DLUTs used for each frame of image. Exemplarily, N can be 3, representing the adjustment strength for high-light area, medium-light area and low-light area of the video respectively. The greater the number of N, the more flexible the real-time adjustment.

[0107] Exemplarily, the above M 3DLUT groups and the prediction model can be obtained through training. The steps of training include but are not limited to the following steps (1)-(3):

[0108] (1) Prepare high-quality video source data, and optimize in different environments to obtain reference data.

[0109] (2) Prepare low-quality video data corresponding to the content, and perform degradation processing according to different parameters (for example, reduce the contrast of the image or downsample, so that the video quality is reduced), to obtain another reference data.

[0110] (3) Train the data pairs obtained in steps (1) and (2) in the network to obtain the prediction model and the M 3DLUT groups.

[0111] The training of the prediction model and the training of the multiple 3DLUT groups mentioned in the foregoing can be performed simultaneously, so that the weight vector output by the prediction model and the multiple 3DLUT groups are matched.

[0112] The above training method is only exemplary, and the embodiments of the present application are not limited thereto. For example, the 3DLUTs can also be generated by manual adjustment by professional engineers, and in the training, the existing 3DLUT groups are used and fixed, and only the prediction model is trained.

[0113] Optionally, the method 300 further includes: obtaining second information, the second information including one or more of data collected by a sensor of the vehicle, vehicle control information, and information related to the first image; and determining the first LUT group from the multiple LUT groups according to the second information.

[0114] The above data collected by the sensor of the vehicle can refer to the description of the data collected by the sensor in the cabin and / or the data collected by the sensor outside the cabin, and the vehicle control information and the information related to the first image can refer to the description in the above embodiments, which will not be repeated here.

[0115] Optionally, the second information comprises data collected by a light sensor, and when the data collected by the light sensor indicates that the light intensity in the vehicle cabin is greater than or equal to a preset light intensity, the brightness of the image processed by the first LUT set is greater than the brightness of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets; and / or when the data collected by the light sensor indicates that the light intensity in the cabin is greater than or equal to the preset light intensity, the contrast of the image processed by the first LUT set is greater than the contrast of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0116] For example, Table 1 shows the correspondence between the light intensity and the 3DLUT set provided by the embodiment of the application.

[0117] Table 1

[0118] Light intensity 3DLUT group Strong 3DLUT group 1 Medium 3DLUT group 2 Weak 3DLUT group 3 … …

[0119] For example, the brightness and contrast of the image processed by the 3DLUT set 1 are higher than the brightness and contrast of the image processed by the 3DLUT set 2 or the 3DLUT set 3, and are more suitable for viewing in strong light.

[0120] Optionally, the data collected by the light sensor can directly determine the light intensity. Further, the direction of light and the position of the viewer can be perceived in combination with the camera inside or outside the cabin, so that more accurate judgment can be made. For example, when the light intensity in the front row area is strong and the light intensity in the rear row area is weak, the 3DLUT set 1 can be used to process the video content played on the center control screen or the co-pilot screen, and the 3DLUT set 3 can be used to process the video content played on the center control screen or the co-pilot screen. In this way, the front row passengers and the rear row passengers can have better effects.

[0121] Optionally, the second information comprises environmental information collected by a sensor outside the cabin, and when the environmental information indicates that it is sunny outside the cabin, the hue of the image processed by the first LUT set is warmer than the hue of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets; and / or when the environmental information indicates that it is sunny outside the cabin, the saturation of the image processed by the first LUT set is higher than the saturation of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0122] Optionally, the second information comprises environmental information collected by a sensor outside the cabin, when the environmental information indicates that it is raining or snowing outside the cabin, the hue of the image processed by the first LUT set is colder than the hue of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets; and / or, when the environmental information indicates that it is raining or snowing outside the cabin, the saturation of the image processed by the first LUT set is lower than the saturation of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0123] For example, Table 2 shows the correspondence between the environment and weather conditions and the 3DLUT sets provided by the embodiments of the present application.

[0124] Table 2

[0125] Environment and weather condition 3DLUT group Outdoor sunny day 3DLUT group 4 Outdoor cloudy, shadow area, early morning or evening 3DLUT group 5 Indoor, outdoor night 3DLUT group 6 Outdoor rainy and snowy day 3DLUT group 7 … …

[0126] For example, the image processed by the 3DLUT set 4 is warmer in hue and higher in saturation than the image processed by other 3DLUT sets, and is more suitable for viewing in a sunny scene; the image processed by the 3DLUT set 7 is colder in hue and lower in saturation than the image processed by other 3DLUT sets, and is suitable for viewing in a rainy day.

[0127] Optionally, the second information comprises vehicle control information, when the vehicle control information indicates that the speed of the vehicle is greater than or equal to a preset speed, the sharpening degree of the image processed by the first LUT set is lower than the sharpening degree of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0128] For example, Table 3 shows the correspondence between the vehicle driving state and the 3DLUT provided by the embodiments of the present application.

[0129] Table 3

[0130] Vehicle driving state 3DLUT group Non-D gear 3DLUT group 8 D range and vehicle speed < 60 km / h 3DLUT group 9 D gear and vehicle speed ≥ 60 km / h 3DLUT group 10 … …

[0131] For example, the sharpening degree of the image processed by the 3DLUT set 10 is lower than the sharpening degree of the image processed by the 3DLUT set 8 and the 3DLUT set 9, so that the image processed by the 3DLUT set 10 is softer than the image processed by the 3DLUT set 8 and the 3DLUT set 9, and is suitable for viewing in a driving scene.

[0132] Optionally, the second information comprises information related to the first image, when the information related to the first image indicates that the first image is a first style image, the brightness of the image processed by the first LUT set is lower than the brightness of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets; or, when the information related to the first image indicates that the first image is a second style image, the contrast of the image processed by the first LUT set is higher than the contrast of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets, and / or, the saturation of the image processed by the first LUT set is higher than the saturation of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0133] Optionally, when the information related to the first image indicates that the first image is a first style image, the brightness of the image processed by the first LUT set is lower than the brightness of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets, the saturation of a certain color phase in the image processed by the first LUT set is lower than the saturation of the color phase in the image processed by the LUT set other than the first LUT set in the plurality of LUT sets, and the contrast of the image processed by the first LUT set is higher than the contrast of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets, so that the image processed by the first LUT set is darker than the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0134] For example, the first style image comprises a science fiction film, and the second style image comprises a comedy film.

[0135] For example, Table 4 shows the correspondence between the video content and the 3DLUT set provided by the embodiment of the present application.

[0136] Table 4

[0137] Video content 3DLUT Person / landscape shooting 3DLUT group 11 Ancient style film 3DLUT group 12 Science fiction film 3DLUT group 13 Comedy film 3DLUT group 14 … …

[0138] For example, the image processed by the 3DLUT set 11 presents a fresh and natural style, and is more suitable for films of characters or natural scenes. The image processed by the 3DLIT set 12 has a bright and bright style, and is more suitable for films of ancient style. The image processed by the 3DLUT set 13 has lower brightness, lower saturation of a certain color phase, and higher contrast, so that the image presents a darker and more textured film style, and is suitable for science fiction films. The 3DLUT set 14 presents a color highlighting effect with higher contrast and saturation, and is suitable for comedy films.

[0139] Optionally, the method 300 comprises: determining the video content according to one or more of an application (app) type corresponding to the video source to be played, a video duration, and a label.

[0140] Optionally, determining the first LUT group from the plurality of LUT groups according to the second information comprises: determining the second LUT group as the first LUT group when the second LUT group is determined from the plurality of LUT groups according to the data collected by the sensor and the third LUT group is determined from the plurality of LUT groups according to the vehicle control information; or determining the fourth LUT group as the first LUT group when the second LUT group is determined from the plurality of LUT groups according to the data collected by the sensor, the third LUT group is determined from the plurality of LUT groups according to the vehicle control information, and the fourth LUT group is determined from the plurality of LUT groups according to the information related to the first image.

[0141] For example, after different LUT groups are obtained through different dimensions, a target LUT group can be selected based on priority information of each dimension. For example, after different LUT groups are obtained through the data collected by the sensor and the vehicle control information respectively, the LUT group determined through the data collected by the sensor can be determined as the target LUT group. For another example, after different LUT groups are obtained through the data collected by the sensor, the vehicle control information, and the information related to the first image respectively, the LUT group determined through the information related to the first image can be determined as the target LUT group.

[0142] For example, the priority of the LUT group determined through the information related to the first image is higher than the priority of the LUT group determined through the data collected by the sensor, and the priority of the LUT group determined through the data collected by the sensor is higher than the priority of the LUT group determined through the vehicle control information.

[0143] The above dimensions are only examples to illustrate what effect the selection of the 3D LUT group under a single dimension can produce. In actual embodiments, the 3D LUT group to be used can be determined according to one or more of the dimensions.

[0144] Optionally, determining the first LUT group from the plurality of LUT groups according to the second information comprises: determining the first LUT group from the plurality of LUT groups according to the second information and a first mapping relationship, the first mapping relationship comprising a corresponding relationship between the vehicle control information, the light intensity, the video content, and the 3D LUT.

[0145] For example, Table 5 shows a corresponding relationship between a vehicle driving state, a light intensity, a video content, and a 3D LUT group provided by an embodiment of the present application.

[0146] Table 5

[0147]

[0148] For example, the brightness and contrast of the image processed by the 3DLUT group 15 are higher than those of the image processed by the 3DLUT group 16.

[0149] For example, the image processed by the 3DLUT group 23 is softer than the image processed by the 3DLUT group 19, and is suitable for viewing in a driving scene.

[0150] Optionally, the first mapping relationship includes a corresponding relationship between the vehicle control information, the environment and weather condition, the light intensity, the video content and the 3DLUT.

[0151] Optionally, the method 300 further includes: controlling the display device to display a plurality of style options, each of the plurality of style options corresponding to a LUT group; and in response to obtaining an input of a user selecting a first style option from the plurality of style options, obtaining a first LUT group, the first style option corresponding to the first LUT group.

[0152] For example, Figure 4 A human machine interface (HMI) provided by an embodiment of the present application is shown.

[0153] As Figure 4 shown, in the process of playing a video picture through the center control screen, the vehicle can display a plurality of style options on the center control screen, such as a warm color option 401, a neutral color option 402 and a cold color option 403. The user can switch different style options on the center control screen by clicking the style options or dragging the style options to the video source, and each style option can correspond to a 3DLUT group.

[0154] Optionally, after determining the first 3DLUT group through the second information, the center control screen can prompt the user of a style option matching the current scene, the style option being the style option corresponding to the first 3DLUT group.

[0155] S330, determining a first LUT according to the first weight vector and the N LUTs in the first LUT group.

[0156] Optionally, the N LUTs in the first LUT group are weighted and fused according to the first weight vector to obtain the first LUT.

[0157] For example, the first 3DLUT group includes N 3DLUTs, i.e., LG=(L1, L2, …, LN), and the first weight vector includes the weights (w1, w2, …, wN) of each 3DLUT. N N Through the weights (w1, w2, …, wN), the first LUT is determined according to the first weight vector and the N LUTs in the first LUT group.​N ) performing weighted fusion on the first 3DLUT group to obtain a final 3DLUT used for the current frame image. Exemplarily, the final 3DLUT used can be obtained by performing weighted fusion through the following formula (1): f :

[0158]

[0159] Optionally, the values stored in the 3DLUT are not all RGB possible values, but can be partial sampling points. Exemplarily, the size of the 3DLUT is 9, 13, 17, 33, etc. The input RGB value can be obtained by using a trilinear interpolation or a tetrahedron interpolation, etc. In this way, the storage space of the 3DLUT can be saved.

[0160] Optionally, the method 300 further includes: in response to obtaining the input of the user selecting the second style option from the plurality of style options, obtaining a fifth LUT group; determining a second LUT according to the first weight vector and the N LUTs in the fifth LUT group; processing the first image according to the second LUT to obtain a third image; and controlling the display device to switch from displaying the second image to displaying the third image.

[0161] Exemplarily, taking the case that the method 300 is executed by the vehicle as an example, M 3DLUT groups preset in the car machine correspond to M different style options respectively. After obtaining the second information, the style option suitable for the current environment can be determined, and the first 3DLUT group corresponding thereto is obtained.

[0162] When the video is played for the first time or the video is switched, a newly calculated style option can be output. During the video playing process, the 3DLUT group is updated at a lower frequency (for example, every 5 minutes, to avoid frequent switching). When the style option is switched due to scene change or video source change, a new style option is obtained. At this time, the vehicle can perform transition between two style options through the recorded switching time. Exemplarily, the 3DLUT groups used before, during and after the transition can be as shown in formula (2):

[0163]

[0164] wherein LG1 and LG2 are the 3DLUT groups corresponding to the style option 1 and the style option 2 respectively. t0 is the time when the style option 1 is switched to the style option 2, and T is the switching transition time length. Exemplarily, T can be set to 5 seconds, which can provide a better experience for the user.

[0165] When the user manually switches the style option in the interface, the set style option is updated to the configuration of the current environment state, forming a user-defined style configuration. When the user is in the same environment state or a similar environment state next time, the updated main style is applied to the user.

[0166] S340, processing the first image according to the first LUT to obtain a second image.

[0167] For example, by traversing the first image, each RGB value in the first image is input into the first LUT, and the corresponding RGB value can be output, so that the output video is enhanced to obtain the second image.

[0168] S350, controlling the display device to display the second image.

[0169] Optionally, according to the first information, the first weight vector is determined, including: when the first information indicates that the difference between the light intensity of the first region and the light intensity of the second region in the vehicle cabin is greater than or equal to a preset difference, determining the weight vector 1 according to the light intensity of the first region and determining the weight vector 2 according to the light intensity of the second region, the N weights in the weight vector 1 correspond to the N LUTs in the LUT group a one by one, and the X weights in the weight vector 2 correspond to the X LUTs in the LUT group b one by one. For example, the first region can be the front row region in the cabin, and the second region can be the rear row region in the cabin.

[0170] Optionally, according to the second information, the first LUT group is determined from the plurality of LUT groups, including: according to the light intensity of the first region, the LUT group a is determined from the plurality of LUT groups, and according to the light intensity of the second region, the LUT group b is determined from the plurality of LUT groups.

[0171] Optionally, according to the first weight vector and the N LUTs in the first LUT group, the first LUT is determined, including: according to the weight vector 1 and the LUT group a, the LUT1 is determined, and according to the weight vector 2 and the LUT group b, the LUT2 is determined.

[0172] Optionally, according to the first LUT, the first image is processed to obtain a second image, including: according to the LUT1, the first image is processed to obtain an image 1, and according to the LUT2, the first image is processed to obtain an image 2.

[0173] Optionally, the display device is controlled to display the second image, including: controlling the display device in the first region to display the image 1, and controlling the display device in the second region to display the image 2.

[0174] For example, during the driving of the vehicle, after the privacy curtain of the rear row is opened, even if the light intensity outside the cabin is high, it can cause the light intensity of the front row area and the rear row area in the cabin to differ greatly. For another example, even if the light intensity outside the cabin is high, but in the case that the rear curtain is opened, it can also cause the light intensity of the front row area and the rear row area in the cabin to differ greatly. At this time, the weight vector 1 can be determined based on the light intensity of the front row area and the weight vector 2 can be determined based on the light intensity of the rear row area. Alternatively, the weight vector 1 can be determined based on the light intensity of the front row area, the vehicle control information and the information related to the first image, and the weight vector 2 can be determined based on the light intensity of the rear row area, the vehicle control information and the information related to the first image.

[0175] Optionally, according to the first information, the first weight vector is determined, comprising: when the first information indicates that the difference between the light intensity of the first area and the light intensity of the second area in the vehicle cabin is less than a preset difference, determining the weight vector 3 according to the information related to the image displayed by the display device of the first area and determining the weight vector 4 according to the information related to the image displayed by the display device of the second area, the N weights in the weight vector 3 correspond to the N LUTs in the LUT group c one by one, and the X weights in the weight vector 4 correspond to the X LUTs in the LUT group d one by one.

[0176] Optionally, according to the second information, the first LUT group is determined from the plurality of LUT groups, comprising: determining the LUT group c from the plurality of LUT groups according to the information related to the image displayed by the display device of the first area and determining the LUT group d from the plurality of LUT groups according to the information related to the image displayed by the display device of the second area.

[0177] Optionally, according to the first weight vector and the N LUTs in the first LUT group, the first LUT is determined, comprising: determining the LUT 3 according to the weight vector 3 and the LUT group c and determining the LUT 4 according to the weight vector 4 and the LUT group d.

[0178] Optionally, according to the first LUT, the first image is processed to obtain a second image, comprising: processing the image displayed by the display device of the first area according to the LUT 3 to obtain the image 3 and processing the image displayed by the display device of the second area according to the LUT 4 to obtain the image 4.

[0179] Optionally, the display device is controlled to display the second image, comprising: controlling the display device in the first area to display the image 3 and controlling the display device in the second area to display the image 4.

[0180] For example, during the driving of the vehicle, the displayed video content of the front region and the rear region is different. At this time, the weight vector 3 can be determined based on information related to the displayed image of the front region (for example, the style of the video content on the center screen), and the weight vector 4 can be determined based on information related to the displayed image of the rear region (for example, the style of the video content on the rear curtain). Alternatively, the weight vector 3 can be determined based on the illumination intensity of the front region, the vehicle control information, and the information related to the displayed image of the front region, and the weight vector 4 can be determined based on the illumination intensity of the rear region, the vehicle control information, and the information related to the displayed image of the rear region.

[0181] Figure 5 A schematic block diagram of an image processing system 500 provided by an embodiment of the present application is shown. The image processing system 500 includes a cockpit multimedia system 510 and a display screen 520, wherein the cockpit multimedia system 510 includes a video codec (MediaCodec) module 511, a display engine 512, and a graphics synthesis and display module 513, and the display screen 520 can include one or more of a center screen, a co-driver screen, or a rear screen.

[0182] In the intelligent cockpit (or terminal device such as a mobile phone, a tablet, a smart TV, etc.), a video app transmits video source data to be played to the video codec module 511, and the video codec module 511 decodes the video data and transmits each frame of data to the display engine 512 in the form of an image. In the display engine 512, the image is processed according to one or more of the information of the perception system, the vehicle control system, the operating system, and the user parameters, to control the display effect of the image. The processed graphics data is sent to each display screen in the cockpit after being fused with other graphics (such as a barrage of text, a UI interface, etc.) through the operating system.

[0183] In the embodiment of the present application, the image is mainly enhanced in the display engine 512 to achieve a better viewing experience. In the technical solution provided by the embodiment of the present application, the display engine 512 receives user configuration parameters, video basic information, in-vehicle perception system data, and out-of-vehicle perception system data in addition to interacting with the video stream data of the upstream video codec module 511 and the downstream graphics synthesis and display module 513, and performs image enhancement according to these data.

[0184] Figure 6Another schematic block diagram of the image processing system 500 is shown. The display engine 512 includes a style control module 5121, an image detection module 5122, and an image enhancement module 5123. The pre-trained 3DLUT groups and the prediction model can refer to the description in the above embodiments. In addition to receiving data from the perception system, the style control module 5121 can also receive data from the vehicle control system and the operating system.

[0185] The image detection module 5122 can be used to perform the step S320 described above. The image enhancement module 5123 can be used to perform the steps S330 and S340 described above.

[0186] Optionally, the style control module 5121 can run at a low frequency. For example, the style control module 5121 can receive data from the perception system, the vehicle control system, and the operating system every 5 minutes. For another example, the style control module 5121 can update the 3DLUT groups after receiving an instruction from the user to switch the style option.

[0187] Optionally, the image detection module 5122 and the image enhancement module 5123 can run at a high frequency. For example, for each frame of the video, the image detection module 5122 can calculate the weight vector and the image enhancement module 5123 can calculate the target LUT based on the weight vector and the 3DLUT groups.

[0188] Optionally, the first image is an Lth frame of image in a plurality of frames of images, L is a positive integer, and the method 300 further includes: obtaining an Sth frame of image in the plurality of frames of images, the Sth frame being a frame of image after the Lth frame, S being a positive integer; determining a first change coefficient according to a pixel mean value and / or a pixel variance of the Lth frame of image and the Sth frame of image; in a case where the first change coefficient is less than or equal to a preset change coefficient, processing the Sth frame of image according to the first LUT to obtain a fourth image; and controlling a display device to display the fourth image.

[0189] Optionally, Figure 7 Another schematic block diagram of the image processing system 500 is shown. The display engine 512 can include a content change detection module 5124.

[0190] Exemplarily, in the case that the method 300 is performed by a vehicle, due to the cabin system performance problem, it can be impossible to calculate the corresponding weight vector for each frame of video image. A content change detection module 5124 can be added. By inputting the current frame image and the historical frame image into the content change detection module 5124, the content change detection module 5124 can output a change coefficient. The content change detection module 5124 can send the change coefficient to the image detection module 5122. The image detection module 5122 can determine whether the operation of outputting the weight vector needs to be performed according to the value of the change coefficient. This scheme can greatly reduce the running frequency of the image detection module 5122, and can also achieve better video enhancement effect.

[0191] Exemplarily, the content change detection module 5124 is added as a pre-processing module of the image detection module 5122, and the definition and calculation steps of the change coefficient are as follows:

[0192] S1: receive the current frame image at time t=N, and downsample the current frame image.

[0193] For example, the current frame image is downsampled to 256*256 size.

[0194] S2: split the image into W*H blocks.

[0195] For example, split into 16*16 blocks, each block being 16*16 pixels in size.

[0196] S3: calculate the average value and variance of the pixels in each block and

[0197] S4: compare the data of the image at time N-1 to calculate the change coefficient D N , as shown in formulas (3) and (4):

[0198]

[0199] wherein, T E is the mean error threshold, for example, T E can be 3.0. T S is the variance error threshold, for example, T S can be 2.0. The selection of the parameters can be determined according to the running frequency that can be supported by the actual product performance.

[0200] S5: save the current time for the calculation of the change coefficient of the next frame.

[0201] After the change coefficient of the incoming image is detected by the image detection module 5122, threshold judgment is performed. If the change coefficient is less than the threshold, the image detection module 5122 no longer needs to perform the operation of outputting the weight vector, and directly uses the cached historical weight vector. If the coefficient change is greater than or equal to the threshold, the image detection module 5122 can perform the operation of outputting the weight vector based on the current frame image and / or the current sensor data, and update the weight vector. Exemplarily, the threshold can be 0.1.

[0202] Optionally, the first image is a Pth image in the plurality of images, P being a positive integer, and the method 300 further includes: processing, according to the first LUT, images after the Pth image in the plurality of images within a preset time length since the display device displays the second image, to obtain a fifth image; and controlling the display device to display the fifth image.

[0203] Here, a minimum interval time can be added to the image detection module 5122 to avoid frequent execution of the operation of outputting the weight vector, which can cause running to be stuck. Exemplarily, the minimum interval time can be 1s. The selection of specific parameters can be determined according to the running frequency that can be supported by the actual product performance.

[0204] Optionally, the method 300 further includes: at the end of the preset time length, obtaining third information, the third information including a Qth image in the plurality of images and / or second sensor data, Q being a positive integer; determining a second weight vector according to the third information, N weights in the second weight vector corresponding to N LUTs in a sixth LUT group one by one; determining a third LUT according to the second weight vector and the N LUTs in the sixth LUT group; processing the Qth image according to the third LUT to obtain a sixth image; and controlling the display device to display the sixth image.

[0205] Optionally, the first image is a part of the plurality of images, and the method further includes: processing, according to the first LUT, other images in the plurality of images except the part of the images to obtain a seventh image; and controlling the display device to display the seventh image.

[0206] In some embodiments, due to the performance problem of the cockpit system, the step S320 cannot be frequently run. Then it can be simplified again, and a better video enhancement effect can be achieved.

[0207] Exemplarily, Figure 8 Another schematic block diagram of the image processing system 500 provided by the embodiments of the present application is shown. The display engine 512 can include a frame extraction module 5125.

[0208] The image detection module 5122 no longer runs in real time. Instead, when the video starts playing, it extracts a portion of the video frames (such as the first frame, cover frame, and middle frames) using the frame extraction module 5125. The frame extraction module 5125 sends the extracted frame images to the image detection module 5122. The image detection module 5122 can determine the weight vector of the video based on these frame images. When switching videos, it uses the data from the new video to re-extract frames from the new video, thereby updating the output weight vector.

[0209] Optionally, before processing the images other than some of the frames in the multi-frame images according to the first LUT, method 300 further includes: determining that the playback duration of the multi-frame images is less than or equal to a preset duration.

[0210] For example, Figure 9 Another schematic block diagram of the image processing system 500 provided in an embodiment of this application is shown. Compared to Figure 6 , Figure 9 The image processing system 500 shown does not include an image detection module 5122. After acquiring data from the perception system, vehicle control execution, and operating system inputs, the style control module 5121 can select a target LUT from multiple 3DLUTs. The image enhancement module 5123 can process the image based on this target LUT to obtain a processed image. This method still offers superior video viewing experience compared to current solutions that adjust screen brightness solely based on ambient light intensity.

[0211] The above method 300 can be executed by the vehicle 100; or by the computing platform 120; or by the processor, chip or circuit in the computing platform 120; or by the image processing system 500.

[0212] Figure 10 A schematic block diagram of an image processing apparatus 1000 provided in an embodiment of this application is shown. The apparatus 1000 includes: an acquisition unit 1010, configured to acquire first information, the first information including a first image and / or first sensor data; a determination unit 1020, configured to determine a first weight vector based on the first information, wherein N weights in the first weight vector correspond one-to-one with N LUTs in a first color lookup table (LUT) group, and N is an integer greater than 1; the determination unit 1020 is further configured to determine a first LUT based on the first weight vector and the N LUTs in the first LUT group; an image processing unit 1030, configured to process the first image according to the first LUT to obtain a second image; and a control unit 1040, configured to control a display device to display the second image.

[0213] Optionally, the acquisition unit 1010 is further configured to acquire second information, the second information comprising one or more of data collected by a sensor of the vehicle, vehicle control information, and information related to the first image; and the determination unit 1020 is configured to determine the first LUT group from the plurality of LUT groups according to the second information.

[0214] Optionally, the second information comprises data collected by an illumination sensor, and when the data collected by the illumination sensor indicates that the illumination intensity inside the vehicle cabin is greater than or equal to a preset illumination intensity, the brightness of the image processed by the first LUT group is greater than the brightness of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups; and / or when the data collected by the illumination sensor indicates that the illumination intensity inside the vehicle cabin is greater than or equal to the preset illumination intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

[0215] Optionally, the second information comprises environmental information collected by a sensor outside the cabin, and when the environmental information indicates that it is sunny outside the cabin, the hue of the image processed by the first LUT group is warmer than the hue of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups; and / or when the environmental information indicates that it is sunny outside the cabin, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

[0216] Optionally, the second information comprises environmental information collected by a sensor outside the cabin, and when the environmental information indicates that it is raining or snowing outside the cabin, the hue of the image processed by the first LUT group is cooler than the hue of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups; and / or when the environmental information indicates that it is raining or snowing outside the cabin, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

[0217] Optionally, the second information comprises vehicle control information, and when the vehicle control information indicates that the speed of the vehicle is greater than or equal to a preset speed, the sharpening degree of the image processed by the first LUT group is lower than the sharpening degree of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

[0218] Optionally, the second information comprises information related to the first image, and when the information related to the first image indicates that the first image is an image of the first style, the brightness of the image processed by the first LUT set is lower than the brightness of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets; or when the information related to the first image indicates that the first image is an image of the second style, the contrast of the image processed by the first LUT set is higher than the contrast of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets, and / or the saturation of the image processed by the first LUT set is higher than the saturation of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

[0219] Optionally, the determining unit is configured to: determine the second LUT set as the first LUT set when the second LUT set is determined from the plurality of LUT sets according to the data collected by the sensor and the third LUT set is determined from the plurality of LUT sets according to the vehicle control information; or determine the fourth LUT set as the first LUT set when the second LUT set is determined from the plurality of LUT sets according to the data collected by the sensor, the third LUT set is determined from the plurality of LUT sets according to the vehicle control information, and the fourth LUT set is determined from the plurality of LUT sets according to the information related to the first image.

[0220] Optionally, the control unit is further configured to control the display device to display a plurality of style options, each style option in the plurality of style options corresponding to a LUT set; and the obtaining unit is further configured to, in response to obtaining an input of a user selecting a first style option from the plurality of style options, obtain the first LUT set, the first style option corresponding to the first LUT set.

[0221] Optionally, the obtaining unit 1010 is further configured to, in response to obtaining an input of a user selecting a second style option from the plurality of style options, obtain a fifth LUT set; the determining unit 1020 is further configured to determine a second LUT according to the first weight vector and N LUTs in the fifth LUT set; the image processing unit 1030 is further configured to process the first image according to the second LUT to obtain a third image; and the control unit 1040 is further configured to control the display device to switch from displaying the second image to displaying the third image.

[0222] Optionally, the first image is an Lth image in the plurality of images, L is a positive integer, the obtaining unit 1010 is further configured to obtain an Sth image in the plurality of images, the Sth image is one image after the Lth image, and S is a positive integer; the determining unit 1020 is further configured to determine the first change coefficient according to the pixel mean value and / or the pixel variance of the Lth image and the Sth image; the image processing unit 1030 is further configured to, in a case where the first change coefficient is less than or equal to a preset change coefficient, process the Sth image according to the first LUT to obtain a fourth image; and the control unit 1040 is further configured to control the display device to display the fourth image.

[0223] Optionally, the first image is a Pth image in the plurality of images, P is a positive integer, and the image processing unit 1030 is further configured to, from a preset time length after the control unit controls the display device to display the second image, process images after the Pth image in the plurality of images according to the first LUT to obtain a fifth image; and the control unit 1040 is further configured to control the display device to display the fifth image.

[0224] Optionally, the obtaining unit 1010 is further configured to, at the end of the preset time length, obtain third information, the third information including a Qth image in the plurality of images and / or second sensing data, Q being a positive integer; the determining unit 1020 is further configured to determine a second weight vector according to the third information, N weights in the second weight vector corresponding to N LUTs in the sixth LUT group one by one; the determining unit 1020 is further configured to determine a third LUT according to the second weight vector and the N LUTs in the sixth LUT group; the image processing unit 1030 is further configured to process the Qth image according to the third LUT to obtain a sixth image; and the control unit 1040 is further configured to control the display device to display the sixth image.

[0225] Optionally, the first image is part of the images in the plurality of images, the image processing unit 1030 is further configured to process other images in the plurality of images except the part of the images according to the first LUT to obtain a seventh image; and the control unit 1040 is further configured to control the display device to display the seventh image.

[0226] Optionally, the determining unit 1020 is further configured to, before the image processing unit processes other images in the plurality of images except the part of the images, determine that a playing time length of the plurality of images is less than or equal to a preset time length.

[0227] Optionally, the determining unit 1020 is configured to: input the first information into a prediction model to obtain the first weight vector.

[0228] It should be understood that the division of units in the above apparatus is only a logical functional division, and all or part of the units can be integrated into a physical entity or physically separated when actually implemented. In addition, the units in the apparatus can be implemented in the form of processor calling software; for example, the apparatus includes a processor connected with a memory, and the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or to realize the functions of the units of the apparatus, wherein the processor is, for example, a general processor such as a CPU or a microprocessor, and the memory is an internal memory of the apparatus or an external memory of the apparatus. Alternatively, the units in the apparatus can be implemented in the form of hardware circuit, and the functions of part or all of the units can be realized by the design of the hardware circuit, which can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of part or all of the units are realized by the design of the logical relationship of elements in the circuit; for example, in another implementation, the hardware circuit is a PLD, and taking FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the units. All units of the above apparatus can be implemented in the form of processor calling software, or all units can be implemented in the form of hardware circuit, or part of the units are implemented in the form of processor calling software, and the remaining part is implemented in the form of hardware circuit.

[0229] In the embodiments of the present application, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as CPU, microprocessor, GPU, or DSP, etc. In another implementation, the processor can realize certain functions through the logical relationship of the hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured, such as ASIC or PLD implemented hardware circuit, such as FPGA. In the reconfigurable hardware circuit, the processor loads the configuration document to realize the hardware circuit configuration process, which can be understood as the process of the processor loading instructions to realize the functions of part or all of the units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a kind of ASIC, such as NPU, TPU, DPU, etc.

[0230] As can be seen, each unit in the above apparatus can be one or more processors (or processing circuits) configured to implement the above methods, such as CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0231] Furthermore, all or some of the units in the above apparatus can be integrated or can be independent. In one implementation, these units are integrated to be implemented in the form of SoC. The SoC can include at least one processor for implementing the functions of the above methods or the units of the apparatus, and the at least one processor can be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, and the like.

[0232] The embodiments of the present application further provide an image processing apparatus, the human-computer interaction apparatus includes a processing unit and a storage unit, wherein the storage unit is configured to store instructions, and the processing unit is configured to execute the instructions stored in the storage unit, so that the apparatus executes the method or the steps executed by the above embodiments.

[0233] Optionally, if the image processing apparatus is located in a vehicle, the processing unit can be Figure 1 the processor 151-15n shown in FIG. 1.

[0234] The embodiments of the present application further provide an image processing system, which can include a computing platform and a display screen, and the computing platform can include the image processing apparatus.

[0235] The embodiments of the present application further provide a vehicle, which can include the image processing apparatus or the image processing system.

[0236] The embodiments of the present application further provide a computer program product, which includes computer program code, when the computer program code is executed on a computer, so that the computer executes the method in the above embodiments.

[0237] The embodiments of the present application further provide a computer readable medium, which stores program code, when the computer program code is executed on a computer, so that the computer executes the method in the above embodiments.

[0238] The embodiments of the present application further provide a chip, which includes a circuit, and the circuit is configured to execute the method in the above embodiments.

[0239] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution completion, or executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0240] It should be understood that in the embodiments of the present application, the memory can include read-only memory and random access memory, and provide instructions and data to the processor.

[0241] It should also be understood that in various embodiments of the present application, the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0242] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0243] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0244] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0245] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0246] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0247] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0248] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized by, The method comprises: obtaining first information, the first information comprising a first image and / or first sensing data; determining a first weight vector according to the first information, N weights in the first weight vector corresponding to N LUTs in a first LUT group one by one, N being an integer greater than 1; determining a first LUT according to the first weight vector and the N LUTs in the first LUT group; processing the first image according to the first LUT to obtain a second image; controlling a display device to display the second image.

2. The method of claim 1, wherein, The method further comprises: obtaining second information, the second information comprising one or more of data collected by a sensor of a vehicle, vehicle control information, and information related to the first image; determining the first LUT group from a plurality of LUT groups according to the second information.

3. The method of claim 2, wherein, The second information comprises data collected by an illumination sensor, when the data collected by the illumination sensor indicates that the intensity of illumination inside the vehicle cabin is greater than or equal to a preset intensity of illumination, the brightness of an image processed by the first LUT group is greater than the brightness of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or, when the data collected by the illumination sensor indicates that the intensity of illumination inside the vehicle cabin is greater than or equal to a preset intensity of illumination, the contrast of an image processed by the first LUT group is greater than the contrast of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups.

4. The method of claim 2, wherein, The second information comprises environmental information collected by a sensor outside the cabin, when the environmental information indicates that it is sunny outside the cabin, the color tone of an image processed by the first LUT group is warmer than the color tone of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or, when the environmental information indicates that it is sunny outside the cabin, the saturation of an image processed by the first LUT group is higher than the saturation of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups.

5. The method of claim 2, wherein, The second information comprises environmental information collected by a sensor outside the cabin, when the environmental information indicates that it is raining or snowing outside the cabin, the color tone of an image processed by the first LUT group is cooler than the color tone of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or, when the environmental information indicates that it is raining or snowing outside the cabin, the saturation of an image processed by the first LUT group is lower than the saturation of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups.

6. The method of claim 2, wherein, The second information comprises the vehicle control information, when the vehicle control information indicates that the speed of the vehicle is greater than or equal to a preset speed, the degree of sharpening of an image processed by the first LUT group is lower than the degree of sharpening of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups.

7. The method of claim 2, wherein, The second information comprises information related to the first image, When the information related to the first image indicates that the first image is a first style image, the brightness of the image processed by the first LUT set is lower than the brightness of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets; or, When the information related to the first image indicates that the first image is a second style image, the contrast of the image processed by the first LUT set is higher than the contrast of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets, and / or, the saturation of the image processed by the first LUT set is higher than the saturation of the image processed by the LUT set other than the first LUT set in the plurality of LUT sets.

8. The method according to any one of claims 2 to 7, characterized in that, The determining the first LUT set from the plurality of LUT sets according to the second information comprises: When a second LUT set is determined from the plurality of LUT sets according to the data collected by the sensor and a third LUT set is determined from the plurality of LUT sets according to the vehicle control information, the second LUT set is determined as the first LUT set; or, When a second LUT set is determined from the plurality of LUT sets according to the data collected by the sensor, a third LUT set is determined from the plurality of LUT sets according to the vehicle control information, and a fourth LUT set is determined from the plurality of LUT sets according to the information related to the first image, the fourth LUT set is determined as the first LUT set.

9. The method according to any one of claims 1 to 8, characterized in that, The method further comprises: controlling the display device to display a plurality of style options, each of the plurality of style options corresponding to a LUT set; in response to obtaining an input of a user selecting a first style option from the plurality of style options, obtaining the first LUT set, the first style option corresponding to the first LUT set.

10. The method of claim 9, wherein, The method further comprises: in response to obtaining an input of a user selecting a second style option from the plurality of style options, obtaining a fifth LUT set; determining a second LUT according to the first weight vector and N LUTs in the fifth LUT set; processing the first image according to the second LUT to obtain a third image; controlling the display device to switch from displaying the second image to displaying the third image.

11. The method according to any one of claims 1 to 10, characterized in that, The first image is an Lth image in a plurality of images, L being a positive integer, and the method further comprises: obtaining an Sth image in the plurality of images, the Sth image being one image after the Lth image, S being a positive integer; determining a first change coefficient according to the pixel average value and / or the pixel variance of the Lth image and the Sth image; in a case where the first change coefficient is less than or equal to a preset change coefficient, processing the Sth image according to the first LUT to obtain a fourth image; controlling the display device to display the fourth image.

12. The method according to any one of claims 1 to 10, characterized in that, The first image is a Pth image in a plurality of images, P being a positive integer, and the method further comprises: The method further includes: controlling the display device to display the fifth image.

13. The method of claim 12, wherein, The method further includes: at the end of the preset time length, obtaining third information, the third information including a Qth image in the plurality of images and / or second sensing data, Q being a positive integer; determining a second weight vector according to the third information, N weights in the second weight vector corresponding to N LUTs in a sixth LUT group one by one; determining a third LUT according to the second weight vector and the N LUTs in the sixth LUT group; processing the Qth image according to the third LUT to obtain a sixth image; controlling the display device to display the sixth image.

14. The method according to any one of claims 1 to 10, characterized in that, The first image is a part of the plurality of images, and the method further includes: processing other images in the plurality of images except the part of the images according to the first LUT to obtain a seventh image; controlling the display device to display the seventh image.

15. The method of claim 14, wherein, Before the processing of the other images in the plurality of images except the part of the images according to the first LUT, the method further includes: determining that a playing time length of the plurality of images is less than or equal to a preset time length.

16. The method according to any one of claims 1 to 15, characterized in that, The determining of the first weight vector according to the first information includes: inputting the first information into a prediction model to obtain the first weight vector.

17. An image processing apparatus characterized by comprising: The method further includes: an obtaining unit configured to obtain first information, the first information including a first image and / or first sensing data; a determining unit configured to determine a first weight vector according to the first information, N weights in the first weight vector corresponding to N LUTs in a first color lookup table (LUT) group one by one, N being an integer greater than 1; the determining unit is further configured to determine a first LUT according to the first weight vector and the N LUTs in the first LUT group; an image processing unit configured to process the first image according to the first LUT to obtain a second image; a control unit configured to control a display device to display the second image.

18. The apparatus of claim 17, wherein the obtaining unit is further configured to obtain second information, the second information including one or more of data collected by a sensor of a vehicle, vehicle control information, and information related to the first image; the determining unit is configured to determine the first LUT group from a plurality of LUT groups according to the second information.

19. The apparatus of claim 18, wherein, The second information includes data collected by an illumination sensor, when the data collected by the illumination sensor indicates that an illumination intensity in a cabin of the vehicle is greater than or equal to a preset illumination intensity, a brightness of an image processed by the first LUT group is greater than a brightness of an image processed by LUT groups other than the first LUT group in the plurality of LUT groups; and / or, When the data collected by the light sensor indicates that the light intensity in the cabin is greater than or equal to a preset light intensity, the contrast of the image processed by the first LUT group is greater than the contrast of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

20. The apparatus of claim 18, wherein, The second information includes environmental information collected by a sensor outside the cabin, When the environmental information indicates that it is sunny outside the cabin, the hue of the image processed by the first LUT group is warmer than the hue of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups; and / or, When the environmental information indicates that it is sunny outside the cabin, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

21. The apparatus of claim 18, wherein, The second information includes environmental information collected by a sensor outside the cabin, When the environmental information indicates that it is raining or snowing outside the cabin, the hue of the image processed by the first LUT group is colder than the hue of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups; and / or, When the environmental information indicates that it is raining or snowing outside the cabin, the saturation of the image processed by the first LUT group is lower than the saturation of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

22. The apparatus of claim 18, wherein, The second information includes the vehicle control information, When the vehicle control information indicates that the speed of the vehicle is greater than or equal to a preset speed, the sharpening degree of the image processed by the first LUT group is lower than the sharpening degree of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

23. The apparatus of claim 18, wherein, The second information includes information related to the first image, When the information related to the first image indicates that the first image is a first style image, the brightness of the image processed by the first LUT group is lower than the brightness of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups; or, When the information related to the first image indicates that the first image is a second style image, the contrast of the image processed by the first LUT group is higher than the contrast of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups, and / or, the saturation of the image processed by the first LUT group is higher than the saturation of the image processed by the LUT groups other than the first LUT group in the plurality of LUT groups.

24. The apparatus of any one of claims 18-23, wherein, The determination unit is configured to: When a second LUT group is determined from the plurality of LUT groups according to the data collected by the sensor and a third LUT group is determined from the plurality of LUT groups according to vehicle control information, the second LUT group is determined as the first LUT group. Or, In a case that the second LUT group is determined from the plurality of LUT groups according to the data collected by the sensor, the third LUT group is determined from the plurality of LUT groups according to the vehicle control information, and the fourth LUT group is determined from the plurality of LUT groups according to the information related to the first image, the fourth LUT group is determined as the first LUT group.

25. The apparatus of any one of claims 17-24, wherein, the control unit is further configured to control the display device to display a plurality of style options, each of the plurality of style options corresponding to a LUT group; the obtaining unit is further configured to, in response to obtaining an input of a user selecting a first style option from the plurality of style options, obtain the first LUT group, the first style option corresponding to the first LUT group.

26. The apparatus of claim 25, wherein, the obtaining unit is further configured to, in response to obtaining an input of a user selecting a second style option from the plurality of style options, obtain a fifth LUT group; the determining unit is further configured to determine a second LUT according to the first weight vector and N LUTs in the fifth LUT group; the image processing unit is further configured to process the first image according to the second LUT to obtain a third image; the control unit is further configured to control the display device to switch from displaying the second image to displaying the third image.

27. The apparatus of any one of claims 17-26, wherein, the first image is an Lth image in a plurality of images, L being a positive integer, the obtaining unit is further configured to obtain an Sth image in the plurality of images, the Sth image being one image after the Lth image, S being a positive integer; the determining unit is further configured to determine a first variation coefficient according to a pixel mean value and / or a pixel variance of the Lth image and the Sth image; the image processing unit is further configured to, in a case that the first variation coefficient is less than or equal to a preset variation coefficient, process the Sth image according to the first LUT to obtain a fourth image; the control unit is further configured to control the display device to display the fourth image.

28. The apparatus of any one of claims 17-26, wherein, the first image is a Pth image in a plurality of images, P being a positive integer, the image processing unit is further configured to, within a preset time period since the control unit controls the display device to display the second image, process images after the Pth image in the plurality of images according to the first LUT to obtain a fifth image; the control unit is further configured to control the display device to display the fifth image.

29. The apparatus of claim 28, wherein, the obtaining unit is further configured to, at the end of the preset time period, obtain third information, the third information comprising an Qth image in the plurality of images and / or second sensor data, Q being a positive integer; the determining unit is further configured to determine a second weight vector according to the third information, N weights in the second weight vector corresponding to N LUTs in a sixth LUT group one by one. The determining unit is further configured to determine a third LUT from the second weight vector and the N LUTs in the sixth LUT group. The image processing unit is further configured to process the Qth frame image according to the third LUT to obtain a sixth image. The control unit is further configured to control the display device to display the sixth image.

30. The apparatus of any one of claims 17-26, wherein, The first image is a part of the multiple frames of images, The image processing unit is further configured to process other images of the multiple frames of images except the part of the multiple frames of images according to the first LUT to obtain a seventh image. The control unit is further configured to control the display device to display the seventh image.

31. The apparatus of claim 30, wherein The determining unit is further configured to determine that a playing time length of the multiple frames of images is less than or equal to a preset time length before the image processing unit processes other images of the multiple frames of images except the part of the multiple frames of images.

32. The apparatus of any one of claims 17-31, wherein, The determining unit is configured to: input the first information into a prediction model to obtain the first weight vector.

33. An image processing apparatus characterized by comprising: comprising: a memory configured to store a computer program; a processor configured to execute the computer program stored in the memory, so that the apparatus executes the method in any one of claims 1 to 16.

34. An image processing system, characterized by comprising a display screen and a computing platform, the computing platform comprising the apparatus in any one of claims 17 to 33.

35. A vehicle characterized by comprising the apparatus in any one of claims 17 to 33, or the system in claim 34.

36. A computer-readable storage medium, characterized in that, instructions stored thereon, which, when executed by a processor, cause the processor to implement the method in any one of claims 1 to 16.

37. A computer program product, characterised in that, The computer program product comprises computer program code which, when executed on a computer, causes the computer to implement the method in any one of claims 1 to 16.

38. A chip, characterized by The chip comprises a circuit configured to execute the method in any one of claims 1 to 16.

Images