Image processing method and apparatus, vehicle, storage medium, and program product

Abstract

The present application relates to the technical field of image processing, and discloses an image processing method, device, vehicle, storage medium and program product, wherein the present application determines a first frame type by identifying the physical source channel of an image to be processed, determines a second frame type by extracting the content feature parameter thereof, combines the content feature parameter with the frame combination relationship of the two frame types, accurately determines the target frame type of the image to be processed and the corresponding target safety level, and avoids inaccurate frame type calibration caused by factors such as low image quality and environmental interference. The entertainment frame is subjected to global unified processing, and the safety frame is subjected to separate processing on the safety layer and the entertainment layer thereof, so that the image to be processed is subjected to differential processing based on the target frame type and the target safety level, a target image is obtained, driving safety information is prevented from being weakened, the recognition degree of driving safety information in the target image is improved, the safety information is accurately identified by the intelligent driving system and the driver, and driving safety in the vehicle driving scene is ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to image processing methods, apparatus, vehicles, storage media, and program products. Background Technology

[0002] In in-vehicle display systems, image frames can be functionally divided into safety frames and entertainment frames. Safety frames contain various information used by Advanced Driver Assistance Systems (ADAS) for identification and to alert the driver, directly affecting driving safety; entertainment frames primarily serve information display and interactive functions, having a smaller impact on driving safety.

[0003] In existing in-vehicle image processing architectures, a uniform image processing strategy is typically used to process image frames globally, without distinguishing between safety frames and entertainment frames. This can easily weaken safety information, affecting the accuracy of ADAS systems and drivers' recognition of safety information, and increasing driving safety hazards. To address these issues, existing technologies also determine safety levels by extracting image features and processing images according to these safety levels. However, the accuracy of image feature extraction is easily affected by factors such as image quality and complex road conditions, which can lead to inaccurate safety level labeling and further weaken safety information. Summary of the Invention

[0004] This invention provides an image processing method, apparatus, vehicle, storage medium, and program product to solve the problem that the existing technology relies heavily on image features to determine image security classification, resulting in low accuracy of image security classification and easy weakening of security information.

[0005] In a first aspect, the present invention provides an image processing method, the method comprising: Acquire the image to be processed and identify the physical source channel of the image; Based on the physical source channel, the first frame type of the image to be processed is determined; wherein, the first frame type is either a safety frame or an entertainment frame, a safety frame is an image frame containing driving safety information, and an entertainment frame is an image frame other than a safety frame; Extract the content feature parameters of the image to be processed, and determine the second frame type of the image to be processed based on the content feature parameters; wherein the second frame type is either a security frame or an entertainment frame. Based on content feature parameters and the frame combination relationship between the first frame type and the second frame type, the target frame type of the image to be processed is determined, and the target security level of the image to be processed is determined according to the target frame type and the frame combination relationship. If the target frame type is detected as an entertainment frame, global processing is performed on the image to be processed to obtain the target image; If the target frame type is detected as a safety frame, the safety layer and entertainment layer in the image to be processed are processed separately based on the target safety level to obtain the target image; wherein, the safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer.

[0006] This invention first identifies the physical source channel of the image to be processed to determine the first frame type, then extracts its content feature parameters to determine the second frame type. Combining the content feature parameters with the frame combination relationship between the two frame types, it accurately determines the target frame type and corresponding target security level of the image to be processed, avoiding inaccurate frame type calibration caused by low image quality or environmental interference. Entertainment frames are processed globally uniformly, while safety frames have their safety and entertainment layers processed separately. Based on the target frame type and target security level, the image to be processed is differentiated to obtain the target image. This prevents the weakening of driving safety information, improves the recognizability of driving safety information in the target image, and facilitates accurate identification of driving safety information by the intelligent driving system and the driver, ensuring driving safety in in-vehicle driving scenarios.

[0007] In some optional implementations, the security layer and entertainment layer in the image to be processed are processed separately based on the target security level to obtain the target image: The image to be processed is split into a security layer and an entertainment layer; Based on the target security level, the security layer and the entertainment layer are processed separately, and then the processed target security layer and target entertainment layer are merged to obtain the target image.

[0008] This invention splits the image into a safety layer composed of driving safety-related pixels and an entertainment layer composed of other pixels, targeting safety frames. It then performs differentiated processing and layered fusion based on the target safety level, simplifying the entertainment frame processing flow and improving system efficiency. Furthermore, it specifically enhances the safety layer, avoiding the weakening of driving safety information due to globally uniform processing, significantly improving the clarity and recognizability of driving safety information and ensuring driving safety.

[0009] In some alternative implementations, the security layer and entertainment layer are processed separately based on the target security level, including: Determine the optimization parameters corresponding to the target security level; wherein the optimization parameters include at least one of the following: color gamut mapping parameters, sharpening intensity, and noise reduction intensity; The security layer is processed based on the first parameter value of the optimization parameters to obtain the target security layer, and the entertainment layer is processed based on the second parameter value of the optimization parameters to obtain the target entertainment layer; wherein the first parameter value and the second parameter value are different.

[0010] This invention first matches the corresponding color gamut mapping, sharpening, and noise reduction optimization parameters according to the target security level. Then, it differentiates the security layer and entertainment layer in the security frame image by using different optimization parameters, thereby enhancing the visual recognition of the security layer while ensuring the visual effect of the entertainment layer. This effectively avoids weakening the security information and also takes into account the overall display quality of the image.

[0011] In some optional implementations, the processed target security layer and target entertainment layer are fused to obtain the target image, including: Determine the blending coefficient corresponding to the target security level; whereby the blending coefficient is used to adjust the transparency of the target entertainment layer; Place the security layer at the top, and then merge the target security layer and the target entertainment layer according to the fusion coefficient to obtain the target image.

[0012] This invention adjusts the transparency of the target entertainment layer by setting a corresponding fusion coefficient based on the target safety level, and merges the target safety layer on top, ensuring that driving safety information is always on the top layer and is not obscured or faded, thus avoiding the weakening of driving safety information due to layer fusion and overlay, and further improving the recognizability and display reliability of driving safety information.

[0013] In some optional implementations, the target frame type of the image to be processed is determined based on content feature parameters and the frame combination relationship between the first frame type and the second frame type, including: If both the first and second frame types are detected as safe frames, the target frame type of the image to be processed is determined to be a safe frame. If a safe frame is detected in either the first frame type or the second frame type, a multi-module redundancy judgment is performed based on the content feature parameters to obtain the target frame type of the image to be processed. If both the first and second frame types are detected as entertainment frames, the target frame type of the image to be processed is determined to be an entertainment frame.

[0014] This invention determines the target frame type by classifying the frame combination relationship between the first frame type and the second frame type. When both are safe frames or entertainment frames, the target frame type is directly determined. When the two determinations conflict, multi-module redundancy judgment is performed based on content feature parameters, thereby improving the accuracy and reliability of frame type determination.

[0015] In some optional implementations, the target security level of the image to be processed is determined based on the target frame type and frame combination relationship, including: If the target frame type is detected as an entertainment frame, the target security level is determined to be the preset minimum level; If the target frame type is detected as a security frame, and the frame combination relationship is that one of the first frame type and the second frame type is a security frame, the target security level is determined to be the first level; wherein the first level is greater than the preset minimum level; If the target frame type is detected as a security frame, and the frame combination relationship is that both the first frame type and the second frame type are security frames, the target security level is determined to be the preset highest level; wherein the preset highest level is greater than the first level.

[0016] This invention classifies security levels by combining target frame type and frame combination relationship. Entertainment frames are set as the lowest preset level, frames that are determined to be safe by a single decision are set as the first medium level, and frames that are determined to be safe by double consistency are set as the highest preset level. This achieves a reasonable distinction of security levels and provides a hierarchical basis for subsequent image processing resource allocation, priority scheduling, and differentiated enhancement.

[0017] In some alternative implementations, after determining the type of the first frame of the image to be processed based on the physical source channel, the method further includes: Obtain the device load rate of the image processing equipment; If the device load rate is detected to be greater than the load rate threshold, determine the initial security level that matches the physical source channel; The first frame type is determined as the target frame type, and the initial security level is determined as the target security level. Then, the steps of processing the image to be processed are performed based on the target frame type and the target security level.

[0018] When the device load rate exceeds the load rate threshold, the present invention directly determines the initial security level based on the physical source channel, and uses the first frame type and the initial security level as the final target frame type and target security level, which greatly reduces the system computing power consumption and processing latency in high load scenarios, and ensures the real-time performance of vehicle image processing and the stability of system operation.

[0019] In some alternative implementations, the method further includes: Control the image display terminal to display the target image, and obtain the standard display parameters of the target image and the real-time display parameters of the image display terminal; The attenuation deviation is obtained based on the real-time display parameters and the standard display parameters, and the corrected display parameters are obtained based on the attenuation deviation. The target image is corrected based on the corrected display parameters.

[0020] This invention obtains the standard display parameters of the target image and the actual real-time display parameters output by the display terminal in real time, calculates the attenuation deviation, and dynamically generates corrective display parameters to correct the target image. It compensates for the brightness attenuation and color shift caused by aging and temperature changes in the display device in real time, avoids unclear identification of safety information due to display deviation, and improves the accuracy of vehicle display and driving safety.

[0021] In a second aspect, the present invention provides an image processing apparatus, the apparatus comprising: The first processing module is used to acquire the image to be processed and identify the physical source channel of the image to be processed; The second processing module is used to determine the first frame type of the image to be processed based on the physical source channel; wherein the first frame type is either a safety frame or an entertainment frame, a safety frame is an image frame containing driving safety information, and an entertainment frame is an image frame other than a safety frame. The third processing module is used to extract the content feature parameters of the image to be processed, and determine the second frame type of the image to be processed based on the content feature parameters; wherein the second frame type is either a security frame or an entertainment frame. The fourth processing module is used to determine the target frame type of the image to be processed based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type, and to determine the target security level of the image to be processed based on the target frame type and the frame combination relationship. The fifth processing module is used to perform global processing on the image to be processed to obtain the target image if the target frame type is detected as an entertainment frame; if the target frame type is detected as a safety frame, it processes the safety layer and entertainment layer in the image to be processed separately based on the target safety level to obtain the target image; wherein, the safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer.

[0022] Thirdly, the present invention provides a vehicle comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the image processing method described in the first aspect or any corresponding embodiment thereof.

[0023] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the image processing method described in the first aspect or any corresponding embodiment thereof.

[0024] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the image processing method described in the first aspect or any corresponding embodiment thereof.

[0025] The beneficial effects of this invention are as follows: This invention first identifies the physical source channel of the image to be processed to determine the first frame type, then extracts its content feature parameters to determine the second frame type. Combining the content feature parameters with the frame combination relationship between the two frame types, it accurately determines the target frame type and corresponding target security level of the image to be processed, avoiding inaccurate frame type calibration caused by low image quality or environmental interference. Entertainment frames are processed globally uniformly, while safety frames have their safety and entertainment layers processed separately. Based on the target frame type and target security level, the image to be processed is differentiated to obtain the target image. This prevents the weakening of driving safety information, improves the recognizability of driving safety information in the target image, and facilitates accurate identification of driving safety information by the intelligent driving system and the driver, ensuring driving safety in in-vehicle driving scenarios. Attached Figure Description

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the first type of image processing method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a second process of an image processing method according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the multi-module redundancy determination according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the third process of the image processing method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the image quality optimization processing architecture according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the screen aging compensation process according to an embodiment of the present invention; Figure 7 This is a schematic diagram of an image processing architecture according to an embodiment of the present invention; Figure 8 This is a structural block diagram of an image processing apparatus according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the hardware structure of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0030] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0031] In the in-vehicle image processing architecture of related technologies, a uniform image processing strategy is typically used to process image frames globally without distinguishing between safety frames and entertainment frames. This can easily lead to weakened safety information, affecting the accuracy of ADAS systems and drivers' recognition of safety information, and increasing driving safety hazards. Based on the above problems, related technologies also determine safety levels by extracting image features and processing images according to these safety levels. However, the accuracy of image feature extraction is easily affected by factors such as image quality and complex road conditions, which can easily lead to inaccurate safety level labeling and further weakened safety information.

[0032] To address the aforementioned issues, this invention provides an image processing method. It determines the first frame type by identifying the physical source channel of the image to be processed, then extracts its content feature parameters to determine the second frame type. By combining the content feature parameters with the frame combination relationship between the two frame types, the method accurately determines the target frame type and corresponding target safety level of the image to be processed, avoiding inaccurate frame type calibration due to low image quality or environmental interference. Finally, based on the target frame type and target safety level, the image to be processed is differentiated to obtain the target image. This prevents the weakening of driving safety information, improves the recognizability of driving safety information in the target image, and facilitates accurate identification of driving safety information by the intelligent driving system and the driver, ensuring driving safety in in-vehicle driving scenarios. Furthermore, entertainment frames that do not contain driving safety information are processed using conventional image processing strategies, avoiding wasted image processing resources and balancing image processing efficiency.

[0033] According to an embodiment of the present invention, an image processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] This embodiment provides an image processing method that can be used in image processing devices, such as in-vehicle terminals and in-vehicle central control screens. Figure 1 This is a flowchart of an image processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the image to be processed and identify the physical source channel of the image to be processed.

[0035] It's important to understand that as the capabilities of automotive system-on-chips (SoCs) continue to improve, image signal processors (ISPs) have been gradually integrated into the SoC, thereby reducing chip costs and fully utilizing the SoC's hardware computing power. Simultaneously, the image quality optimization capabilities of the picture quality processor (PQ), which operates at the image display end, are also continuously being upgraded to meet users' demands for image quality and the scenario adaptation requirements of automotive display systems.

[0036] In this embodiment, the image processing device acquires images to be processed in real time from sensors such as the ISP module and various cameras through various built-in hardware interfaces. The ISP module converts the image signals acquired by the sensors into images to be processed. The image processing device can then use the PQ module to further optimize the image quality, querying a joint parameter mapping table to determine paired ISP parameter groups and PQ parameter groups that match the scene labels. This allows the ISP and PQ parameter groups to work together for image processing, resolving the algorithm silos and conflicts inherent in traditional ISP and PQ modules.

[0037] In this embodiment, the image processing device identifies the physical source channel corresponding to each frame of the image to be processed by reading the hardware interface identifier ID. For example, the hardware interface identifier ID is MIPI-CSI Ch0 corresponding to the front-view camera, MIPI-CSI Ch1 corresponding to the surround-view camera, MIPI-CSI Ch2 corresponding to the cabin camera, USB / Ethernet corresponding to the dashcam, etc. Thus, the physical source channel of the image to be processed is directly determined by the correspondence between the hardware interface identifier ID and the physical source channel stored in the hardware interface register.

[0038] Step S102: Based on the physical source channel, determine the first frame type of the image to be processed; wherein, the first frame type is either a safety frame or an entertainment frame, a safety frame is an image frame containing driving safety information, and an entertainment frame is an image frame other than a safety frame.

[0039] Specifically, security attributes for different physical source channels are pre-set, and these channels are directly bound to frame types to determine the first frame type (safety frame or entertainment frame) of the image to be processed. A safety frame is defined as an image frame containing driving safety information, while all other image frames are defined as entertainment frames. Driving safety information can come from the image itself (such as lane lines, vehicles ahead, tunnel entrances, etc.) used for ADAS recognition to guide ADAS actions, or from marker information (such as red warning signs) output by the ADAS system after monitoring the vehicle and its surroundings. The driving safety information in safety frames is directly related to driving safety. Entertainment frames, on the other hand, often contain display information (such as navigation, videos, user interfaces, etc.) or entertainment interaction information and do not involve driving safety.

[0040] For example, the physical source channel of the image to be processed is a front-view camera, a surround-view camera, or an in-cabin camera, and the first frame type is a safety frame; the physical source channel is a dashcam, and the first frame type is an entertainment frame. The specific type can be adjusted according to the actual scenario requirements, and this application is not limited thereto. Thus, the first frame type can be quickly determined through the hardware-level physical source channel, independent of image content, less affected by factors such as image quality and ambient light intensity, and improving determination efficiency.

[0041] Step S103: Extract the content feature parameters of the image to be processed, and determine the second frame type of the image to be processed based on the content feature parameters; wherein the second frame type is a security frame or an entertainment frame.

[0042] Specifically, content analysis is performed on the acquired image to be processed to extract content feature parameters related to driving safety. These content feature parameters mainly include the type of objects present in the image, the object's motion trajectory, color features, and texture features. Then, based on the object type, object motion trajectory, color features, or texture features, it is determined whether the second frame of the image to be processed is a safety frame or an entertainment frame.

[0043] For example, if the types of objects in the image include obstacles (such as vehicles, pedestrians, etc.) and safety targets such as lane lines and traffic signs, the second frame is determined to be a safety frame; if the types of objects in the image include entertainment information such as maps, music interfaces, or movies, the second frame is determined to be an entertainment frame.

[0044] For example, based on the motion trajectory analysis of the object's motion characteristics in the image, if a collision between the vehicle and the object is detected, the second frame is determined to be a safety frame, and the collision time is determined; if the object is static or moving at a low speed (e.g., below 0.1 m / s), the second frame is determined to be an entertainment frame. The method for obtaining the collision time can be found in the description of related technologies, and will not be elaborated upon here.

[0045] For example, if based on color features, a high proportion of safe colors is detected (e.g., more than 20% of the area is red) and a high saturation (e.g., 0.7 < saturation < 0.9), then the second frame type is determined to be a safe frame; if the color features present a natural color distribution, then the second frame type is determined to be an entertainment frame.

[0046] For example, if the texture edge gradient is detected to be higher than the edge gradient threshold (e.g., 0.3) based on texture features, the second frame type is determined to be a safe frame; if the texture features are smooth, it is determined to be an entertainment frame.

[0047] In this embodiment, if any one of the object type, object motion trajectory, color features, and texture features is determined to be a safe frame, then the final second frame type is determined to be a safe frame. Thus, by using the actual content feature parameters of the image to be processed, a secondary frame type determination is performed, reflecting the true security attributes of the image.

[0048] Step S104: Based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type, determine the target frame type of the image to be processed, and determine the target security level of the image to be processed according to the target frame type and the frame combination relationship.

[0049] Specifically, the frame combination relationship between the first frame type and the second frame type includes: both the first frame type and the second frame type are security frames; both the first frame type and the second frame type are entertainment frames; the first frame type is a security frame and the second frame type is an entertainment frame; and the first frame type is an entertainment frame and the second frame type is a security frame. If the frame combination relationship indicates that the frame types determined by the hardware and the content recognition are consistent, the target frame type can be directly determined; if the frame types determined by the two determinations conflict, the target frame type of the image to be processed is determined by combining the content feature parameters, and the target security level of the image to be processed is classified.

[0050] In this embodiment, the content feature parameters may also include the object type recognition confidence level and the collision time between the vehicle and the obstacle. By analyzing parameters such as recognition confidence level and collision time, the urgency of the current driving scenario is analyzed to obtain the target safety level. This target safety level is used to determine the optimization strategy or parameters for the image to be processed, so as to optimize the image to be processed in a way that matches the target safety level. Finally, each frame of the image to be processed corresponds to a unique target frame type and target safety level.

[0051] Step S105: Based on the target frame type and target security level, process the image to be processed to obtain the target image.

[0052] Specifically, based on the target frame type and target safety level, differentiated image processing strategies are employed to process the images to be processed, resulting in the target images and ensuring that driving safety information is displayed with priority. If the target frame type is a safety frame, processing resources are allocated according to the target safety level; the higher the safety level, the higher the processing priority. Targeted image enhancement is also performed based on the target safety level to highlight the clarity and recognizability of driving safety information in the target image. If the target frame type is an entertainment frame, its processing priority is lower than that of safety frames, and conventional image processing strategies are used to ensure image processing efficiency.

[0053] In this embodiment, if the target frame type is detected as an entertainment frame, global processing is performed on the image to be processed to obtain the target image; if the target frame type is detected as a safety frame, the safety layer and entertainment layer in the image to be processed are processed separately based on the target safety level to obtain the target image; wherein, the safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer.

[0054] The image processing method provided in this embodiment first identifies the physical source channel of the image to be processed to determine the first frame type, then extracts its content feature parameters to determine the second frame type. Combining the content feature parameters with the frame combination relationship between the two frame types, the method accurately determines the target frame type and corresponding target security level of the image to be processed, avoiding inaccurate frame type calibration caused by low image quality, environmental interference, or other factors. For entertainment frames, a globally unified processing method is adopted; for safety frames, their safety and entertainment layers are processed separately. Therefore, based on the target frame type and target security level, the image to be processed is differentiated to obtain the target image. This prevents the weakening of driving safety information, improves the recognizability of driving safety information in the target image, facilitates accurate identification of driving safety information by the intelligent driving system and the driver, and ensures driving safety in in-vehicle driving scenarios.

[0055] This embodiment provides an image processing method that can be used in image processing devices, such as in-vehicle terminals and in-vehicle central control screens. Figure 2 This is a flowchart of an image processing method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Acquire the image to be processed and identify the physical source channel of the image. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0056] Step S202: Based on the physical source channel, determine the first frame type of the image to be processed; wherein, the first frame type is either a safety frame or an entertainment frame, a safety frame is an image frame containing driving safety information, and an entertainment frame is an image frame other than a safety frame.

[0057] In some alternative implementations, after determining the type of the first frame of the image to be processed based on the physical source channel, the following steps are performed: Step a1: Obtain the device load rate of the image processing device; if the device load rate is detected to be greater than the load rate threshold, determine the initial security level that matches the physical source channel.

[0058] Specifically, the GPU or CPU load of the image processing device is monitored in real time to obtain the device load rate. The device load rate is compared with a preset load rate threshold. If the device load rate is greater than the load rate threshold (which can be set or adjusted according to the scene), it indicates that the current resources are relatively tight. At this time, content recognition is no longer performed, and the initial security level corresponding to the physical source channel of the image to be processed is determined directly. If the device load rate is not greater than the load rate threshold, step S203 is executed.

[0059] For example, the initial security level of the forward-view camera can be the highest preset level, the initial security level of the surround-view camera and the cabin camera can be the first level (lower than the highest preset level), and the initial security level of the dashcam can be the lowest preset level. The specific level can be adjusted according to the actual needs of the scenario, and this application is not limited thereto.

[0060] Step a2: Determine the first frame type as the target frame type and the initial security level as the target security level, then execute step S205.

[0061] Specifically, the first frame type is directly determined as the target frame type of the image to be processed, and the initial security level is determined as the target security level of the image to be processed. Then, the process jumps directly to the subsequent step S205 and no longer performs image content feature extraction and frame type combination arbitration operations, thereby saving image processing resources and improving image processing efficiency.

[0062] When the device load rate exceeds the load rate threshold, the present invention directly determines the initial security level based on the physical source channel, and uses the first frame type and the initial security level as the final target frame type and target security level, which greatly reduces the system computing power consumption and processing latency in high load scenarios, and ensures the real-time performance of vehicle image processing and the stability of system operation.

[0063] Step S203: Extract content feature parameters of the image to be processed, and determine the second frame type of the image based on the content feature parameters; wherein the second frame type is either a safety frame or an entertainment frame. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0064] Step S204: Based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type, determine the target frame type of the image to be processed, and determine the target security level of the image to be processed according to the target frame type and the frame combination relationship.

[0065] Specifically, step S204 includes: Step S2041: If both the first frame type and the second frame type are detected as safe frames, determine that the target frame type of the image to be processed is a safe frame.

[0066] For example, such as Figure 3 As shown, if the first frame type determined by hardware marking is a secure frame, and the second frame type determined by content feature parameters is also a secure frame, then the final target frame type is marked as a secure frame.

[0067] Step S2042: If a security frame is detected in either the first frame type or the second frame type, multi-module redundancy judgment is performed based on the content feature parameters to obtain the target frame type of the image to be processed.

[0068] For example, see again Figure 3 When a conflict arises between the determination of the first frame type and the second frame type, for example, if the first frame type is a safety frame and the second frame type is an entertainment frame, conflict arbitration is required for the determination of the safety frame. This will involve using multi-module redundancy judgment, such as Triple Modular Redundancy (TMR). If the judgment result is 2:1, the safety frame determination is maintained; if the judgment result is 1:2, it is downgraded to an entertainment frame, thus obtaining the target frame type of the image to be processed.

[0069] It should be noted that TMR uses three identical functional modules working in parallel and employs a majority voting mechanism to mask the failure of a single module. Specifically, the process of determining the type of the second frame based on content feature parameters in step S203 can be encapsulated into three identical modules. These three identical modules simultaneously receive the same input, namely the content feature parameters, and independently execute the same function. The outputs of the three modules are sent to a voter to compare their consistency. The voter determines which two or three outputs are the same based on the principle of majority rule. If at least two modules output the same value, that output value is taken as the judgment result. If all three outputs are different (which is usually extremely unlikely and may indicate that multiple modules have failed or the voter has failed), an alarm can be triggered or other redundant judgment measures can be taken.

[0070] In some embodiments, if the first frame type is an entertainment frame and the second frame type is a security frame, multi-module redundancy judgment, such as TMR, is enabled, and the target frame type of the image to be processed is obtained based on the TMR judgment result.

[0071] Step S2043: If the first frame type and the second frame type are both detected as entertainment frames, determine that the target frame type of the image to be processed is an entertainment frame.

[0072] In this embodiment, if both the first frame type and the second frame type are entertainment frames, then the final target frame type is determined to be an entertainment frame.

[0073] In other embodiments, when the first frame type is detected to be an entertainment frame, the target frame type can be directly determined to be an entertainment frame, and even if the second frame type may be determined to be a safety frame, TMR judgment will not be performed, thereby improving image processing efficiency.

[0074] This invention determines the target frame type by classifying the frame combination relationship between the first frame type and the second frame type. When both are safe frames or entertainment frames, the target frame type is directly determined. When the two determinations conflict, multi-module redundancy judgment is performed based on content feature parameters, thereby improving the accuracy and reliability of frame type determination.

[0075] Step S2044: Determine the target security level of the image to be processed based on the target frame type and frame combination relationship.

[0076] In some optional implementations, step S2044 above includes: Step b1: If the target frame type is detected as an entertainment frame, determine the target security level as the preset minimum level.

[0077] Specifically, if the target frame type is determined to be an entertainment frame, the target security level of the image to be processed is set to the preset minimum level, which can be represented by "00".

[0078] Step b2: If the target frame type is detected as a security frame, and the frame combination relationship is that one of the first frame type and the second frame type is a security frame, the target security level is determined to be the first level; wherein the first level is greater than the preset minimum level.

[0079] Specifically, if the target frame type is determined to be a security frame, and the corresponding frame combination relationship is that only one of the first frame type and the second frame type is a security frame, then the target security level is determined to be the first level, where the first level is higher than the preset minimum level, and the first level can be represented by "01".

[0080] Step b3: If the target frame type is detected as a security frame, and the frame combination relationship is that both the first frame type and the second frame type are security frames, the target security level is determined to be the preset highest level; wherein the preset highest level is greater than the first level.

[0081] Specifically, if the target frame type is determined to be a security frame, and the corresponding frame combination relationship is that both the first frame type and the second frame type are security frames, then the target security level is determined to be the preset highest level, where the preset highest level is higher than the first level.

[0082] In some embodiments, the content feature parameters may further include the recognition confidence of the object type and the collision time between the vehicle and the obstacle. Combining the recognition confidence and the collision time, the safety level can be further subdivided. For example, if the target frame type is a safe frame, and the frame combination relationship is that both the first frame type and the second frame type are safe frames, and it is further detected that the recognition confidence is greater than a confidence threshold (e.g., 90%), the target safety level is determined to be the second level "10", which is higher than the preset highest level; if it is further detected that the vehicle collision time is less than a collision time threshold (e.g., 5 s), the target safety level is determined to be the third level "11", which is higher than the second level.

[0083] This invention classifies security levels by combining target frame type and frame combination relationship. Entertainment frames are set as the lowest preset level, frames that are determined to be safe by a single decision are set as the first medium level, and frames that are determined to be safe by double consistency are set as the highest preset level. This achieves a reasonable distinction of security levels and provides a hierarchical basis for subsequent image processing resource allocation, priority scheduling, and differentiated enhancement.

[0084] Step S205: Based on the target frame type and target security level, process the image to be processed to obtain the target image.

[0085] Specifically, step S205 includes: Step S2051: If the target frame type is detected as an entertainment frame, perform global processing on the image to be processed to obtain the target image.

[0086] Specifically, such as Figure 4 As shown, the image processing device can receive images to be processed sent by the ISP input interface and other sensors through an interrupt-triggered mechanism. After determining the target frame type of the image to be processed by integrating hardware tags and content feature parameters, a security tagger is used to mark the image to be processed. The security tagger is used to insert a 2-bit frame type tag, and the tagging delay is ≤0.1 μs. Then, the path arbiter selects the data link of the image to be processed based on the frame type tag of the image to be processed, and the switching delay of the path arbiter is ≤0.5 μs.

[0087] In this embodiment, if the path arbitrator identifies an entertainment frame, it performs global unified processing on the image to be processed through a programmable PQ core to obtain the target image. The programmable PQ core can use general-purpose computing units such as GPUs to process the image using general algorithms (e.g., general brightness enhancement, image quality optimization algorithms). The specific algorithm can be selected and adjusted according to actual needs.

[0088] Step S2052: If the target frame type is detected as a security frame, the image to be processed is split into a security layer and an entertainment layer. The security layer and the entertainment layer are processed separately based on the target security level, and the processed target security layer and target entertainment layer are merged to obtain the target image.

[0089] Specifically, the path arbitrator detects that the target frame type is a safe frame and processes the image to be processed through the fixed-function PQ pipeline. The fixed-function PQ pipeline, accelerated by hardware, excels in noise reduction, sharpening, and color gamut remapping, while also offering high security. The pixel processing frequency of the fixed-function PQ pipeline can be 48 pixels / cycle, with a main frequency of 500 Hz. It should be noted that the PQ parameter sets of the fixed-function PQ pipeline can be determined through a mapping table register. This register stores paired ISP parameter sets and PQ parameter sets corresponding to scene labels. The ISP and PQ parameter sets have a cooperative constraint relationship to avoid algorithmic conflicts between the ISP and PQ modules.

[0090] In some optional implementations, step S2052 above includes: Step c1: Split the image to be processed into a security layer and an entertainment layer.

[0091] Specifically, the safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer. This embodiment identifies pixels related to driving safety information in the image to be processed, marks the corresponding pixel areas as safety mask areas, and marks the remaining pixels as non-safety mask areas, generating a region mask map. Then, based on the region mask map, the pixels in the safety frame image are separated to extract the safety layer and the entertainment layer.

[0092] Step c2: Determine the optimization parameters corresponding to the target security level; wherein the optimization parameters include at least one of the following: color gamut mapping parameters, sharpening intensity, and noise reduction intensity.

[0093] Specifically, the fixed-function PQ pipeline can perform color gamut remapping, noise reduction, and sharpening on the image to be processed. The target safety level can be used to adjust the optimization intensity of the optimization parameters on the pixels. The higher the target safety level, the higher the optimization intensity of the optimization parameters on the pixels, and the higher the visual recognition of the pixels. In this embodiment, the matching first parameter value and second parameter value are determined according to the target safety level.

[0094] In some embodiments, such as Figure 5 As shown, the fixed-function PQ pipeline uses fixed hardware units to process fixed algorithm strategies to improve execution efficiency. It includes a temporal denoising unit, a spatial sharpening unit, and a color gamut remapping matrix. Specifically, the temporal denoising unit achieves image motion compensation through 3-frame temporal fusion, improving image display quality and reducing motion blur under high-speed vehicle movement; the spatial sharpening unit improves image clarity through 8-directional gradient processing; and the color gamut remapping matrix unit further distinguishes the temporal mapping strategies for the safety layer and the entertainment layer, enhancing the clarity of the safety layer by applying non-uniform gain to the safety and entertainment layers in the safety frame image.

[0095] Step c3: Process the security layer based on the first parameter value of the optimization parameters to obtain the target security layer, and process the entertainment layer based on the second parameter value of the optimization parameters to obtain the target entertainment layer; wherein the first parameter value and the second parameter value are different.

[0096] Specifically, the first parameter value of the optimization parameters is used to process the security layer, and the second parameter value is used to process the entertainment layer, thereby highlighting the pixel information of the security layer. For example, the first parameter value of the optimization parameters is: a first preset value for enhancing the Y component in the image's YUV (e.g., 15%), and a second preset value for enhancing the V component (e.g., 30%); edge sharpening intensity is 1.2x; and noise reduction intensity is 38 dB. Alternatively, the first parameter value is: a standard sRGB color gamut mapping strategy, a global sharpening intensity of 1.0x, and a noise reduction intensity of 20 dB. These values ​​can be adjusted according to the actual scenario, and this application is not limited to these values. Thus, different color gamut remapping strategies are used for pixels in different layers, and the core logic is as follows: def layer_split(yuv_frame, security_mask): # Target Security Layer security_layer = apply_mask(yuv_frame, security_mask); # Target Entertainment Layer entertainment_layer = apply_mask(yuv_frame, ~security_mask); return security_layer, entertainment_layer; In some embodiments, the data links for the security layer and the entertainment layer can also be selected. For example, the security layer can be processed through a hardware-accelerated fixed-function PQ pipeline, controlling the processing latency of the security layer to ≤15 ms; the entertainment layer can be processed through a programmable PQ core, for example, by using the GPU to process the entertainment layer, controlling the GPU rendering latency to ≤30 ms.

[0097] This invention first matches the corresponding color gamut mapping, sharpening, and noise reduction optimization parameters according to the target security level. Then, it differentiates the security layer and entertainment layer in the security frame image by using different optimization parameters, thereby enhancing the visual recognition of the security layer while ensuring the visual effect of the entertainment layer. This effectively avoids weakening the security information and also takes into account the overall display quality of the image.

[0098] Step c4: Determine the blending coefficient corresponding to the target security level; whereby the blending coefficient is used to adjust the transparency of the target entertainment layer.

[0099] Specifically, the fusion coefficient The goal is to ensure sufficient recognition of the target security layer. When the recognition between the security layer and the entertainment layer decreases, the blending coefficient can be increased. The value is used to reduce the transparency of the entertainment layer and highlight the safety layer, thereby improving the recognition rate of ADAS. The higher the target safety level, the higher the fusion coefficient, the lower the transparency of the target entertainment layer, and the more prominent the driving safety information.

[0100] Step c5: Position the security layer to the top, and then merge the target security layer and the target entertainment layer according to the fusion coefficient to obtain the target image.

[0101] Specifically, there is only one target security layer, while there can be multiple target entertainment layers. For example, on the basis of the entertainment layers extracted from the image, various entertainment interactive information output by systems such as ADAS systems, such as navigation thumbnails, music playback information, AR information, etc., can be superimposed.

[0102] In this embodiment, there is only one target security layer, which is always on top. The Z-axis sorting of other target entertainment layers can be based on importance, with higher importance layers appearing higher in the list. Further, see again... Figure 4 The target security layer and the target entertainment layer are alpha-blended using an overlay compositor (blending precision of 0.1%), and the calculation formula is as follows:

[0103] in, The fusion coefficient is... Indicates the target entertainment layer. Indicates the target security layer. This represents the target image.

[0104] This invention adjusts the transparency of the target entertainment layer by setting a corresponding fusion coefficient based on the target safety level, and merges the target safety layer on top, ensuring that driving safety information is always on the top layer and is not obscured or faded, thus avoiding the weakening of driving safety information due to layer fusion and overlay, and further improving the recognizability and display reliability of driving safety information.

[0105] The above embodiments employ globally unified processing for entertainment frames and, for safety frames, split the image into a safety layer composed of driving safety-related pixels and an entertainment layer composed of the remaining pixels. Differential processing and layered fusion are then performed based on the target safety level, simplifying the entertainment frame processing flow and improving system efficiency. Furthermore, targeted enhancement of the safety layer prevents the weakening of driving safety information due to globally unified processing, significantly improving the clarity and recognizability of driving safety information and ensuring driving safety.

[0106] In some alternative implementations, after obtaining the target image, the following steps are performed: Step d1: Control the image display terminal to display the target image, and obtain the standard display parameters of the target image and the real-time display parameters of the image display terminal.

[0107] Specifically, such as Figure 6 As shown, the real-time values ​​of the RGB parameters of the image display can be monitored by a color sensor to obtain the real-time display parameters of the image display and to obtain the theoretical values ​​of the RGB parameters of the target image.

[0108] Step d2: Based on the real-time display parameters and the standard display parameters, obtain the attenuation deviation, and then obtain the corrected display parameters based on the attenuation deviation.

[0109] Specifically, see again Figure 6 The decay formula for the decay calculator can be expressed as follows:

[0110] in, This represents the corrected display parameters corresponding to pixel (i,j). This represents the positional weight corresponding to pixel (i,j) (1.0 for the center region and 0.7 for the edge region). Indicates screen temperature. Indicates screen usage time. This indicates the temperature difference between the screen temperature and the ideal screen temperature. This is the temperature drift coefficient. Temperature-related aging rate, For calibration values, .

[0111] In this embodiment, the attenuation deviation between the theoretical values ​​and real-time values ​​of the RGB parameters is calculated, and the position weights of the attenuation formula are adjusted according to the attenuation deviation. Parameters such as aging rate are updated according to the updated attenuation formula. .

[0112] Step d3: Correct the target image based on the corrected display parameters.

[0113] Specifically, see again Figure 6 Finally Used in the PQ module, it compensates for the RGB values ​​of pixels at corresponding locations in the target image. The greater the attenuation deviation, the better. The larger the value, the greater the pixel compensation. Then, the real-time values ​​of the RGB parameters are continuously detected to achieve a closed-loop feedback process, thereby solving the problem of brightness and color deviation caused by the increase of screen temperature and usage time, and compensating for screen degradation caused by temperature and usage time.

[0114] This invention obtains the standard display parameters of the target image and the actual real-time display parameters output by the display terminal in real time, calculates the attenuation deviation, and dynamically generates corrective display parameters to correct the target image. It compensates for the brightness attenuation and color shift caused by aging and temperature changes in the display device in real time, avoids unclear identification of safety information due to display deviation, and improves the accuracy of vehicle display and driving safety.

[0115] According to embodiments of the present invention, an image processing architecture is provided. For example... Figure 7 As shown, the image processing architecture adopts a three-stage pipeline architecture, namely the preprocessing stage, the joint processing stage, and the postprocessing stage.

[0116] In the preprocessing stage, preprocessing such as RAW depigmentation, black level calibration, and lens shading correction are performed to ensure image usability and remove the influence of image acquisition equipment on the image.

[0117] In the joint processing phase, the algorithm parameters of the ISP and PQ modules are matched by scene labels to solve the algorithm silo problem caused by the separation of algorithm parameters between the ISP and PQ modules. By dynamically constructing region mask maps, safe and non-safe pixels in the image are distinguished and differentiated, highlighting key driving safety information and addressing the problem of weakened safety information. Hardware acceleration significantly reduces software computing power.

[0118] In the post-processing stage, focusing on the image display end, the safety layer (containing driving safety information) and entertainment layer in the image are identified. The safety layer is given higher priority, and layered rendering of the safety and entertainment layers is adopted (suitable for multi-layer scenarios, such as when ADAS prompts, navigation, and entertainment screens need to be rendered simultaneously). The safety layer containing driving safety information is placed on top to maximize the recognizability of safety information, while hardware adds pass-through channels to reduce latency. A compensation algorithm based on screen aging is also added to address brightness reduction and color shift during screen use.

[0119] This embodiment also provides an image processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] This embodiment provides an image processing device, such as... Figure 8 As shown, it includes: The first processing module 801 is used to acquire the image to be processed and identify the physical source channel of the image to be processed; The second processing module 802 is used to determine the first frame type of the image to be processed based on the physical source channel; wherein the first frame type is either a safety frame or an entertainment frame, a safety frame is an image frame containing driving safety information, and an entertainment frame is an image frame other than a safety frame. The third processing module 803 is used to extract the content feature parameters of the image to be processed, and determine the second frame type of the image to be processed based on the content feature parameters; wherein the second frame type is a security frame or an entertainment frame. The fourth processing module 804 is used to determine the target frame type of the image to be processed based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type, and to determine the target security level of the image to be processed based on the target frame type and the frame combination relationship. The fifth processing module 805 is used to perform global processing on the image to be processed to obtain the target image if the target frame type is detected as an entertainment frame; if the target frame type is detected as a safety frame, it processes the safety layer and entertainment layer in the image to be processed separately based on the target safety level to obtain the target image; wherein, the safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer.

[0121] In some alternative implementations, after determining the first frame type of the image to be processed based on the physical source channel, the second processing module 802 is further configured to: Obtain the device load rate of the image processing equipment; If the device load rate is detected to be greater than the load rate threshold, determine the initial security level that matches the physical source channel; The first frame type is determined as the target frame type, and the initial security level is determined as the target security level. Then, the steps of processing the image to be processed are performed based on the target frame type and the target security level.

[0122] In some alternative implementations, the fourth processing module 804 is further configured to: If both the first and second frame types are detected as safe frames, the target frame type of the image to be processed is determined to be a safe frame. If a safe frame is detected in either the first frame type or the second frame type, a multi-module redundancy judgment is performed based on the content feature parameters to obtain the target frame type of the image to be processed. If both the first and second frame types are detected as entertainment frames, the target frame type of the image to be processed is determined to be an entertainment frame.

[0123] In some alternative implementations, the fourth processing module 804 is further configured to: If the target frame type is detected as an entertainment frame, the target security level is determined to be the preset minimum level; If the target frame type is detected as a security frame, and the frame combination relationship is that one of the first frame type and the second frame type is a security frame, the target security level is determined to be the first level; wherein the first level is greater than the preset minimum level; If the target frame type is detected as a security frame, and the frame combination relationship is that both the first frame type and the second frame type are security frames, the target security level is determined to be the preset highest level; wherein the preset highest level is greater than the first level.

[0124] In some optional implementations, the fifth processing module 805 is further configured to: The image to be processed is split into a security layer and an entertainment layer; Based on the target security level, the security layer and the entertainment layer are processed separately, and then the processed target security layer and target entertainment layer are merged to obtain the target image.

[0125] In some optional implementations, the fifth processing module 805 is further configured to: Determine the optimization parameters corresponding to the target security level; wherein the optimization parameters include at least one of the following: color gamut mapping parameters, sharpening intensity, and noise reduction intensity; The security layer is processed based on the first parameter value of the optimization parameters to obtain the target security layer, and the entertainment layer is processed based on the second parameter value of the optimization parameters to obtain the target entertainment layer; wherein the first parameter value and the second parameter value are different.

[0126] In some optional implementations, the fifth processing module 805 is further configured to: Determine the blending coefficient corresponding to the target security level; whereby the blending coefficient is used to adjust the transparency of the target entertainment layer; Place the security layer at the top, and then merge the target security layer and the target entertainment layer according to the fusion coefficient to obtain the target image.

[0127] In some optional implementations, the fifth processing module 805 is further configured to: Control the image display terminal to display the target image, and obtain the standard display parameters of the target image and the real-time display parameters of the image display terminal; The attenuation deviation is obtained based on the real-time display parameters and the standard display parameters, and the corrected display parameters are obtained based on the attenuation deviation. The target image is corrected based on the corrected display parameters.

[0128] The image processing apparatus provided in this embodiment of the invention can execute the image processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0129] Figure 9 This is a structural schematic diagram of a vehicle provided in an embodiment of the present invention.

[0130] The following is a detailed reference. Figure 9 The diagram illustrates a structural schematic suitable for implementing a vehicle according to an embodiment of the present invention. The vehicle may include a processor (e.g., a central processing unit, graphics processor, etc.) 901, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 902 or a program loaded from memory 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for vehicle operation. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0131] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows the vehicle to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 Vehicles with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0132] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a memory 908, or installed from a ROM 902. When the computer program is executed by the processor 901, it performs the functions defined in the image processing method of the embodiments of the present invention.

[0133] Figure 9 The vehicle shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0134] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the image processing method shown in the above embodiments is implemented.

[0135] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0136] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: The system acquires an image to be processed and identifies the physical source channel of the image; wherein the physical source channel includes an image signal processor, a dashcam, and various cameras. Based on the physical source channel, the first frame type of the image to be processed is determined; wherein, the first frame type is a safety frame or an entertainment frame, the safety frame is an image frame containing driving safety information, and the entertainment frame is an image frame other than a safety frame; Extract the content feature parameters of the image to be processed, and determine the second frame type of the image to be processed based on the content feature parameters; wherein the second frame type is a safety frame or an entertainment frame, and the content feature parameters include the object type, object motion trajectory, color features and texture features present in the image to be processed; Based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type, the target frame type of the image to be processed is determined, and the target security level of the image to be processed is determined according to the target frame type and the frame combination relationship; wherein, the frame combination relationship includes the first frame type and the second frame type being both safe frames, one of the first frame type and the second frame type being a safe frame, and the first frame type and the second frame type being both entertainment frames; If the target frame type is detected to be an entertainment frame, global processing is performed on the image to be processed to obtain the target image; If the target frame type is detected as a safety frame, the safety layer and entertainment layer in the image to be processed are processed respectively based on the target safety level to obtain the target image; wherein, the safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer.

2. The image processing method according to claim 1, characterized in that, The process of processing the security layer and entertainment layer in the image to be processed based on the target security level to obtain the target image includes: The image to be processed is split into a security layer and an entertainment layer; Based on the target security level, the security layer and the entertainment layer are processed separately, and the processed target security layer and target entertainment layer are merged to obtain the target image.

3. The image processing method according to claim 2, characterized in that, The process of processing the security layer and the entertainment layer based on the target security level includes: Determine the optimization parameters corresponding to the target security level; wherein the optimization parameters include at least one of color gamut mapping parameters, sharpening intensity, and noise reduction intensity; The security layer is processed based on the first parameter value of the optimization parameters to obtain a target security layer, and the entertainment layer is processed based on the second parameter value of the optimization parameters to obtain a target entertainment layer; wherein the first parameter value and the second parameter value are different.

4. The image processing method according to claim 2, characterized in that, The target security layer and target entertainment layer, after fusion processing, yield a target image, including: Determine the blending coefficient corresponding to the target security level; wherein the blending coefficient is used to adjust the transparency of the target entertainment layer; The security layer is placed on top, and the target security layer and the target entertainment layer are merged according to the fusion coefficient to obtain the target image.

5. The image processing method according to claim 1, characterized in that, Determining the target frame type of the image to be processed based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type includes: If both the first frame type and the second frame type are detected to be safe frames, the target frame type of the image to be processed is determined to be a safe frame. If a safe frame is detected in either the first frame type or the second frame type, a three-module redundancy judgment is performed based on the content feature parameters to obtain the target frame type of the image to be processed. If both the first frame type and the second frame type are detected as entertainment frames, the target frame type of the image to be processed is determined to be an entertainment frame.

6. The image processing method according to claim 5, characterized in that, Determining the target security level of the image to be processed based on the target frame type and the frame combination relationship includes: If the target frame type is detected as an entertainment frame, the target security level is determined to be the preset minimum level; If the target frame type is detected to be a security frame, and the frame combination relationship is that one of the first frame type and the second frame type is a security frame, the target security level is determined to be the first level; wherein, the first level is greater than the preset minimum level; If the target frame type is detected to be a security frame, and the frame combination relationship is that both the first frame type and the second frame type are security frames, the target security level is determined to be the preset highest level; wherein the preset highest level is greater than the first level.

7. The image processing method according to any one of claims 1-6, characterized in that, After determining the first frame type of the image to be processed based on the physical source channel, the method further includes: Obtain the device load rate of the image processing equipment; If the device load rate is detected to be greater than the load rate threshold, an initial security level matching the physical source channel is determined; The first frame type is determined as the target frame type, and the initial security level is determined as the target security level. The step of processing the image to be processed based on the target frame type and the target security level is then performed.

8. The image processing method according to any one of claims 1-6, characterized in that, The method further includes: The system controls the image display terminal to display the target image and obtains the standard display parameters of the target image and the real-time display parameters of the image display terminal. Based on the real-time display parameters and the standard display parameters, the attenuation deviation is obtained, and the corrected display parameters are obtained based on the attenuation deviation. The target image is corrected based on the corrected display parameters.

9. An image processing apparatus, characterized in that, The device includes: The first processing module is used to acquire the image to be processed and identify the physical source channel of the image to be processed; wherein, the physical source channel includes an image signal processor, a dashcam, and various cameras; The second processing module is used to determine the first frame type of the image to be processed based on the physical source channel; wherein the first frame type is a safety frame or an entertainment frame, the safety frame is an image frame containing driving safety information, and the entertainment frame is an image frame other than a safety frame; The third processing module is used to extract the content feature parameters of the image to be processed, and determine the second frame type of the image to be processed based on the content feature parameters; wherein the second frame type is a safety frame or an entertainment frame, and the content feature parameters include the object type, object motion trajectory, color features and texture features present in the image to be processed; The fourth processing module is used to determine the target frame type of the image to be processed based on the content feature parameters and the frame combination relationship between the first frame type and the second frame type, and to determine the target security level of the image to be processed according to the target frame type and the frame combination relationship; wherein, the frame combination relationship includes the first frame type and the second frame type being both safe frames, one of the first frame type and the second frame type being a safe frame, and the first frame type and the second frame type being both entertainment frames; The fifth processing module is used to perform global processing on the image to be processed to obtain a target image if the target frame type is detected as an entertainment frame; and to process the safety layer and entertainment layer in the image to be processed respectively based on the target safety level to obtain a target image if the target frame type is detected as a safety frame. The safety layer is a layer composed of pixels related to driving safety information, and the entertainment layer is the layer remaining after the image is split from the safety layer.

10. A vehicle, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the image processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the image processing method according to any one of claims 1 to 8.

Images