Display method and device of vehicle-mounted image, electronic equipment and vehicle

Abstract

This application relates to a method, apparatus, electronic device, and vehicle for displaying in-vehicle images. The method includes: intercepting a target image to be sent to an in-vehicle display; processing the target image to determine a target hazard level, wherein the target hazard level indicates the degree of impact of the target image on driving safety after being projected onto the head-up display; if the target hazard level is greater than a preset level, determining the target image as an abnormal image, wherein an abnormal image refers to an image that displays abnormally and produces strong light interference; performing graded adaptation processing on the target image based on the level range of the target hazard level to generate a safe display image, and outputting the safe display image to the head-up display. This application can improve driving safety.

Description

Technical Field

[0001] This application relates to the field of smart cockpit technology, and in particular to a method, device, electronic device, and vehicle for displaying in-vehicle images. Background Technology

[0002] A head-up display (HUD) projects navigation, instrument panel, and other information into the driver's field of vision, improving driving safety. With the increasing prevalence of screen mirroring capabilities, various in-vehicle applications can display their images on the HUD.

[0003] However, existing head-up display (HUD) projection solutions are designed only for normal application operation and do not consider the security risks caused by program malfunctions. When an application experiences rendering errors, crashes, or data interruptions, it may project uncontrollable abnormal images onto the HUD, such as white screens, black screens, checkerboard patterns, or flickering images. These bright and glaring images can severely interfere with the driver's vision, especially at night or in tunnels, and can easily affect driving safety.

[0004] like Figure 1 As shown, when the map application fails to render the base map due to a program error or data interruption, the head-up display will show a large area of ​​white checkerboard background, which will seriously affect the driver's vision. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and vehicle for displaying in-vehicle images to solve the problem that bright and glaring abnormal images interfere with the driver's vision, thereby causing low driving safety.

[0006] In a first aspect, this application provides a method for displaying in-vehicle images, the method comprising: Intercept the target image to be sent to the vehicle's display; By processing the target image, the target hazard level of the target image is determined, wherein the target hazard level is used to indicate the degree of impact on driving safety after the target image is projected onto the head-up display; If the target hazard level is greater than the preset level, the target image is determined to be an abnormal image, wherein the abnormal image refers to an image that displays abnormalities and produces strong light interference; Based on the level range of the target hazard level, the target image is subjected to hierarchical adaptation processing to generate a safe display image, and the safe display image is output to the head-up display.

[0007] Optionally, determining the target hazard level of the target image by processing the target image includes: The target image is subjected to abnormal feature identification by a preset recognition model to determine the baseline hazard level of the target image, wherein the baseline hazard level is used to indicate the degree of abnormality of the content of the target image itself; The value of each adjustment factor is determined based on the current driving scenario, wherein the value of the adjustment factor is used to indicate the degree of influence of the driving scenario on the abnormal image; The target hazard level of the target image is obtained by superimposing the value of each adjustment factor on the baseline hazard level.

[0008] Optionally, the baseline hazard level of the target image is determined by identifying abnormal features using a preset recognition model, including: The deep features in the target image are extracted step by step by the encoder in the recognition model, wherein the deep features are used to indicate the hierarchical features of the target image from low-level edges and textures to high-level semantics; The deep features are classified by a classifier, and the confidence scores corresponding to each anomaly dimension are output. The anomaly dimension is used to indicate different types of scene rendering anomalies. The anomaly score is obtained by weighted summation of the confidence scores for each anomaly dimension. Based on the comparison between the abnormal score and the set score, the baseline hazard level of the target image is determined.

[0009] Optionally, determining the value of each adjustment factor based on the current driving scenario includes: Acquire information such as in-vehicle light intensity, vehicle operating status, current time, and the driver's line of sight. If the current environment is determined to be low light based on the in-vehicle light intensity, then the value of the light adjustment factor is determined to be 1; If the current operating condition is determined to be high-risk based on the vehicle's operating status, then the value of the operating condition adjustment factor is determined to be 1. If the current time indicates that it is nighttime, then the time period adjustment factor is set to 1. If it is determined that the driver is currently looking at the head-up display based on the driver's field of vision, then the gaze adjustment factor is set to 1.

[0010] Optionally, the graded adaptation processing of the target image based on the grade range of the target hazard level includes: If the target hazard level is in the first level range, then an abnormal region in the target image is identified and the brightness of the abnormal region is reduced, wherein the abnormal region is a local pixel region in the target image where there is abnormal display; If the target hazard level is in the second level range, a dark mask layer is generated by the overlay graphics generator and used to cover the target image on the head-up display.

[0011] Optionally, after determining the target hazard level of the target image, the method further includes: If the target hazard level is less than the preset level, then the target image is determined to be a normal image; The normal image is output to the head-up display.

[0012] Optionally, the training process of the recognition model includes: Acquire the normal image displayed on the head-up display; By inducing rendering anomalies in the head-up display, a fault is injected into the normal image, and an image with edge anomalies is generated by image processing of the normal image. The normal image, the image after fault injection, and the image with abnormal edges are used as training samples, and the hazard level of the sample marked on the training samples is determined. The initial identification model is trained based on the training samples and the hazard levels of the samples to obtain the trained identification model.

[0013] Secondly, this application provides a display device for in-vehicle images, the device comprising: The interception module is used to intercept the target image to be sent to the vehicle display; The processing module is used to determine the target hazard level of the target image by processing the target image, wherein the target hazard level is used to indicate the degree of impact of the target image on driving safety after it is projected onto the head-up display; The determination module is used to determine that the target image is an abnormal image if the target hazard level is greater than a preset level, wherein the abnormal image refers to an image that displays abnormalities and produces strong light interference; The output module is used to perform hierarchical adaptation processing on the target image based on the level range of the target hazard level, generate a safety display image, and output the safety display image to the head-up display.

[0014] Thirdly, this application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0015] Fourthly, this application provides a vehicle including a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to implement the method described in any of the above-mentioned embodiments.

[0016] The technical solutions provided in this application have the following advantages compared with the prior art: In this application, the target image is intercepted and analyzed before being sent to the in-vehicle display to determine its target hazard level. The target hazard level refers to the degree of impact on driving safety after the target image is projected onto the head-up display. If the target hazard level is greater than a preset level, it indicates that the target image is an abnormal image that affects driving safety. At this time, based on the level range of the target hazard level, the target image of the corresponding hazard level is differentiated to generate a safe display image that meets the driver's visual comfort needs and does not cause visual interference. This safe display image is then output to the head-up display. In this way, even if the target image has strong light interference, the head-up display can replace it with a safe image before the abnormal image is actually displayed, achieving proactive visual threat elimination and thus improving driving safety. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 This is a schematic diagram of an abnormal image displayed on a head-up monitor. Figure 2 A schematic diagram of a vehicle-mounted image display system provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for displaying in-vehicle images provided in an embodiment of this application; Figure 4 This is a schematic diagram of the overall process of vehicle-mounted image display provided in an embodiment of this application; Figure 5 A schematic diagram of a vehicle-mounted image display device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0023] The vehicle image display method provided in this application can be applied to processing scenarios where the vehicle processor operates independently, as well as to hardware environments where the vehicle and server process collaboratively. In collaborative processing scenarios, such as... Figure 2 As shown, server 203 connects to vehicle 201 via a network. Vehicle 201 captures the target image and sends it to server 203. Server 203 generates a secure display image of the target image and sends it back to vehicle 201. Vehicle 201 outputs the secure display image to a head-up display. Database 205 can be set up on the server or independently of the server to provide data storage services for server 203. The aforementioned network includes, but is not limited to, wide area network, metropolitan area network or local area network.

[0024] Furthermore, embodiments of this application can also be implemented as functional plug-ins, integrated into the vehicle processor, directly generating corresponding safety display images based on the target image and outputting them to the head-up display.

[0025] The following will describe in detail a method for displaying vehicle images provided in this application embodiment, taking an application to a processor as an example. Figure 3 As shown, the specific steps are as follows: Step 301: Intercept the target image to be sent to the vehicle display; Step 302: By processing the target image, determine the target hazard level of the target image, wherein the target hazard level is used to indicate the degree of impact on driving safety after the target image is projected onto the head-up display; Step 303: If the target hazard level is greater than the preset level, the target image is determined to be an abnormal image. An abnormal image refers to an image that displays abnormalities and produces strong light interference. Step 304: Based on the target hazard level range, perform hierarchical adaptation processing on the target image to generate a safe display image, and output the safe display image to the head-up display.

[0026] In step 301, the processor intercepts the target image to be sent to the in-vehicle display. The target image refers to the screen sent by various in-vehicle applications to the head-up display for projection display, such as the navigation map screen, the speedometer screen, or the in-vehicle multimedia playback interface, and can be any content to be displayed under the head-up display's all-scene projection function.

[0027] The processor's interception action is not performed after the head-up display shows the image, but rather after the image data has been rendered by the vehicle's infotainment system and sent to the head-up display's image pipeline, but before it has been sent to the head-up display's projection module.

[0028] In step 302, the processor performs a risk assessment on the captured target image to determine the target hazard level corresponding to the target image. The target hazard level is an indicator used by the processor to quantify the safety of the target image, measuring whether the target image, when projected onto the head-up display, will affect driving safety.

[0029] The higher the target hazard level, the more likely the target image is to produce glaring light due to problems such as excessive brightness, abnormal flickering, large areas of solid color, or sudden changes in the image, which may interfere with the driver's vision, distract the driver's attention, and thus increase the driving safety hazard; the lower the target hazard level, the safer the target image is, and it will not cause significant interference to driving.

[0030] In step 303, if the processor determines that the target hazard level exceeds the preset level, it means that the currently captured target image has obvious driving safety risks, such as large areas of bright white screen, high frequency flickering, and solid color collapse. The strong light on the screen may interfere with the driver's vision, distract the driver's attention, and thus increase the driving safety hazard. Therefore, the target image is an abnormal image.

[0031] If the processor determines that the target hazard level does not exceed the preset level, it means that the target image is rendered normally and will not cause visual interference to the driver. In this case, the target image is determined to be a normal image and is output to the head-up display for normal display.

[0032] This application determines whether a target image is abnormal by calculating its target hazard level and comparing it with a preset level. This enables a quantitative assessment of display risks. The target hazard level directly reflects the potential impact of an image on driving safety when displayed on a head-up display, making the criteria for determining abnormal images uniform, objective, and quantifiable.

[0033] In step 304, the target hazard level of the abnormal image is divided into multiple level intervals. The degree of image abnormality corresponding to each level interval is different, so the image processing method is also different. Therefore, the abnormal image can be optimized in a differentiated manner according to the level interval of the target hazard level to eliminate visual interference and strong light hazards, and generate a safe display image with soft brightness that meets driving safety standards and driver visual comfort requirements.

[0034] The processor sends the safety-processed image to the head-up display (HUD) via an image transmission channel. Upon receiving the image, the HUD projects it onto the driver's field of vision according to the normal projection process. By reducing the brightness of bright images, the HUD avoids the driving safety hazards posed by excessively bright projections, thus improving driving safety.

[0035] In this application, the target image is intercepted and analyzed before being sent to the in-vehicle display to determine its target hazard level. The target hazard level refers to the degree of impact on driving safety after the target image is projected onto the head-up display. If the target hazard level is greater than a preset level, it indicates that the target image is an abnormal image that affects driving safety. At this time, based on the level range of the target hazard level, the target image of the corresponding hazard level is differentiated to generate a safe display image that meets the driver's visual comfort needs and does not cause visual interference. This safe display image is then output to the head-up display. In this way, even if the target image has strong light interference, the head-up display can replace it with a safe image before the abnormal image is actually displayed, achieving proactive visual threat elimination and thus improving driving safety.

[0036] As an optional implementation, step 302, determining the target hazard level of the target image by processing the target image, includes: Step S11: The target image is subjected to abnormal feature recognition by a preset recognition model to determine the baseline hazard level of the target image. The baseline hazard level is used to indicate the degree of abnormality of the content of the target image itself. Step S12: Determine the value of each adjustment factor based on the current driving scenario, wherein the value of the adjustment factor is used to indicate the degree of influence of the driving scenario on the abnormal image; Step S13: Add the value of each adjustment factor to the baseline hazard level to obtain the target hazard level of the target image.

[0037] In step S11, the processor analyzes the target image synchronously through two parallel analysis paths, and finally fuses the output results of the two paths to generate the target hazard level of the target image, thereby achieving a comprehensive assessment of the safety risks of the target image.

[0038] In one analysis path, the processor invokes a pre-trained recognition model, inputting the captured target image as RGB image frames. The model performs pixel-level depth feature extraction, scanning frame by frame and accurately identifying various potential anomalies, including but not limited to solid color collapse, texture loss, checkerboard anomalies, high-frequency flickering, and image freezing. Based on these anomalies, the processor determines the baseline hazard level of the target image. The baseline hazard level is a basic level determined by the degree of anomaly in the target image's content. It characterizes whether the target image itself poses a visual safety risk and the degree of that risk; it assesses the inherent safety attributes of the target image without considering the influence of external driving scenarios.

[0039] The recognition model in this application is a lightweight convolutional neural network (CNN), with an encoder and a fully connected classifier as its core structure. For example, the recognition model can be a MobileNet (mobile convolutional neural network) model, a queueNet (compressed network) model, or a shuffleNet (shuffle network) model. This application does not impose specific restrictions on the type of recognition model.

[0040] In step S12, in another analysis path, the processor, in conjunction with the current driving scenario, uses scenario-based analysis to determine the sensitivity and risk amplification degree of the current driving scenario to abnormal images on the head-up display, thereby determining each adjustment factor and its value. The adjustment factor is used to correct the hazard level of the baseline hazard level; the value of the adjustment factor is 1 or 0. If the adjustment factor is 1, it indicates that the current scenario amplifies the safety risk of the abnormal image, and the baseline hazard level needs to be increased by 1 to improve the rigor of the risk level determination. If the adjustment factor is 0, it indicates that the current scenario will not have any additional impact on the safety risk of the abnormal image and will not interfere with the baseline hazard level.

[0041] In step S13, the processor superimposes the baseline hazard level with the values ​​of each adjustment factor to obtain the final target hazard level, thereby achieving a comprehensive risk assessment of the target image.

[0042] Target hazard level = baseline hazard level + sum of the values ​​of each adjustment factor.

[0043] By superimposing and fusing the output results of the two analysis paths, a deep integration of the target image's own anomaly judgment and the real-time driving scenario risk judgment is achieved, so that the assessment of the target hazard level is based on the target image itself and fits the actual driving scenario, thereby improving the accuracy of the target hazard level assessment.

[0044] This application utilizes a recognition model to extract pixel-level deep features, enabling precise identification of various subtle abnormal features and ensuring the accuracy of the baseline hazard level determination. This avoids safety hazards caused by inaccurate abnormal image recognition. Furthermore, by combining real-time vehicle environmental information and operating status to determine adjustment factors, dynamic adaptation of risk assessment is achieved. This allows for flexible adjustment of risk levels according to different driving scenarios, making the assessment results more aligned with actual driving needs and improving the scenario adaptability of risk assessment. Finally, by superimposing the baseline hazard level and adjustment factors, a comprehensive consideration of both the inherent risk of the image and the amplified risk of the scenario is achieved, significantly improving the accuracy of target hazard level assessment.

[0045] As an optional implementation, in step S11, the abnormal feature identification of the target image is performed using a preset recognition model to determine the baseline hazard level of the target image, including: Step S111: Extract deep features from the target image step by step by the encoder in the recognition model, wherein the deep features are used to indicate the hierarchical features of the target image from low-level edges and textures to high-level semantics; Step S112: Classify the deep features using a classifier and output the confidence scores corresponding to each anomaly dimension. The anomaly dimension is used to indicate different types of rendering anomalies. Step S113: Calculate the weighted sum of the confidence scores for each anomaly dimension to obtain the anomaly score; Step S114: Determine the baseline hazard level of the target image based on the comparison result between the abnormal score and the set score.

[0046] In step S111, the processor inputs the RGB image frames captured from the head-up display's graphics pipeline into the encoder of the recognition model. This encoder, composed of multiple stacked convolutional layers, activation layers, and pooling layers, possesses powerful feature learning capabilities and can automatically and progressively extract hierarchical features from the target image. The encoder first captures basic features such as low-level edges and textures of the target image, and then delves deeper to extract high-level semantic features, forming a complete feature representation from low to high level. This representation covers both the external visual details of the target image and its inherent semantic information. This hierarchical feature extraction method can accurately capture potential abnormal details in the target image, avoiding the omission of any image issues that may affect driving safety.

[0047] In step S112, the classifier of the recognition model classifies the deep features output by the encoder. This classifier is a fully connected classifier, which is specifically used to identify and classify deep features, and then outputs the confidence scores corresponding to each anomaly dimension.

[0048] The anomaly dimension explicitly corresponds to five common rendering anomalies: solid color collapse, texture loss, checkerboard anomaly, high-frequency flicker, and screen freeze. These correspond to the five-dimensional confidence vector output by the fully connected classifier: [n1 solid color collapse, n2 texture loss, n3 checkerboard anomaly, n4 high-frequency flicker, n5 screen freeze]. Each dimension's confidence score ranges from 0 to 30, with a total score of 100. This score directly reflects the severity of the corresponding anomaly; a higher score indicates a more pronounced anomaly in that dimension and a greater impact on driving safety.

[0049] In step S113, the processor performs a weighted summation of the confidence scores for each anomaly dimension to obtain the anomaly score of the target image. Here, all weighting coefficients are set to 1, that is, the anomaly score = n1 score + n2 score + n3 score + n4 score + n5 score. This anomaly score can reflect the overall degree of anomaly of the target image.

[0050] In step S114, the processor determines the baseline hazard level of the target image based on the comparison between the abnormal score and the set score. If the abnormal score is greater than the set score, for example, the set score > 60 points, it is determined that the current image has a visual hazard that needs to be addressed, and the baseline hazard level is set to level 1, indicating that the image has a significant safety hazard and requires further processing; if the abnormal score is less than or equal to the set score, for example, the set score ≤ 60 points, it is determined that the current image has no significant safety risk, and the baseline hazard level is set to level 0, indicating that the image can be displayed normally and no additional processing is required.

[0051] This application extracts hierarchical deep features step by step through an encoder, outputs multi-dimensional anomaly confidence scores through a classifier, and obtains anomaly scores through weighted summation. The scores are then compared with the set scores to determine the baseline hazard level. This enables accurate quantitative judgment of anomalies in head-up display images, effectively distinguishing normal images from abnormal images such as rendering failures and crashes, reducing the false judgment rate, and improving the reliability of baseline hazard level assessment.

[0052] As an optional implementation, in step S12, determining the value of each adjustment factor based on the current driving scenario includes: Step S121: Obtain the in-vehicle light intensity, vehicle operating status, current time, and driver's line of sight area; Step S122: If it is determined that the current environment is a low light environment based on the light intensity inside the vehicle, then the value of the light adjustment factor is set to 1; Step S123: If the current operating condition is determined to be high-risk based on the vehicle's operating status, then the value of the operating condition adjustment factor is set to 1. Step S124: If it is determined that the current time is nighttime based on the current time, then the value of the time period adjustment factor is determined to be 1; Step S125: If it is determined that the driver is currently looking at the head-up display based on the driver's field of vision area, then the value of the gaze adjustment factor is determined to be 1.

[0053] In step S121, the processor acquires in real time the in-vehicle light intensity, vehicle operating status, current time, and driver's line of sight area. The in-vehicle light intensity indicates the brightness of the in-vehicle environment; the vehicle operating status indicates whether the vehicle is in a high-risk scenario; the current time distinguishes between daytime and nighttime; and the driver's line of sight area indicates whether the driver is looking at the head-up display.

[0054] In step S122, the processor analyzes and judges the collected in-vehicle light intensity data, and analyzes the impact of the lighting environment on the driver's visual perception. If the in-vehicle light intensity is lower than a preset threshold, it is determined to be a low-light environment. At this time, the driver's visual sensitivity is greatly increased, and any abnormal image on the head-up display will cause strong stimulation to the driver's vision. Compared with the normal lighting environment, the interference risk of abnormal images is greatly amplified. Therefore, the value of the light adjustment factor is set to 1, indicating that a risk assessment level needs to be added on the basis of the baseline hazard level. If the in-vehicle light intensity does not meet the low-light environment conditions, that is, it is in a normal or strong light environment, the driver's visual sensitivity is relatively low, the interference impact of abnormal images is small, and the risk level is low. Therefore, the value of the light adjustment factor is set to 0, and the baseline hazard level is not adjusted.

[0055] In step S123, the processor determines whether the current operating condition is a high-risk condition based on the vehicle's operating status, and conducts a detailed analysis of the impact of the operating condition on the driver's attention and risk tolerance: If the vehicle is in a high-risk condition such as high-speed driving or emergency lane changes, the driver's attention is highly focused on the road conditions, and they need to pay attention to the road ahead, surrounding vehicles, and changes in road conditions at all times. Any abnormal images appearing on the head-up display will suddenly grab the driver's attention, causing the driver's gaze to temporarily deviate from the road. Minor abnormalities that would normally be acceptable will drastically amplify the driving risk under high-risk conditions. Therefore, the value of the operating condition adjustment factor is set to 1, indicating that a risk assessment level needs to be added on top of the baseline hazard level. If the vehicle is in a normal driving condition, such as low-speed driving or constant-speed driving, the road conditions are relatively smooth, and the driver does not need to be highly tense at all times. Some attention can be diverted to the image on the head-up display. The interference risk caused by abnormal images is low and will not have a significant impact on driving safety. Therefore, the value of the operating condition adjustment factor is set to 0, and the baseline hazard level is not adjusted.

[0056] In step S124, the processor determines whether it is nighttime based on the current time and analyzes the visual characteristics and risks of nighttime driving: If it is nighttime, the ambient light outside the vehicle is insufficient, the driver mainly relies on the vehicle lights to observe the road conditions, resulting in higher visual fatigue and increased sensitivity to changes in light. Abnormal images on the head-up display will strongly interfere with the driver's vision and may even cause glare, affecting the driver's judgment of the road surface. The safety risk of abnormal images is amplified. Therefore, the time period adjustment factor is set to 1, indicating that a risk assessment level needs to be added on top of the baseline hazard level. If it is daytime, the ambient light outside the vehicle is sufficient, the driver's visual conditions are better, and the tolerance for abnormal images is higher. The interference risk caused by abnormal images is relatively low. Therefore, the time period adjustment factor is set to 0, and the baseline hazard level is not adjusted.

[0057] In step S125, the processor determines whether the driver is currently looking at the head-up display based on the driver's gaze area, and analyzes the impact of the driver's gaze state on the interference effect of abnormal images: If the driver's gaze is continuously focused on the head-up display area, the driver is actively receiving information transmitted by the head-up display and is paying close attention to the image. Once an abnormal image appears, it will directly interfere with the driver's gaze judgment, distract driving attention, and maximize the interference effect of the abnormal image, thereby increasing the driving safety hazard. Therefore, the gaze adjustment factor is set to 1, indicating that a risk assessment level needs to be added on the basis of the baseline hazard level. If the driver's gaze is not focused on the head-up display area, but on the road or other directions, the abnormal image is difficult for the driver to notice, and even if noticed, it will not cause significant interference to driving attention. The risk level is low. Therefore, the gaze adjustment factor is set to 0, and the baseline hazard level is not adjusted.

[0058] Table 1 shows the values ​​of the adjustment factors under various judgment dimensions.

[0059] Table 1

[0060] This application constructs a dynamic scene correction mechanism for the hazard level of target images by setting an adjustment factor, thereby upgrading the comprehensive risk assessment of both the image itself and the scene. When the adjustment factor is 1, the hazard level is automatically increased; when the adjustment factor is 0, the original level remains unchanged. This mechanism can dynamically compensate for the risk amplification effect of abnormal images according to different driving scenarios, avoiding the shortcomings of a single baseline hazard level that cannot adapt to complex driving scenarios, and significantly improving the accuracy and scene adaptability of target hazard level assessment.

[0061] As an optional implementation, in step 304, the classification and adaptation processing of the target image based on the level range of the target hazard level includes: Step S21: If the target hazard level is in the first level range, then identify the abnormal area in the target image and reduce the brightness of the abnormal area. The abnormal area is a local pixel area in the target image that has abnormal display. Step S22: If the target hazard level is in the second level range, a dark mask layer is generated by the overlay graphics generator and used to cover the target image on the head-up display.

[0062] In step S21, if the processor determines that the target hazard level is less than the preset level and that the current target image does not pose a visual risk, then the image is directly output to the head-up display.

[0063] If the processor determines that the target's hazard level falls within the first level range, it classifies the current target image as posing a low-risk visual hazard. The processor locates the abnormal region in the target image, which is a local pixel area displaying abnormally. Subsequently, the processor initiates a region adaptive suppression algorithm to specifically reduce the brightness and saturation of the abnormal region while preserving the display effect of the normal areas in the target image. This effectively weakens glaring visual stimuli while maximizing the readability of normal content.

[0064] In step S22, if the processor determines that the target hazard level is in the second level range, it determines that the current target image poses a high level of visual risk. The processor immediately activates the overlay graphics generator to generate a global dark mask layer. This mask layer is an opaque or highly transparent dark layer, such as a black layer with 80% transparency. The processor uses this dark mask layer to cover the target image on the head-up display, and simultaneously draws high-contrast, highly readable system status prompts on the mask layer, such as... The displayed content is abnormal or Note that an interactive interface is provided that allows you to directly turn off the head-up display via steering wheel control buttons and center console screen buttons. The final safety screen includes a mask layer and system status prompts.

[0065] This application implements graded handling of abnormal images by setting two hazard levels. When the target hazard level is in the Level 1 range, an adaptive regional suppression algorithm reduces the brightness and saturation of only the abnormal area, eliminating local visual interference while preserving the readability of most normal information, avoiding the impact of over-processing on user experience. When the target hazard level is in the Level 2 range, a dark masking layer completely covers the dangerous image, supplemented by clear system prompts and quick closure of the entry point, fundamentally blocking the visual impact of the abnormal image and ensuring driving safety. This graded handling logic balances safety and user experience, providing precise and efficient response strategies based on different risk levels, improving the head-up display's safety protection capabilities and scenario adaptability when applications malfunction.

[0066] As an optional implementation method, the training process of the recognition model includes: Step S31: Obtain the normal image displayed on the head-up display; Step S32: By inducing rendering anomalies on the head-up display, a fault is injected into the normal image, and an edge-abnormal image is generated by image processing of the normal image. Step S33: Use normal images, images after fault injection, and images with abnormal edges as training samples, and determine the sample hazard level marked on the training samples. Step S34: Train the initial identification model based on the training samples and the sample hazard level to obtain the trained identification model.

[0067] Step S31: The processor acquires the display screen of the head-up display under normal operating conditions. To ensure the scene coverage and authenticity of the training data, normal display screens under multiple scenarios can be collected through real-vehicle road sampling. The collection objects cover various vehicle platforms, and the collection environment includes various lighting conditions such as daytime lighting, nighttime lighting, and tunnel lighting transitions. The collected content includes the navigation interface, vehicle instrument interface, multimedia interface, vehicle control interaction interface, and other conventional head-up display content, thereby forming a comprehensive and scene-rich normal screen dataset.

[0068] Step S32: The processor generates anomalous samples in two ways to expand the diversity and generalization of training samples. Firstly, in a laboratory environment, rendering anomalies are actively induced on the head-up display and fault injection is performed through software simulation or hardware interference. Real-world anomalous images are captured, including white or black screens caused by application process crashes, snowflake stripes and color block disorder caused by graphics interface call errors, checkerboard and mosaic effects caused by texture loss, continuous frame freezing, and low frame rate stuttering caused by rendering delays. Secondly, normal images are edited using image processing techniques. Edge-abnormal images are generated through overexposure enhancement, color shift, high-frequency noise injection, and local solid color coverage, enabling the recognition model to learn the features of easily confused samples such as weak anomalies and local anomalies.

[0069] Step S33: The processor combines the above normal images, the real abnormal images obtained through fault injection, and the edge abnormal images generated through image processing to form a large-scale and diverse training sample set. Combined with the labeled data of the training samples, the processor determines the sample hazard level corresponding to each training sample, so that the recognition model establishes a mapping relationship from deep image features to risk level.

[0070] Step S34: The processor uses the constructed training samples and corresponding labeled hazard levels as supervised data to iteratively train the initial recognition model. During the training process, the recognition model continuously optimizes the encoder's feature extraction capability and the classifier's discrimination capability, gradually improving its understanding of image semantics and contextual information, and finally converges to obtain the trained recognition model, enabling it to have stable and reliable anomaly recognition and hazard level determination capabilities.

[0071] This application constructs a large-scale, diverse training dataset by combining real-vehicle road sampling, fault injection, and adversarial example generation. This enables the initial recognition model to fully learn the essential feature differences between normal images and various abnormal images. After training, the model can deeply understand the semantics and contextual information of the image, accurately distinguishing between images with solid color collapse caused by anomalies and similar display content such as normal white backgrounds and snow scenes, reducing the false positive rate caused by scene similarity. Simultaneously, the initial recognition model's ability to perceive and recognize irregular, non-global anomalies such as local color blocks, image tearing, and partial texture loss is far superior to traditional simple detection rules based on fixed thresholds, effectively improving the accuracy of anomaly feature recognition.

[0072] This application provides a schematic diagram of the overall process of vehicle-mounted image display, such as... Figure 4 As shown, it includes the following content.

[0073] Step 401: Train and build the recognition model.

[0074] Normal images of head-up displays under various vehicle models, lighting conditions, and interfaces are collected; various abnormal images and edge-abnormal images are generated through fault injection and image processing; a training dataset containing normal samples, fault samples, and edge-abnormal samples is constructed and its levels are labeled; the model is trained based on a lightweight convolutional neural network to obtain an anomaly recognition model with semantic understanding capabilities.

[0075] Step 402: Determine the baseline hazard level of the target image by recognizing the model.

[0076] The captured RGB image frames are input into the encoder, which extracts deep features from low-level edges and textures to high-level semantics step by step; the five-dimensional confidence scores of anomalies are output through a fully connected classifier to form a five-dimensional confidence vector; the confidence scores of each dimension are weighted and summed to obtain the total anomaly score; the baseline hazard level of the target image is determined by comparing the total anomaly score with the set score.

[0077] Step 403: Determine the value of the adjustment factor for each driving scenario.

[0078] The system collects four types of information in real time: in-vehicle light intensity, vehicle driving status, current time, and driver's field of vision. Based on in-vehicle light intensity, it determines whether the environment is low-light and assigns a light adjustment factor. Based on vehicle driving status, it determines whether the operating condition is high-risk and assigns a condition adjustment factor. Based on the current time, it determines whether it is nighttime and assigns a time adjustment factor. Based on the driver's line of vision, it determines whether the driver is looking at the head-up display and assigns a gaze adjustment factor.

[0079] Step 404: Determine the target hazard level by combining the baseline hazard level and the values ​​of each adjustment factor.

[0080] The baseline hazard level is superimposed with the values ​​of each adjustment factor, and the image anomalies and the risk amplification effect of the driving scenario are combined to obtain the final target hazard level.

[0081] Step 405: Graded safety procedures and output of final safety screen.

[0082] When the target hazard level is 0, no processing is performed, and the original image is output directly. When the target hazard level is 1 (first-level range), adaptive regional suppression is activated, reducing only the brightness and saturation of abnormal areas. When the target hazard level is 2 (second-level range), a dark mask layer is generated to fully cover the image, and a high-contrast safety warning is superimposed. The final safe image is then output to a head-up display.

[0083] Based on the same technical concept, this application provides a display device for in-vehicle images, such as... Figure 5 As shown, the device includes: The interception module 501 is used to intercept the target image to be sent to the vehicle display; The processing module 502 is used to determine the target hazard level of the target image by processing the target image, wherein the target hazard level is used to indicate the degree of impact on driving safety after the target image is projected onto the head-up display; The determination module 503 is used to determine that if the target hazard level is greater than the preset level, the target image is an abnormal image, wherein the abnormal image refers to an image that displays abnormalities and produces strong light interference; The output module 504 is used to perform hierarchical adaptation processing on the target image based on the level range of the target hazard level, generate a safety display image, and output the safety display image to the head-up display.

[0084] Optionally, the processing module 502 is specifically used for: The target image is identified by a pre-set recognition model to determine the baseline hazard level of the target image. The baseline hazard level is used to indicate the degree of abnormality of the content of the target image itself. The value of each adjustment factor is determined based on the current driving scenario, where the value of the adjustment factor is used to indicate the degree of influence of the driving scenario on the abnormal image; The target hazard level of the target image is obtained by superimposing the value of each adjustment factor on the baseline hazard level.

[0085] Optionally, the processing module 502 is specifically used for: The deep features in the target image are extracted step by step by the encoder in the recognition model. The deep features are used to indicate the hierarchical features of the target image from low-level edges and textures to high-level semantics. The deep features are classified by a classifier, and the confidence scores corresponding to each anomaly dimension are output. The anomaly dimension is used to indicate different types of rendering anomalies. The anomaly score is obtained by weighted summation of the confidence scores for each anomaly dimension. The baseline hazard level of the target image is determined by comparing the abnormal score with the set score.

[0086] Optionally, the processing module 502 is specifically used for: Acquire information such as in-vehicle light intensity, vehicle operating status, current time, and the driver's line of sight. If the current environment is determined to be low light based on the light intensity inside the vehicle, then the light adjustment factor is set to 1. If the current operating condition is determined to be high-risk based on the vehicle's operating status, then the value of the operating condition adjustment factor is set to 1. If the current time indicates that it is nighttime, then the time period adjustment factor is set to 1. If it is determined that the driver is currently looking at the head-up display based on the driver's field of vision, then the gaze adjustment factor is set to 1.

[0087] Optionally, the output module 504 is used for: If the target hazard level is in the first level range, then identify the abnormal area in the target image and reduce the brightness of the abnormal area. The abnormal area is a local pixel area in the target image where there is abnormal display. If the target hazard level is in the second level range, a dark mask layer is generated by the overlay graphics generator and used to cover the target image on the head-up display.

[0088] Optionally, the device is also used for: If the target hazard level is lower than the preset level, the target image is determined to be a normal image; Output the normal image to the head-up display.

[0089] Optionally, the device is also used for: To capture the normal image displayed on the head-up monitor; By inducing rendering anomalies on the head-up display, faults are injected into the normal image, and image processing is performed on the normal image to generate an image with abnormal edges. Normal images, images after fault injection, and images with edge anomalies are used as training samples, and the hazard level of the samples labeled on the training samples is determined. The initial identification model is trained based on the training samples and the hazard level of the samples to obtain the trained identification model.

[0090] like Figure 6 As shown, this application provides an electronic device including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0091] Memory 603 is used to store computer programs.

[0092] In one embodiment of this application, the processor 601, when executing the program stored in the memory 603, implements the vehicle image display method provided in any of the foregoing method embodiments.

[0093] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the vehicle image display method provided in any of the foregoing method embodiments.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0096] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0097] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A display method of a vehicle-mounted image, characterized by, The method includes: Intercept the target image to be sent to the vehicle's display; By processing the target image, the target hazard level of the target image is determined, wherein the target hazard level is used to indicate the degree of impact on driving safety after the target image is projected onto the head-up display; If the target hazard level is greater than the preset level, the target image is determined to be an abnormal image, wherein the abnormal image refers to an image that displays abnormalities and produces strong light interference; Based on the level range of the target hazard level, the target image is subjected to hierarchical adaptation processing to generate a safe display image, and the safe display image is output to the head-up display.

2. The method of claim 1, wherein, By processing the target image, the target hazard level of the target image is determined, including: The target image is subjected to abnormal feature identification by a preset recognition model to determine the baseline hazard level of the target image, wherein the baseline hazard level is used to indicate the degree of abnormality of the content of the target image itself; The value of each adjustment factor is determined based on the current driving scenario, wherein the value of the adjustment factor is used to indicate the degree of influence of the driving scenario on the abnormal image; The target hazard level of the target image is obtained by superimposing the value of each adjustment factor on the baseline hazard level.

3. The method of claim 2, wherein, The baseline hazard level of the target image is determined by identifying abnormal features using a preset recognition model, including: The deep features in the target image are extracted step by step by the encoder in the recognition model, wherein the deep features are used to indicate the hierarchical features of the target image from low-level edges and textures to high-level semantics; The deep features are classified by a classifier, and the confidence scores corresponding to each anomaly dimension are output. The anomaly dimension is used to indicate different types of scene rendering anomalies. The anomaly score is obtained by weighted summation of the confidence scores for each anomaly dimension. Based on the comparison between the abnormal score and the set score, the baseline hazard level of the target image is determined.

4. The method of claim 2, wherein, The value of each adjustment factor is determined based on the current driving scenario, including: Acquire information such as in-vehicle light intensity, vehicle operating status, current time, and the driver's line of sight. If the current environment is determined to be low light based on the in-vehicle light intensity, then the value of the light adjustment factor is determined to be 1; If the current operating condition is determined to be high-risk based on the vehicle's operating status, then the value of the operating condition adjustment factor is determined to be 1. If the current time indicates that it is nighttime, then the time period adjustment factor is set to 1. If it is determined that the driver is currently looking at the head-up display based on the driver's field of vision, then the gaze adjustment factor is set to 1.

5. The method of claim 1, wherein, Based on the level range of the target hazard level, the hierarchical adaptation processing of the target image includes: If the target hazard level is in the first level range, then an abnormal region in the target image is identified and the brightness of the abnormal region is reduced, wherein the abnormal region is a local pixel region in the target image where there is abnormal display; If the target hazard level is in the second level range, a dark mask layer is generated by the overlay graphics generator and used to cover the target image on the head-up display.

6. The method of claim 1, wherein, After determining the target hazard level of the target image, the method further includes: If the target hazard level is less than the preset level, then the target image is determined to be a normal image; The normal image is output to the head-up display.

7. The method of claim 2, wherein, The training process of the recognition model includes: Acquire the normal image displayed on the head-up display; By inducing rendering anomalies in the head-up display, a fault is injected into the normal image, and an image with edge anomalies is generated by image processing of the normal image. The normal image, the image after fault injection, and the image with abnormal edges are used as training samples, and the hazard level of the samples marked on the training samples is determined. The initial identification model is trained based on the training samples and the hazard levels of the samples to obtain the trained identification model.

8. A display device for a vehicle-mounted image, characterized by comprising: The device includes: The interception module is used to intercept the target image to be sent to the vehicle display; The processing module is used to determine the target hazard level of the target image by processing the target image, wherein the target hazard level is used to indicate the degree of impact of the target image on driving safety after it is projected onto the head-up display; The determination module is used to determine that the target image is an abnormal image if the target hazard level is greater than a preset level, wherein the abnormal image refers to an image that displays abnormalities and produces strong light interference; The output module is used to perform hierarchical adaptation processing on the target image based on the level range of the target hazard level, generate a safety display image, and output the safety display image to the head-up display.

9. An electronic device, comprising: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A vehicle characterized by comprising: The vehicle includes a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to implement the method as described in any one of claims 1 to 7.

Images