Head-up display instrument detection method and device and vehicle

Abstract

The invention relates to a head-up display instrument detection method and device and a vehicle, and the method comprises the steps: collecting first image data corresponding to a to-be-displayed image from a head-up display instrument, and enabling the head-up display instrument to display the to-be-displayed image according to the first image data; determining visual elements contained in the to-be-displayed image according to the first image data; and analyzing the working state of the head-up display instrument based on the visual elements. According to the technical scheme, the image data used for displaying the image is directly collected from the head-up display instrument, and the visual elements are extracted from the image data to analyze the accuracy of information displayed by the head-up display instrument. Therefore, since the image data is directly acquired from the head-up display, a shooting device does not need to shoot and transmit pictures, the influence of factors such as environment and detection equipment hardware in the detection process of the head-up display is avoided, the method can adapt to detection in a complex driving scene, and the detection precision is improved; the detection data acquisition real-time performance is improved, and the hardware cost of the detection equipment is also reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a head-up display detection method, device and vehicle. Background Art

[0002] Head-Up Display (HUD) is a device that can project the driving assistance information of the Advanced Driving Assistance System (ADAS) directly onto a transparent screen in front of the driver's line of sight. It has been widely used in smart cars. To ensure the safety and comfort of smart driving, it is crucial that the HUD can correctly and reliably receive and display the ADAS driving assistance information. Summary of the invention

[0003] The present application provides a head-up display detection method, device and vehicle to solve the technical problems in the prior art that HUD detection is easily affected by environmental factors, has low detection accuracy, and is difficult to adapt to complex driving scenarios.

[0004] According to one aspect of an embodiment of the present application, a head-up display detection method is provided, including:

[0005] collecting first image data corresponding to the image to be displayed from the head-up display, and displaying the image to be displayed according to the first image data by the head-up display;

[0006] Determining visual elements included in the image to be displayed according to the first image data;

[0007] The operating status of the head-up display is analyzed based on the visual elements.

[0008] Optionally, the method further includes:

[0009] Collecting assisted driving information from a driving assistance system, wherein the head-up display generates image data according to the assisted driving information;

[0010] The analyzing the working state of the head-up display based on the visual element includes:

[0011] extracting driving characteristics from the auxiliary driving information;

[0012] The working state of the head-up display is analyzed according to the matching degree between the driving characteristics and the visual elements.

[0013] Optionally, determining the visual elements included in the image to be displayed according to the first image data includes:

[0014] Analyzing the first image data to obtain element information corresponding to each of the visual elements;

[0015] determining the visual element according to the element information;

[0016] and / or,

[0017] The determining, according to the first image data, the visual elements contained in the image to be displayed comprises:

[0018] Perform image reconstruction based on the first image data to obtain an image to be detected;

[0019] The visual element is extracted from the image to be detected.

[0020] Optionally, the method further includes:

[0021] Collecting the assisted driving information sent by the driving assistance system and the driving reminder information generated after the head-up display collects and processes the information; wherein the head-up display generates the driving reminder information according to the received assisted driving information, and generates the first image data according to the driving reminder information;

[0022] Comparing the auxiliary driving information with the driving reminder information;

[0023] The working status of the head-up display is analyzed according to the comparison result.

[0024] Optionally, comparing the visual element with a preset visual element includes:

[0025] determining a current driving scene according to the image data;

[0026] Selecting a preset element corresponding to the current driving scene from a preset element feature library;

[0027] The visual element is compared with a preset element corresponding to the current driving scene.

[0028] Optionally, when the analysis result shows that the working state of the head-up display is abnormal, the method further includes:

[0029] Sending the analysis result to the head-up display and the auxiliary driving system, the head-up display restarts after receiving the analysis result, and the auxiliary driving system resends the driving assistance information to the head-up display after receiving the analysis result;

[0030] collecting second image data regenerated by the head-up display according to the driving assistance information resent by the driving assistance system;

[0031] The operating status of the head-up display is analyzed according to the second image data.

[0032] According to another aspect of an embodiment of the present application, a head-up display detection device is provided, comprising:

[0033] An image data acquisition module, used for acquiring first image data corresponding to the image to be displayed from the head-up display, and the head-up display displays the image to be displayed according to the first image data;

[0034] A visual element determination module, configured to determine the visual elements included in the image to be displayed according to the first image data;

[0035] An analysis module is used to analyze the working status of the head-up display based on the visual elements.

[0036] According to another aspect of an embodiment of the present application, a vehicle is provided, comprising the above-mentioned head-up display detection device.

[0037] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0038] According to another aspect of the embodiment of the present application, there is provided an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0039] The memory is used to store computer programs;

[0040] The processor is used to implement the above method steps when executing a computer program.

[0041] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method steps are implemented.

[0042] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0043] In the embodiment of the present application, image data used to display images is directly collected from the head-up display, and visual elements are extracted from the image data to analyze the accuracy of the information displayed by the head-up display. In this way, since the image data is directly obtained from the head-up display, there is no need for a camera to take pictures and transmit pictures. During the head-up display detection process, the influence of factors such as the environment and the hardware of the detection equipment is avoided, and the detection in complex driving scenes can be adapted to improve the detection accuracy; the real-time acquisition of detection data is improved, and the hardware cost of the detection equipment is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the architecture of a HUD detection system provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a head-up display detection device provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a head-up display detection method provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a flow chart of a head-up display detection method provided in another embodiment of the present application;

[0048] Figure 5 A schematic diagram of a flow chart of a head-up display detection method provided in another embodiment of the present application;

[0049] Figure 6 A schematic diagram of a flow chart of a head-up display detection method provided in another embodiment of the present application;

[0050] Figure 7 A schematic diagram of visual elements provided for an embodiment of the present application;

[0051] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0053] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0054] For ease of description, spatial relative terms may be used herein to describe the relative positional relationship or movement of one element or feature relative to another element or feature as shown in the figure, such as "inside", "outside", "inner side", "outer side", "below", "below", "above", "above", "front", "back", etc. Such spatial relative terms are intended to include different orientations of the device in use or operation in addition to the orientation depicted in the figure. For example, if the device in the figure undergoes a position flip or a change in posture or a change in motion state, then these directional indications will also change accordingly, for example: an element described as "below other elements or features" or "below other elements or features" will subsequently be oriented as "above other elements or features" or "above other elements or features". Therefore, the example term "below..." can include both upper and lower orientations. The device can be oriented otherwise (rotated 90 degrees or in other directions) and the spatial relative descriptors used herein are interpreted accordingly.

[0055] At present, the detection of HUD generally involves shooting the image displayed by the HUD, and analyzing whether the HUD display is normal based on the shot image. However, since the image displayed by the HUD is generally projected on a transparent screen in front of the driver's line of sight, it is difficult to ensure that the HUD display image is shot in real time in the direction of the driver's line of sight during driving. Secondly, affected by the ambient light, the optical hardware conditions of the detection equipment shooting device, etc., the captured image has quality problems such as ghosting and distortion. It is necessary to use an algorithm to process the captured image before subsequent analysis can be performed, which also affects the real-time performance of HUD detection. The accuracy of HUD detection is not only affected by the quality of the captured image, but also further affected by the image processing algorithm and real-time performance, and the accuracy is seriously insufficient. Furthermore, the related technology generally analyzes the HUD display effect based on the grayscale value, pixel value, etc. of the captured image, and does not analyze and verify the accuracy of the actual display content of the HUD.

[0056] In the embodiments of the present application, HUD detection is performed by directly acquiring image data for generating an image to be displayed from the HUD, so as to solve the technical problems in the prior art that HUD detection is easily affected by environmental factors, has low detection accuracy, and is difficult to adapt to complex driving scenarios.

[0057] Figure 1 A schematic diagram of the system architecture of HUD detection provided in the embodiment of the present application. Figure 1As shown, the HUD 10 receives the auxiliary driving information from the ADAS driving assistance system 30, generates a corresponding image and projects it onto the windshield in front of the driver's line of sight for display. The HUD detection device 20 collects image data for generating the image to be displayed from the HUD 10, determines the visual elements contained in the image to be displayed according to the image data, and analyzes the working state of the HUD 10 based on the visual elements.

[0058] The visual elements refer to the basic units that constitute the image to be displayed, including but not limited to text, signs, vehicle models, lane lines, warning waves and other elements. Figure 7 In the image to be displayed, the visual elements include: text prompts, lane lines, own vehicle model, warning waves, and collision target model.

[0059] Among them, Figure 1 As shown, the HUD 10 includes the following main parts:

[0060] The information acquisition module 11 is used to receive the auxiliary driving information sent by the ADAS driving assistance system 30; perform format conversion, analysis, filtering and other processing on the received information, and obtain driving reminder information to send to the control module 12;

[0061] The control module 12 is used to generate image data according to the driving reminder information, generate an image to be displayed according to the image data, and send the image to be displayed to the projection display module 13;

[0062] The projection display module 13 is used to project the image to be displayed onto a transparent screen in front of the driver's sight, such as onto the front windshield.

[0063] Optionally, the HUD detection device 20 may collect image data from the control module 12 and extract visual elements from the image data to analyze the accuracy of the HUD display information.

[0064] Optionally, the HUD detection device 20 can also collect assisted driving information from the ADAS driving assistance system 30, extract driving characteristics from the assisted driving information, and analyze whether the HUD accurately displays the assisted driving information through the matching degree between the driving characteristics and the visual elements.

[0065] Optionally, the HUD detection device 20 can also collect driving reminder information from the information collection module 11 and collect auxiliary driving information from the ADAS driving assistance system 30, and analyze whether the working status of the HUD 10 is normal through information pairing.

[0066] If the analysis result of the HUD detection device 20 is that the HUD 10 is working abnormally, the HUD detection device 20 can send the analysis result to the HUD 10 and the ADAS driving assistance system 30. The HUD 10 restarts after receiving the analysis result, and the ADAS driving assistance system 30 resends the driving assistance information to the HUD 10 after receiving the analysis result. The HUD 10 regenerates the image data based on the resent driving assistance information. The HUD detection device 20 obtains the image data from the control module 12 again and reanalyzes the working status of the HUD 10.

[0067] The HUD detection device 20 can also upload the analysis results to a cloud server or send them to a designated terminal for staff to view, monitor and analyze. The analysis results can be a separate report, which can include the test process, evaluation results, log records, problem diagnosis and improvement suggestions.

[0068] In the embodiment of the present application, the HUD detection device 20 may be a smart terminal, which may exchange information with the HUD 10 and the ADAS driving assistance system 30 via communication methods such as WIFI, Bluetooth, and NFC.

[0069] Figure 2 This is a schematic diagram of the structure of a head-up display detection device provided in an embodiment of the present application. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 2 As shown, the HUD detection device 20 includes:

[0070] An image data acquisition module 21 is used to acquire first image data corresponding to an image to be displayed from a head-up display, and the head-up display displays the image to be displayed according to the first image data;

[0071] A visual element determination module 22, configured to determine the visual elements included in the image to be displayed according to the first image data;

[0072] The analysis module 23 is used to analyze the working status of the head-up display based on the visual elements.

[0073] The image data acquisition module 21 may acquire image data from the control module 12 of the HUD 10 .

[0074] Optionally, the visual element determination module 22 is used to parse the first image data to obtain element information corresponding to each of the visual elements; and determine the visual element according to the element information. Alternatively, the visual element determination module 22 is used to reconstruct an image based on the first image data to obtain an image to be detected; and extract the visual element from the image to be detected.

[0075] Optionally, the visual element determination module 22 may also extract visual elements by using the above two methods at the same time, and compare and remove duplicate visual elements extracted by the two methods to obtain visual elements for subsequent analysis.

[0076] Optionally, the analysis module 23 may include:

[0077] An element comparison submodule 231, used to compare the visual element with a preset visual element;

[0078] The analysis submodule 232 is used to analyze the working status of the head-up display according to the comparison result.

[0079] Among them, a preset visual element library can be set in the HUD detection device 20, and a matching algorithm is used to automatically match the preset visual element corresponding to the image element extracted based on the image data from the preset visual element library, and the visual element is compared with the preset visual element to determine whether the visual element can be correctly displayed in the image to be displayed generated according to the image data.

[0080] Optionally, the element comparison submodule 231 is used to determine the current driving scene based on the image data; select a preset element corresponding to the current driving scene from a preset element feature library; and compare the visual element with the preset element corresponding to the current driving scene.

[0081] Optionally, the HUD detection device 20 further includes: a detection information acquisition module 24 for acquiring auxiliary driving information from an ADAS driving assistance system 30, wherein the head-up display generates the image data according to the auxiliary driving information;

[0082] The analysis module 23 may include:

[0083] A feature extraction submodule 233 is used to extract driving features from the auxiliary driving information;

[0084] A matching submodule 234 is used to determine the matching degree between the driving feature and the visual element.

[0085] The analysis submodule 232 is used to analyze the working status of the head-up display according to the matching degree.

[0086] Optionally, the detection information collection module 24 is also used to collect auxiliary driving information from the ADAS driving assistance system 30, and collect driving reminder information output by the information collection module 11 of the HUD 10.

[0087] The analysis module 23 may also include:

[0088] An information comparison submodule 235 is used to compare the auxiliary driving information with the driving reminder information;

[0089] The analysis submodule 232 is used to analyze the working status of the head-up display according to the comparison result.

[0090] Optionally, the analysis module 23 is also used to send the analysis result to HUD10 and ADAS assisted driving system 30 when the analysis result shows that the working state of the head-up display is abnormal. Among them, the analysis module 23 sends the analysis result to the control module 12 of HUD10. The control module 12 controls HUD10 to restart according to the analysis result. After restarting, HUD10 receives the driving assistance information resent by the ADAS assisted driving system 30, and generates image data again according to the driving assistance information. The HUD detection device 20 collects the second image data regenerated by the head-up display according to the driving assistance information resent by the assisted driving system; and analyzes the working state of the head-up display according to the second image data.

[0091] Optionally, the analysis module 23 can also report the analysis results in the form of a report to a cloud server or a designated terminal for testers to view and analyze; or, send the analysis results to the vehicle to which the head-up display belongs to remind the driver of the HUD working status.

[0092] A head-up display detection method provided by an embodiment of the present invention is introduced below.

[0093] The detection method of the embodiment of the present application can be applied to any vehicle with a HUD. The HUD can be detected during the vehicle testing process, or the HUD can be detected in real time during the actual driving of the vehicle.

[0094] During the test, in order to ensure the comprehensiveness and accuracy of HUD detection, it is necessary to select roads with different traffic conditions for testing, including highways, city streets, complex intersections, etc., and ensure that there are real driving assistance scenarios during the test. The ADAS driving assistance system sends the assisted driving information to the HUD. The assisted driving information sent by the ADAS driving assistance system to the HUD may include but is not limited to vehicle positioning, maps, navigation, pedestrian / obstacle detection, lane keeping assistance, and road safety warning information.

[0095] Figure 3 A schematic diagram of a head-up display detection method provided in an embodiment of the present application. Figure 3 As shown, the method includes the following steps S11-S13.

[0096] Step S11 : collecting first image data corresponding to the image to be displayed from the head-up display, and the head-up display displays the image to be displayed according to the first image data.

[0097] Step S12: determining visual elements included in the image to be displayed according to the first image data.

[0098] Among them, the visual elements contained in the HUD display image include but are not limited to the following types: the vehicle model itself, vehicle driving parameters such as speed, the vehicle's surrounding environment such as lane lines, traffic lights, etc.; if a collision warning occurs, the warning text, collision target (including front and rear pedestrians, vehicles, cyclists or unknown obstacles, etc.), radar warning waves, etc.

[0099] Step S13: analyzing the working status of the head-up display based on the visual elements.

[0100] In this embodiment, the HUD generates image data for generating an image to be displayed based on the assisted driving information sent by the ADAS driving assistance system. For example, the assisted driving information sent by the ADAS driving assistance system is a collision warning, and the image generated by the HUD based on the assisted driving information should contain visual elements related to the collision warning, such as collision targets such as pedestrians, vehicles, cyclists, and unknown obstacles. The HUD detection device collects image data for generating an image to be displayed from the HUD, extracts visual elements that should be included in the image to be displayed based on the image data, and judges whether the HUD is working normally or not by the extracted visual elements. In this way, image data is directly obtained from the head-up display, and there is no need for a camera to take pictures and transmit pictures. The influence of factors such as the environment and the hardware of the detection equipment is avoided during the detection process of the head-up display, and it can adapt to detection in complex driving scenes and improve detection accuracy; it improves the real-time acquisition of detection data and reduces the hardware cost of the detection equipment.

[0101] After the HUD detection device collects image data from the HUD, it can directly extract visual elements from the image data, or reconstruct the image to be displayed based on the image data and extract visual elements from the reconstructed image. The following is a detailed description of the method for extracting visual elements in step S12.

[0102] Method 1: Direct extraction

[0103] The above step S12 includes: parsing the first image data to obtain element information corresponding to each of the visual elements; and determining the visual element according to the element information.

[0104] The image to be displayed generated by the HUD usually contains multiple visual elements, such as a vehicle model and pedestrians. From the first image data, element information corresponding to multiple different elements can be parsed, so that the visual element can be determined according to the element information.

[0105] The image data generated by the HUD can be stored in the form of one or more digital matrices, and the size of the matrix depends on the number of pixels in the image. Black and white images can store data in one matrix, and color images can store data in three matrices (or channels) - red, green, and blue. Of course, if the color image uses other color modes besides RGB, such as HSB, CMYK or LAB, the color mode corresponding matrix is ​​used to store the data.

[0106] Visual elements are extracted based on the pixel value of each pixel point in the image data. For example, the original pixels in the image data can be used as separate features for image feature extraction; for color images, image features can be extracted based on the average pixel value of each channel; or, edge features can be extracted by identifying drastic changes in pixel values; and so on. For another example, due to the particularity of HUD display images, a visual element extraction model can be pre-trained to extract the visual elements to be displayed in the image to be displayed from the image data through the model. There are many ways to directly extract visual elements based on image data, which will not be repeated again.

[0107] The image data generated by the HUD may also be a collection of element information of visual elements contained in the image to be displayed. Based on the element information after the image data is parsed, all the visual elements contained in the image to be displayed are obtained.

[0108] Method 2: Reconstruct the image and then extract

[0109] The above step S12 includes: reconstructing an image based on the first image data to obtain an image to be detected; and extracting the visual element from the image to be detected.

[0110] After the HUD detection device collects image data, it can reconstruct the image to be displayed based on the image data, and then use image feature extraction algorithms, image recognition algorithms, etc. to extract visual elements from the reconstructed image to be displayed.

[0111] Since HUD generally projects images onto the windshield in front of the driver, in order to ensure the effect of displaying the image on the windshield, the image may be specially processed. If the image is reconstructed directly based on the image data, the reconstructed image may have problems such as distortion and disordered positions of visual elements, which may lead to inaccurate visual elements recognized based on the reconstructed image. For example, the navigation direction indication is wrong. Therefore, before reconstructing the image, an image processing algorithm can be used to pre-process the image data, such as correction, denoising, enhancement and other operations, so that the reconstructed image is reliable and accurate, and the visual elements therein are clear and prominent and can be accurately identified.

[0112] For the reconstructed image, a feature extraction algorithm can be used to extract key features in the image, and an image recognition algorithm can be used to identify visual elements in the image. Alternatively, a visual element recognition model can be pre-trained based on the characteristics of the HUD display image, and the visual elements in the reconstructed image can be identified through the model. Among them, the feature extraction algorithms that can be used include but are not limited to the following algorithms: Scale-invariant feature transform (SIFT), Histogram of Oriented Gradient (HOG), Local Binary Pattern (LBP), Features from Accelerated Segment Test (FAST), etc. The image recognition algorithms that can be used include but are not limited to: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Support Vector Machine (SVM), pre-trained models (such as VGG16, ResNet, Inception, etc.).

[0113] In the embodiment of the present application, after the visual elements are extracted from the image data collected by the HUD, the extracted visual elements can be compared with the preset visual elements to determine whether the HUD display is correct. Figure 4 As shown, the above step S13 includes: step S131, comparing the visual element with the preset visual element, wherein the preset visual element is a visual element in the preset element database; step S132, determining the working state of the HUD according to the comparison result.

[0114] The preset element database contains a large number of preset elements. In order to quickly screen out the preset visual elements for comparison, the driving scene corresponding to the image data can be determined, such as a collision warning scene, a high-speed driving scene, a parking scene, etc., and the corresponding preset visual elements can be screened out from the preset element database based on the driving scene for subsequent visual element comparison. Optionally, the above step S13 includes steps A1-A4.

[0115] Step A1, determining a current driving scene according to the image data;

[0116] Step A2, selecting a preset element corresponding to the current driving scene from a preset element feature library;

[0117] Step A3, comparing the visual element with a preset element corresponding to the current driving scene;

[0118] Step A4: analyzing the working status of the head-up display according to the element comparison result.

[0119] In order to quickly compare the visual elements, the element features of the visual elements and the preset visual elements can be extracted for comparison, or the element features of each preset visual element are pre-stored in the preset element data volume. For example, the type, size, color and other features of the visual elements are compared. Whether the extracted visual elements are consistent with the preset visual elements is determined by feature comparison.

[0120] In this embodiment, the consistency between the visual elements and the preset visual elements can be compared to determine whether the HUD is working normally. In order to more comprehensively and accurately determine whether the HUD has correctly displayed all visual elements, the HUD display content can be further verified through the assisted driving information sent by the ADAS assisted driving system. In another embodiment, the method also includes: step S14, collecting assisted driving information from the driving assistance system, wherein the head-up display generates image data based on the assisted driving information; the above step S13 includes: step S133, extracting driving characteristics from the assisted driving information; step S134, analyzing the working status of the HUD based on the matching degree between the driving characteristics and the visual elements.

[0121] For example, based on the auxiliary driving information, it is determined that the driving characteristics are collision warnings, and the collision target is pedestrians. Based on these driving characteristics, the image displayed by the HUD should include visual elements such as the vehicle model, pedestrians, and radar warning waves. If the visual elements collected based on the image data include the above three visual elements, it can be determined that the driving characteristics match the visual elements.

[0122] Optionally, in step S13, the matching degree between the driving characteristics extracted from the auxiliary driving information and the visual elements can be determined first. If they match, the visual elements are then compared with the preset visual elements, and whether the HUD is working properly is determined based on the comparison result; if they do not match, the HUD can be directly determined to be abnormal, without the need to compare the visual elements with the preset visual elements. Alternatively, the visual elements can be compared with the preset visual elements first, and when the comparison results are consistent, the matching degree between the driving characteristics and the visual elements can be determined; when the comparison results are inconsistent, the HUD is directly determined to be abnormal.

[0123] When the extracted visual elements are inconsistent with the preset visual elements, or the driving characteristics do not match the visual elements, the HUD is abnormal. However, it is necessary to further determine whether the HUD is due to an abnormal information collection module that fails to collect the correct auxiliary driving information or fails to correctly process the auxiliary driving information to obtain driving reminder information, whether the control module is abnormal and fails to correctly generate image data based on the collected information, or whether the ADAS auxiliary driving system sends the wrong auxiliary driving information. In order to accurately locate the cause of the abnormality, such as Figure 5 As shown, the method of the embodiment of the present application also includes the following steps:

[0124] Step S15, collecting the assisted driving information sent by the driving assistance system and the driving reminder information generated after the head-up display collects and processes the information; wherein the head-up display generates the driving reminder information according to the received assisted driving information, and generates the first image data according to the driving reminder information;

[0125] Step S16, comparing the auxiliary driving information with the driving reminder information;

[0126] Information comparison can be performed by comparing the driving characteristics analyzed from the auxiliary driving information and the driving reminder information respectively to determine the consistency of the two information.

[0127] Step S17, analyzing the working status of the head-up display according to the information comparison result.

[0128] For example, the collision warning driving characteristics obtained from the analysis of the assisted driving information include: the collision target is a pedestrian, the distance between the own vehicle and the pedestrian, the position of the own vehicle, the lane lines and traffic lights around the own vehicle, etc., while the collision warning driving characteristics analyzed from the driving reminder information generated after HUD collection and processing do not include the collision target pedestrian, then the information collection module of the HUD is abnormal; if the driving characteristics analyzed from the assisted driving information and the driving reminder information are the same, both include the collision target pedestrian, but the visual element of the pedestrian is not identified in the image data collected by the HUD, then the control module of the HUD is abnormal; if there is a collision warning scene but the target collision object is not analyzed in the assisted driving information, then the ADAS assisted driving system is abnormal.

[0129] It can be seen that by comparing the auxiliary driving information and driving reminder information, combined with the analysis of visual elements, it is not only possible to accurately verify whether the HUD is working properly, but also to further analyze the location where the abnormality occurs, so that testers or users can quickly and accurately locate abnormal equipment and solve faults in a timely manner.

[0130] In the embodiment of the present application, when the analysis result is that the HUD working state is abnormal, the HUD detection device can not only send the analysis result in the form of a report to the cloud server or the designated terminal for the tester or user to view, but also perform a fault repair operation. The method also includes steps B1-B3:

[0131] Step B1, sending a restart instruction to the head-up display and the auxiliary driving system, the head-up display restarts after receiving the restart instruction, and the auxiliary driving system resends driving assistance information to the head-up display after receiving the restart instruction;

[0132] Step B2, collecting second image data regenerated by the head-up display according to the driving assistance information resent by the driving assistance system;

[0133] Step B3: analyzing the working status of the head-up display according to the second image data.

[0134] The HUD detection device sends a restart command to the HUD and ADAS assisted driving system. The HUD restarts, and the ASAS assisted driving system resends the assisted driving information. After the HUD restarts, the assisted driving information is re-formatted, parsed, filtered, and other processes are performed, and image data is generated based on the processed driving reminder information; the HUD detection device collects the image data again to detect and analyze the HUD. If the detection and analysis results show that the HUD is abnormal, a reminder can be issued to the tester or the vehicle driver. Remind the driver to take appropriate temporary measures to improve driving safety.

[0135] The HUD detection device may be provided with a data storage module for storing the detection analysis results; and a fault management module for generating the above-mentioned restart instruction and sending it to the HUD and ADAS assisted driving system when the analysis result shows that the HUD working state is abnormal.

[0136] The method of the embodiment of the present application is described in detail below with a specific implementation method.

[0137] like Figure 6 As shown, the HUD detection method includes the following steps:

[0138] Step S501, collecting image data corresponding to the image to be displayed from the HUD;

[0139] Step S502, extracting visual elements from the image data; executing step S507;

[0140] Step S503, collecting assisted driving information from the ADAS assisted driving system;

[0141] Step S504, extracting driving characteristics from the auxiliary driving information;

[0142] Step S505, determining whether the auxiliary driving information includes all the driving features that the corresponding driving scene should include; if yes, executing step S507, if not, executing step S506;

[0143] Step S506, determining that the ADAS assisted driving system is abnormal; executing step S514;

[0144] Step S507, determining whether the visual element matches the driving feature, if yes, executing step S508; if no, executing step S510;

[0145] Step S508, determining whether the visual element matches the preset visual element, if yes, executing step S509; if no, executing step S510;

[0146] Step S509, determining that the HUD works normally;

[0147] Step S510, collecting the driving assistance information collected by the HUD and processing the generated driving reminder information;

[0148] Step S511, determining whether the auxiliary driving information matches the driving reminder information; if so, executing step S512, if not, executing step S513;

[0149] Step S512, determining that a control module used by the HUD to generate image data is operating abnormally;

[0150] Step S513, determining that the HUD information collection module is operating abnormally;

[0151] Step S514, generating an analysis report;

[0152] Step S515: storing and / or submitting the analysis report.

[0153] Through the above HUD detection method, image data is directly obtained from the head-up display, without the need for a camera to take pictures and transmit pictures. During the head-up display detection process, the influence of factors such as the environment and the hardware of the detection equipment is avoided, and the detection in complex driving scenes can be adapted to improve the detection accuracy; the real-time acquisition of detection data is improved, and the hardware cost of the detection equipment is reduced. In addition, by comparing the assisted driving information collected from the ADAS assisted driving system and the driving reminder information generated from the HUD based on the assisted driving information, combined with the analysis of visual elements, it is not only possible to accurately verify whether the HUD is working properly, but also to further analyze the location of the abnormality, so as to facilitate testers or users to quickly and accurately locate abnormal equipment and promptly eliminate and solve the fault.

[0154] Taking the collision warning scenario as an example, the HUD detection process is described as follows:

[0155] The image data acquisition module 21 of the HUD detection device receives the image data generated by the HUD through WIFI. The visual element determination module 22 uses advanced image processing algorithms to pre-process, denoise, and enhance the data before reconstructing the image, and then extracts visual elements from the image through feature extraction algorithms, such as Figure 7 As shown, it includes relevant text warnings from the rear, the vehicle model, rear collision targets (including rear vehicles, pedestrians, cyclists, unknown obstacles, etc.), radar warning waves, and the vehicle's surrounding environment (lane lines, traffic lights, etc.).

[0156] The analysis module 23 automatically matches and processes the extracted visual elements with the visual elements in the preset image feature database through a feature matching algorithm. The visual elements in the preset image feature database should include all visual elements of the rear collision warning function. For example, if the rear collision target is a person, the extracted visual elements and the visual elements in the preset image feature database should both have the element of a person. Further, if the element features of the two are consistent, it proves that when the rear collision target is a person, the HUD can correctly display the element. If the extracted visual element does not match the human element in the preset image feature database or the element features of the two are inconsistent, it proves that when the rear collision warning occurs, an error will occur when the rear collision target output by the HUD is a person, and the HUD cannot display the element normally. It is necessary to combine signal detection to further determine whether the control unit or acquisition unit of the HUD is abnormal, or whether there is a problem with the signal transmitted by the ADAS assisted driving system.

[0157] The analysis module 23 processes and analyzes the signals collected and processed by the HUD data acquisition unit, and then uses a high-performance hardware platform and advanced signal processing algorithms to capture and analyze the signals sent by the ADAS system. The analyzed signal information includes the target type of the rear collision object such as pedestrians, two-wheeled vehicles, water barriers, cars, etc., the distance between the rear collision object and the own vehicle, the position of the own vehicle, and the surrounding scene reconstruction elements (lane lines, traffic lights, etc.).

[0158] The detection information collection module 24 collects the driving reminder information collected and processed by the HUD when the rear collision warning occurs and the auxiliary driving information issued by the ADAS auxiliary driving system. The analysis module 23 uses an automated comparison algorithm to automatically identify and classify different types of errors (such as signal loss, false alarm, and missed alarm). By comparing visual elements, it can only be determined that when the rear collision target is a person, the HUD cannot display the element normally. Through this step of detection, it can be determined whether the control unit or collection unit of the HUD is abnormal, or whether there is a problem with the signal transmitted by the ADAS system.

[0159] If there is a problem with the HUD control unit in processing the signal, then the signal collected and processed by the HUD information collection unit and the signal sent by the ADAS assisted driving system should contain information about people, and the information parsed from the signal should be consistent with the expected result; if the HUD information collection unit fails to accurately collect the rear collision warning rear collision object signal, then the signal sent by the ADAS assisted driving system contains information about people and the information transmitted is correct, while the signal collected and processed by the HUD information collection unit does not contain information about people or the related information is incorrect, and the HUD information collection unit needs to be optimized.

[0160] If the ADAS assisted driving system transmits an error, then the signal sent by the ADAS assisted driving system does not contain the information about people or the relevant information is incorrect, and the ADAS assisted driving system needs to further modify the algorithm related to the determination of the type of rear collision object. Finally, the evaluation results of the signal accuracy and the cause of the problem are passed to the analysis module.

[0161] The analysis module 23 first synthesizes the evaluation results of visual element performance and signal accuracy to generate a comprehensive HUD performance and reliability evaluation report, which should include the test process, evaluation results, log records, problem diagnosis and improvement suggestions. Then the test report is uploaded to the cloud in real time through the cloud storage and remote monitoring function for remote analysis and monitoring by testers. Finally, the test results are compared with the fault causes stored in the data storage and management module, and the test results and temporary solutions are fed back to the ADAS assisted driving system and HUD.

[0162] After the ADAS assisted driving system and HUD receive the signal that the HUD cannot normally receive and display all information related to the rear collision warning, the HUD is first restarted, and then the ADAS assisted driving system resends the rear collision warning related information to the HUD to try to solve the problem. If the problem cannot be solved, the fault cause is compared with the analysis module 23, and the user is prompted with temporary solutions to improve driving safety, such as voice reminders of collision warnings to remind users to pay attention.

[0163] The embodiment of the present application also provides a vehicle, including the HUD detection device of the above embodiment. The vehicle is provided with a HUD and an ADAS assisted driving system. The HUD 10 receives assisted driving information from the ADAS driving assistance system 30, generates a corresponding image and projects it onto the windshield in front of the driver's line of sight for display. The HUD detection device 20 collects image data for generating an image to be displayed from the HUD 10, determines the visual elements contained in the image to be displayed based on the image data, and analyzes the working status of the HUD 10 based on the visual elements.

[0164] The present application also provides an electronic device, such as Figure 8 As shown, the electronic device may include: a processor 1501 , a communication interface 1502 , a memory 1503 and a communication bus 1504 , wherein the processor 1501 , the communication interface 1502 , and the memory 1503 communicate with each other via the communication bus 1504 .

[0165] Memory 1503, used for storing computer programs;

[0166] The processor 1501 is used to implement the steps of the above method embodiment when executing the computer program stored in the memory 1503.

[0167] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method provided in any of the aforementioned method embodiments are implemented.

[0168] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0170] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0171] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A head-up display detection method, characterized in that: include: collecting first image data corresponding to the image to be displayed from the head-up display, and displaying the image to be displayed according to the first image data by the head-up display; Determining, according to the first image data, visual elements included in the image to be displayed; The operating status of the head-up display is analyzed based on the visual elements.

2. The method according to claim 1, characterized in that The method further comprises: Collecting assisted driving information from a driving assistance system, wherein the head-up display generates image data according to the assisted driving information; The analyzing the working state of the head-up display based on the visual element includes: extracting driving characteristics from the auxiliary driving information; The working state of the head-up display is analyzed according to the matching degree between the driving characteristics and the visual elements.

3. The method according to claim 1, characterized in that The determining, according to the first image data, the visual elements contained in the image to be displayed comprises: Analyzing the first image data to obtain element information corresponding to each of the visual elements; determining the visual element according to the element information; and / or, The determining, according to the first image data, the visual elements contained in the image to be displayed comprises: Perform image reconstruction based on the first image data to obtain an image to be detected; The visual element is extracted from the image to be detected.

4. The method according to claim 1, characterized in that The method further comprises: Collecting the assisted driving information sent by the driving assistance system and the driving reminder information generated after the head-up display collects and processes the information; wherein the head-up display generates the driving reminder information according to the received assisted driving information, and generates the first image data according to the driving reminder information; Comparing the auxiliary driving information with the driving reminder information; The working status of the head-up display is analyzed according to the information comparison result.

5. The method according to claim 1, characterized in that The analyzing the working state of the head-up display based on the visual element includes: determining a current driving scene according to the image data; Selecting a preset element corresponding to the current driving scene from a preset element feature library; Comparing the visual element with a preset element corresponding to the current driving scene; The working status of the head-up display is analyzed according to the element comparison result.

6. The method according to claim 1, characterized in that When the analysis result shows that the working state of the head-up display is abnormal, the method further includes: Sending a restart instruction to the head-up display and the auxiliary driving system, the head-up display restarts after receiving the restart instruction, and the auxiliary driving system resends driving assistance information to the head-up display after receiving the restart instruction; collecting second image data regenerated by the head-up display according to the driving assistance information resent by the driving assistance system; The operating status of the head-up display is analyzed according to the second image data.

7. A head-up display detection device, characterized in that: include: An image data acquisition module, used for acquiring first image data corresponding to the image to be displayed from the head-up display, and the head-up display displays the image to be displayed according to the first image data; A visual element determination module, configured to determine the visual elements included in the image to be displayed according to the first image data; An analysis module is used to analyze the working status of the head-up display based on the visual elements.

8. A vehicle, characterized in that: It comprises the head-up display detection device as described in claim 7.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images