ARHUD performance detection method and device based on image recognition

Abstract

The invention provides an ARHUD performance detection method and device based on image recognition, and belongs to the technical field of automobile driving. The ARHUD performance detection method based on image recognition comprises the following steps: S1, collecting superimposed image data in a preset range of a driver view, wherein the superimposed image data comprises driving image data and virtual image data; s2, recognizing the superimposed image data through an image recognition algorithm to respectively obtain a first target object in the driving image data and a second target object in the virtual image data; and S3, calculating the first target object and the second target object to obtain key performance indexes in the ARHUD system. According to the method, the first target object and the second target object are calculated, the four key performance indexes in the ARHUD system can be accurately and rapidly calculated, the detection precision and efficiency are greatly improved through automatic image recognition and processing procedures, and subjectivity and inconsistency in manual detection are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle driving, and more particularly, to a method and device for detecting the performance of an AR HUD based on image recognition. Background Art

[0002] Augmented Reality Head-Up Display (AR HUD) refers to a technology that combines virtual information with the driver's real vision and superimposes information such as navigation, vehicle speed, and warning information on the driver's line of sight. AR HUD can provide a more intuitive and convenient information display, improving driving safety and convenience.

[0003] With the wide application of Augmented Reality (AR) technology in the field of vehicle Head-Up Displays (HUDs), AR HUD has become one of the key technologies of Advanced Driver Assistance Systems (ADAS), capable of directly superimposing virtual information on the driver's line of sight. In the current market, the performance detection of AR HUD systems mainly relies on physical sensors and manual testing methods, which are highly subjective, resulting in inconsistent results and low detection accuracy. Most traditional AR HUD performance detection systems focus on the testing of basic display functions, such as the clarity, brightness, and color of virtual information. However, there are obvious limitations when detecting the dynamic performance of the detection system (such as latency, fitting degree, and display stability). Existing testing methods are difficult to simultaneously detect multiple performance indicators (such as latency, fitting degree, and stability), and multiple tests are required to cover all indicators, increasing the testing cost and time, and the complexity is relatively high.

[0004] Existing technologies usually have difficulty in automatically and efficiently and accurately evaluating the performance of AR HUD, especially when it comes to the synchronization and tracking of virtual images and the real environment, and cannot achieve precise dynamic response. In order to improve the user experience and safety of AR HUD, there is an urgent need for a solution to automatically detect its performance through image recognition technology to improve the accuracy, efficiency, and consistency of performance testing. Summary of the Invention

[0005] To solve the above problems, the embodiments of the present application provide a method and device for detecting the performance of an AR HUD based on image recognition.

[0006] In a first aspect, the embodiments of the present application provide a method for detecting the performance of an AR HUD based on image recognition, including the following steps:

[0007] S1: Collect superimposed image data within a preset range of the driver's field of view. The superimposed image data includes driving image data for reflecting the driving environment and virtual image data corresponding to the driving image data displayed by the AR HUD system;

[0008] S2: Identify the superimposed driving image data through an image recognition algorithm to separately obtain the first target object in the driving image data and the second target object in the virtual image data;

[0009] S3: Calculate the first target object and the second target object to obtain the key performance indicators in the AR HUD system. The key performance indicators include the AR image recognition latency, the AR image tracking latency, the AR image recognition target fitting degree, and the AR image recognition target display stability.

[0010] Preferably, step S2 specifically includes:

[0011] Preprocess the driving image data to obtain preprocessed data. The preprocessing at least includes image denoising;

[0012] Extract features from the preprocessed data to obtain the first target object and the second target object.

[0013] Preferably, obtaining the AR image recognition latency in the key performance indicators in step S3 specifically includes:

[0014] When the vehicle is in a static scene, respectively obtain the first acquisition timestamp corresponding to the first time the first target object is collected and the first display timestamp corresponding to the first time the second target object is displayed;

[0015] Calculate the first time difference between the first acquisition timestamp and the first display timestamp and use this first time difference as the AR image recognition latency.

[0016] Preferably, obtaining the AR image tracking latency in the key performance indicators in step S3 specifically includes:

[0017] When the vehicle is in a dynamic scene, obtain the first movement time when the first target object moves to the first preset position;

[0018] Obtain the first update time when the second target object in the virtual image data is updated to be correspondingly set with the first target object at the first preset position;

[0019] Calculate the second time difference between the first movement time and the first update time and use this second time difference as the AR image tracking latency.

[0020] Preferably, obtaining the AR image recognition target fitting degree in the key performance indicators in step S3 specifically includes:

[0021] Calculate the geometric deviation between the first target object and the second target object respectively through a geometric matching algorithm;

[0022] Obtain the fitting degree of the AR image recognition target based on geometric deviation calculation.

[0023] Preferably, in step S3, obtaining the display stability of the AR image recognition target in the key performance indicators specifically includes:

[0024] When the vehicle is in a dynamic scenario, obtain the first detection image of the second target object including the current moment in the superimposed image data, and obtain multiple first adjacent images adjacent to the first detection image of the current moment in the superimposed image data;

[0025] Obtain multiple display positions of the second target object in multiple first adjacent images respectively;

[0026] Based on the multiple display positions, calculate and obtain the jitter frequency of the second target object respectively;

[0027] Calculate and obtain the display stability score based on the jitter frequency.

[0028] Preferably, after step S3, it further includes:

[0029] S4: Calculate the performance index calculation value by calculating the AR image recognition delay time, the AR image following delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability according to the preset weighting value, and compare the performance index calculation value with the preset threshold to obtain the comprehensive performance score of the AR HUD system.

[0030] In a second aspect, an AR HUD performance detection device based on image recognition provided by an embodiment of the present application includes

[0031] A data acquisition module for collecting superimposed image data within a preset range of the driver's perspective, where the superimposed image data includes driving image data for reflecting the driving environment and virtual image data corresponding to the driving image data displayed by the AR HUD system;

[0032] A data processing module for obtaining the first target object in the superimposed image data and the second target object in the virtual image data through an image recognition algorithm;

[0033] A performance index calculation module for calculating the key performance indicators in the AR HUD system for the first target object and the second target object, where the key performance indicators include the AR image recognition delay time, the AR image following delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method provided in the first aspect or any possible implementation manner of the first aspect are implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the first aspect or any possible implementation manner of the first aspect are implemented.

[0036] The beneficial effects of the present invention are as follows:

[0037] The present application calculates the first target object and the second target object, and can accurately and quickly calculate four key performance indicators in the AR HUD system: AR image recognition delay time, AR image following delay time, AR image recognition target fitting degree, and AR image recognition target display stability. Through an automated image recognition and processing process, the accuracy and efficiency of detection are greatly improved, and the subjectivity and inconsistency in manual detection are avoided. The present application is not only applicable to static testing, but also can track and analyze the performance of the AR HUD in real time in a dynamic driving scenario, providing a scientific basis for system optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0039] Figure 1 FIG. is a schematic flow chart of a method for detecting AR HUD performance based on image recognition provided by an embodiment of the present application;

[0040] Figure 2 FIG. is a schematic structural diagram of a device for detecting AR HUD performance based on image recognition provided by an embodiment of the present application;

[0041] Figure 3 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

[0043] In the following description, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of the present application, and different embodiments can be replaced or combined. Therefore, the present application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments that contain one or more of all other possible combinations of A, B, C, and D, even though such embodiments may not be explicitly described in the following content.

[0044] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes can be made to the functions and arrangements of the described elements without departing from the scope of the content of the present application. Various processes or components can be appropriately omitted, substituted, or added to each example. For example, the described method can be performed in a different order than the described order, and various steps can be added, omitted, or combined. In addition, the features described for some examples can be combined into other examples.

[0045] Please refer to Figure 1 。 Figure 1 It is a schematic flowchart of a method for detecting the performance of an AR HUD based on image recognition provided by an embodiment of the present application. In the embodiment of the present application, the method includes the following steps:

[0046] S1: Collect superimposed image data within a preset range of the driver's perspective. The superimposed image data includes driving image data for reflecting the driving environment and virtual image data displayed by the AR HUD system;

[0047] S2: Identify the superimposed image data through an image recognition algorithm to respectively obtain a first target object in the driving image data and a second target object in the virtual image data;

[0048] S3: Calculate the first target object and the second target object to obtain key performance indicators in the AR HUD system. The key performance indicators include AR image recognition latency, AR image tracking latency, AR image recognition target fitting degree, and AR image recognition target display stability.

[0049] In the embodiments of the present application, the present application introduces an image recognition algorithm into the performance detection of AR HUD, and uses computer vision algorithms to automatically recognize, compare, and analyze virtual image data and real target objects. By capturing images in the driving scenario, the system can automatically detect the latency, fit, and display stability of virtual information, overcoming the limitations of relying on manual labor and external sensors in traditional detection. Through an automated image recognition and processing process, the accuracy and efficiency of detection are greatly improved, avoiding subjectivity and inconsistency in manual detection.

[0050] In the real-time examples of the present application, the present application is used for real-time detection of vehicles; when the AR HUD system projects virtual image data onto the front windshield, the superimposed image data within a preset range of the driver's perspective is captured in real time by a high-definition camera installed at the driver's perspective. The high-definition camera can collect the superimposed image data in front of the driver's perspective. The superimposed image data includes both driving image data reflecting the vehicle driving environment and virtual image data displayed by the AR HUD system on the front windshield. The virtual image data at each moment corresponds to the driving image data at that moment. The superimposed image data is video data, and all the video data is detected in the present application. In the embodiments of the present application, the latency time refers to the time difference experienced from when a target object or event is detected until the corresponding virtual information is displayed on the AR HUD. The length of the latency time directly affects the response speed of the system and the user experience. The fit refers to the degree of matching between the virtual image data displayed by the AR HUD and the actual object in terms of spatial position. It indicates whether the virtual information is accurately superimposed on the target object in the real scene. The higher the fit, the more precise the match between virtual and real. The display stability refers to whether the virtual information in the AR HUD can be displayed smoothly and without jitter in dynamic scenarios such as vehicle movement, road surface bumps, or other situations. Higher stability means fewer jitter and drift phenomena of the virtual image data during driving.

[0051] In the embodiments of the present application, image recognition refers to automatically detecting and identifying specific targets or feature points in an image through computer vision technology. The jitter frequency refers to the number of minor jitters that occur in the AR HUD virtual image data during the display process. The higher the jitter frequency, the greater the instability of the image, which will affect the driver's visual experience and attention. The frame rate refers to the number of image frames displayed by the system per second, usually measured in "frames per second" (FPS). The frame rate directly affects the smoothness and real-time performance of the image. In the AR HUD system, a higher frame rate can provide a smoother visual experience. Target tracking means whether the virtual image data can track real-world target objects in real time when the vehicle is moving or the perspective changes. It measures the dynamic response ability of the system. The image processing algorithm is a series of mathematical operations used to analyze, process, and transform image data. In the present invention, the image processing algorithm is used to calculate metrics such as the delay, fit, and stability between the AR HUD virtual image data and the real target.

[0052] In an implementable manner, step S2 specifically includes:

[0053] Preprocess the driving image data to obtain preprocessed data, and the preprocessing at least includes image denoising;

[0054] Extract features from the preprocessed data to obtain the first target object and the second target object respectively.

[0055] In the embodiments of the present application, the present invention captures and processes the driving image data from the driver's perspective through a camera, and captures and processes the virtual image data displayed by the AR HUD system. By capturing the superimposed image data of the external real-world scene and virtual information, using image recognition technology, automatically analyze the response and position changes of the virtual image data. In the embodiments of the present application, the driving image data of the driving environment is captured in real time through a high-definition camera installed in the driver's perspective, and at the same time, the virtual image data displayed by the AR HUD system is captured. The driving image data needs to be preprocessed to ensure image quality and feature availability. This step includes preprocessing steps such as image denoising, resolution adjustment, and feature extraction. Clean the noise in the image through image filtering algorithms (such as Gaussian filtering, mean filtering, etc.). The target object can contain multiple feature points. Adopt feature extraction algorithms such as SIFT and SURF to extract the edge points or key points of the target object, use the edge points or key points of the target object as the feature points of the target object, and find the target object (such as lane lines, traffic signs, etc.) in the driving image data according to the feature points of the target object to prepare for subsequent analysis. ROI (Region of Interest) extraction, extract the regions related to driving, such as roads, lane lines, pedestrians, etc., reduce the calculation amount, and improve the detection efficiency. Select one from multiple target objects as the first target object.

[0056] In the embodiment of the present application, the first target object may be a lane line or a road traffic sign, and the second target object may be a steering arrow sign. The high-definition camera is turned on to capture the superimposed image data within the preset range of the driver's perspective in real time. The superimposed image data is frame-processed to obtain a first image set. The first image set is screened to select a first screened image containing the first target object and the second target object; the first screened image is preprocessed to produce a first sample set; the Fast R-CNN network is multi-task trained through a first training set; the first sample set is input into the trained Fast R-CNN network, and after passing through several convolutional layers and pooling layers, a first feature map is obtained; the Selective Search algorithm is used to extract a number of first candidate boxes. According to the first mapping relationship between the first candidate boxes in the original image and the first feature map, the corresponding first feature boxes of each first candidate box are found in the first feature map, and each first feature box is pooled to a fixed size in the ROI pooling layer; the first feature box passes through a fully connected layer to obtain a first feature vector of a fixed size, and the first feature vector passes through its respective fully connected layer to obtain two first output vectors of classification score and window regression respectively; all the results are subjected to non-maximum suppression processing to obtain the first target object; the Selective Search algorithm is used to extract a number of second candidate boxes. According to the second mapping relationship between the second candidate boxes in the original image and the first feature map, the corresponding second feature boxes of each second candidate box are found in the first feature map, and each second feature box is pooled to a fixed size in the ROI pooling layer; the second feature box passes through a fully connected layer to obtain a second feature vector of a fixed size, and the second feature vector passes through its respective fully connected layer to obtain two second output vectors of classification score and window regression respectively; all the results are subjected to non-maximum suppression processing to obtain the second target object.

[0057] In the embodiment of the present application, the preprocessing of the first screened image to produce the first sample set may specifically include: in the first screened image, the first target area is taken out and scaled to a fixed size of 224×224. To enhance the contrast, the first target area is then subjected to contrast enhancement processing to obtain a first original training set, and the test set is processed in the same way; the first original training set is rotated [-12°, 12°] and scaled [0.4, 1.6], and then added to the original data set to form a new training set; in the new data set, samples equivalent to the number of the test set are randomly taken out to form a validation set, and the remaining samples form the final first training set.

[0058] In the embodiments of the present application, the Fast R-CNN network structure may include: 13 convolutional layers, 4 pooling layers, 1 ROI pooling layer, 2 fully connected layers, and two parallel layers. In the ROI pooling layer, each feature box is pooled into a fixed size of 7×7. The fully connected output of the multi-task training Fast R-CNN network includes two branches: the cls_score layer and the bbox_pred layer. The cls_score layer is used for classification, and the bbox_pred layer is used to adjust the position of the candidate box. When the feature vector passes through its respective fully connected layer, it is accelerated by singular value decomposition (SVD) to obtain two output vectors, namely the classification score of Softmax and the bounding-box window regression. For the two branches of the fully connected output, the classification layer and the regression layer of the output layer are trained using the stochastic gradient descent method until the loss functions of classification and regression converge. The step of performing non-maximum suppression processing on all results specifically includes: according to the two output branches, using the window scores to perform non-maximum suppression on each type of object to remove overlapping candidate boxes, and finally obtaining the window with the highest score after regression correction in each category.

[0059] In an implementable manner, the specific steps of obtaining the AR image recognition delay time in key performance indicators in step S3 include:

[0060] When the vehicle is in a static scenario, respectively obtain the first acquisition timestamp corresponding to the first time the first target object is collected and the first display timestamp corresponding to the first time the second target object is displayed;

[0061] Calculate and obtain the first time difference between the first acquisition timestamp and the first display timestamp, and use this first time difference as the AR image recognition delay time.

[0062] In the embodiments of the present application, first, detect and identify objects (such as pedestrians, vehicles, road signs, etc.) in the driving scenario, and record the first acquisition timestamp when the object first appears in the field of view, that is, detect and record the first acquisition timestamp when the first target object in the real scenario first appears. Subsequently, detect the first display timestamp of the corresponding virtual information (such as navigation marks or warning symbols) in the virtual image data in the AR HUD. By comparing these two timestamps, calculate the AR image recognition delay time, that is, calculate the first time difference between the first acquisition timestamp and the first display timestamp and use the first time difference as the AR image recognition delay time. Respectively obtain the first acquisition time T real , the first display timestamp T display , and based on the formula T delay = T display - T real calculate and obtain the AR image recognition delay time Tdelay 。

[0063] In an implementable manner, obtaining the AR image following delay time in step S3 specifically includes:

[0064] Obtaining the first movement time when the first target object moves to the first preset position when the vehicle is in a dynamic scenario;

[0065] Obtaining the first update time when the virtual image data updates the second target object to be set corresponding to the first target object at the first preset position;

[0066] Calculating and obtaining the second time difference between the first movement time and the first update time, and using this second time difference as the AR image following delay time.

[0067] In the embodiments of the present application, when the vehicle is moving or the driver changes the viewing angle, the system can monitor in real time whether the virtual image data can accurately and timely follow the target object. By continuously capturing multiple frames of images, the system calculates the time difference between the change in the object position and the display of the virtual image data to obtain the AR image following delay time. It can detect whether the virtual information follows the object change in a timely manner, ensuring that the driver can obtain accurate information prompts in real time. In a dynamic scenario, monitor the movement trajectory of the object and the response time of the virtual image data. Record the first movement time corresponding to when the first target object moves to the first preset position, and at the same time record the first update time when the second target object is updated to be set corresponding to the first target object at the first preset position, calculate and obtain the second time difference between the first movement time and the first update time, and use this second time difference as the AR image following delay time. The AR image following delay time in the present application can reflect whether the virtual information can follow the physical target in real time. The function of obtaining the AR image following delay time is to ensure that the driver obtains accurate information feedback in a dynamic environment. Respectively obtain the first movement time T motion 、the first update time T update , and based on the formula T follow =T update -T motion calculate and obtain the AR image following delay time T follow 。

[0068] In the embodiments of the present application, the present application adopts the timestamp synchronization technology, which can accurately measure the AR image recognition delay time from when the real target object is detected by the camera to the display of the virtual image data. Through the image capture and comparison technology, the response of the virtual image data is monitored in real time to ensure the accurate calculation of the AR image following delay time. Compared with the traditional manual recording or physical sensor measurement, the precision of the present invention in time synchronization and delay detection is significantly improved, with extremely small errors, and can meet the response speed requirements in a high-speed driving environment.

[0069] In an embodiment of the present application, the real-time speed of the vehicle is obtained through a speed sensor, and the steering angle of the vehicle is obtained through an angle sensor; the first method for determining whether the vehicle is in a dynamic scenario: obtain the real-time speed of the vehicle, determine that the vehicle is in a dynamic scenario when the real-time speed is greater than a first preset speed, and determine that the vehicle is in a static scenario when the real-time speed is not greater than the first preset speed; the second method for determining whether the vehicle is in a dynamic scenario: obtain the real-time speed and steering angle of the vehicle, determine that the vehicle is in a dynamic scenario when the real-time speed is greater than the first preset speed and the steering angle is greater than a first preset angle, and determine that the vehicle is in a static scenario when the real-time speed is not greater than the first preset speed and the steering angle is not greater than the first preset angle; the first preset speed can be 0, and the steering angle can be 0 degrees.

[0070] In an implementable manner, obtaining the AR image recognition target fitting degree in the key performance indicators in step S3 specifically includes:

[0071] Calculate the geometric deviation between the first target object and the second target object through a geometric matching algorithm;

[0072] Calculate the AR image recognition target fitting degree based on the geometric deviation.

[0073] In an embodiment of the present application, the geometric matching algorithm is an image recognition algorithm. The present invention uses the image recognition algorithm to compare the fitting degree of the virtual image data and the real object in the geometric position. Through the image matching technology, analyze the spatial position, size and other parameters of the virtual object and the real object, calculate the offset between the two, and obtain the AR image recognition target fitting degree. The present invention automatically analyzes the relative position, size and angle between the virtual image data and the real target object through the geometric matching algorithm, and calculates the AR image recognition target fitting degree. Compared with the traditional manual comparison or simple geometric calculation, the algorithm of the present invention can accurately and automatically calculate the fitting degree, especially maintaining high precision in a dynamic scenario, and can ensure the accurate matching of virtual information and the real scenario under different scenarios and driving conditions, improving the user experience and safety.

[0074] In an embodiment of the present application, calculating the geometric deviation between the first target object and the second target through the geometric matching algorithm may specifically include: obtaining a frame of superimposed image data including the first target object and the second target object, respectively obtaining the first coordinate and the second coordinate of the first target object and the second target object in the corresponding camera coordinate system; determining the third coordinate of the camera in the world coordinate system based on the encoder information of the guide rail where the camera is installed, and obtaining the coordinate conversion relationship from the camera coordinate system to a preset world coordinate system based on the third coordinate; determining the first position coordinate (x of the first target object in the preset world coordinate system according to the first coordinate and the coordinate conversion relationshipreal , y real ), determine the second position coordinates (x virtual , y virtual ) of the second target object in the preset world coordinate system according to the second coordinate and the coordinate conversion relationship, and calculate and obtain the geometric deviation (x real - x virtual , y real - y virtual ) based on the first position coordinates and the second position coordinates.

[0075] In the embodiments of the present application, calculating and obtaining the AR image recognition target fitting degree based on the geometric deviation may specifically include: based on the formula calculate and obtain the AR image recognition target fitting degree D align .

[0076] In the embodiments of the present application, the role of obtaining the AR image recognition target fitting degree is to ensure the precise alignment of the virtual image data in the AR HUD with the real object and improve the display accuracy.

[0077] In one implementable manner, obtaining the AR image recognition target display stability in step S3 specifically includes:

[0078] When the vehicle is in a dynamic scenario, obtain the first detection image of the second target object at the current moment included in the superimposed image data, and obtain multiple first adjacent images adjacent to the first detection image at the current moment in the superimposed image data;

[0079] Obtain the multiple display positions of the second target object in multiple first adjacent images respectively;

[0080] Based on the multiple display positions, calculate and obtain the jitter frequency of the second target object respectively;

[0081] Calculate and obtain the display stability score based on the jitter frequency.

[0082] In the embodiments of the present application, by monitoring the jitter or drift of consecutive multiple frames of images in the virtual image data, analyze the display stability of the virtual information. In a dynamic scenario, the system analyzes the relative position change between the virtual image data and the background, calculates the jitter frequency, and generates the display stability score. The role of obtaining the AR image recognition target display stability is to quantify the display stability of the virtual image data in a dynamic environment and ensure that the driver obtains a stable visual experience during driving. Obtain the jitter frequency F jitter , and calculate and obtain the display stability score S based on the formula S stability = 1 / (1 + F jitter ) stability .

[0083] In the embodiments of the present application, by continuously collecting multiple frames of images, the present application can detect the jitter frequency and jitter amplitude of virtual image data in real time under conditions such as vehicle bumpiness, turning, or acceleration, and automatically generate a display stability score. This quantitative analysis overcomes the traditional detection method that relies on visual observation and can accurately measure the stability of virtual image data in a dynamic environment. Through the quantitative stability score, the performance of virtual information in different driving environments can be more clearly understood, which helps to optimize the display algorithm of the AR HUD system, reduce jitter, and improve the smoothness of the display.

[0084] In the embodiments of the present application, the current moment of the second target object is obtained. The first detection image including the second target object at the current moment is obtained from the superimposed image data. Multiple detection moments continuously set before and after the current moment are obtained. Multiple first adjacent images including the second target object at the multiple detection moments are obtained from the superimposed image data, so as to obtain multiple first adjacent images adjacent to the first detection image in the superimposed image data; a reference coordinate system is established for each of the multiple first adjacent images, the same preset position on the multiple first adjacent images is used as a reference point, the second target object in the first detection image and the multiple first adjacent images is respectively subjected to feature extraction, the center points of the second target object in the multiple first adjacent images are obtained, the first detection coordinates of the second target object in the reference coordinate system are obtained based on the center points and the reference point, and the multiple display positions of the second target object in the multiple first adjacent images are obtained according to the first detection coordinates; since the reference coordinate systems and reference points of the multiple first adjacent images are the same, based on the first detection coordinates of the second target object in each of the multiple first adjacent images, the multiple display positions of the second target object in the multiple first adjacent images are plotted in a reference coordinate system, and the multiple display positions are numbered in the order of time sequence. The position change amplitude between two display positions is calculated based on the first detection coordinates of the two display positions, and the position change amplitude is used as the jitter amplitude of the second target object; if the jitter amplitude is greater than a preset value, the detection moments corresponding to the two first adjacent images are respectively obtained, the detection time difference is obtained based on the detection moments of the two first adjacent images, and the jitter frequency is obtained based on the detection time difference; if the jitter amplitude is not greater than the preset value, two other first adjacent images outside the range of the two first adjacent images used to calculate the current jitter amplitude are obtained, and the jitter amplitude is continuously calculated based on the new two first adjacent images until the calculated jitter amplitude is greater than the preset value.

[0085] In the embodiments of the present application, when obtaining multiple first adjacent images adjacent to the first detection image at the current moment in the superimposed image data, at least two first adjacent images at one second before and after the current moment can be selected.

[0086] In an implementable manner, after step S3, the following is further included:

[0087] S4: Calculate the performance metric calculation value by calculating the AR image recognition delay time, the AR image tracking delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability according to the preset weighting values, and compare the performance metric calculation value with the preset threshold to obtain the comprehensive performance score of the AR HUD system.

[0088] In the embodiment of the present application, according to the calculation results of various indicators, a comprehensive evaluation and analysis are carried out. Multiple performance indicators such as delay time, fitting degree, and display stability are weighted and summarized, and compared with the preset threshold to obtain the overall performance of the AR HUD system. The data analysis module uses the topsis and AHP algorithms, comprehensively considers the importance of different indicators, and generates the final comprehensive performance score. A visualization tool is used to generate real-time analysis charts to help developers intuitively understand the system performance. The comprehensive performance score and analysis data are presented in the form of charts and reports, supporting export to multiple formats (such as CSV, PDF, Excel, etc.), and providing multi-dimensional performance analysis reports.

[0089] In the embodiment of the present application, through a unified image recognition and processing platform, the present invention can simultaneously detect multiple key performance indicators, including the AR image recognition delay time, the AR image tracking delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability, and achieve real-time analysis in dynamic driving scenarios. This multi-dimensional synchronous detection greatly improves the test coverage and accuracy. The present application can continuously and real-time detect and analyze the performance of the AR HUD system in complex driving environments, provide multi-index comprehensive evaluation, and help optimize the system performance.

[0090] In the embodiment of the present application, the detection method of the present invention is fully automated from image acquisition, processing, analysis to result generation, without manual intervention. The system can independently complete complex multi-dimensional performance evaluations and generate detailed analysis reports. The automated detection in the present application greatly improves the detection efficiency and reduces human errors, especially suitable for large-scale tests and the optimization process of system development.

[0091] In the embodiment of the present application, the present application has the following advantages:

[0092] 1. Automated detection: Through image recognition technology, the detection of key performance indicators of AR HUD is automated, reducing manual operation errors and improving detection efficiency.

[0093] 2. Strong adaptability to multiple scenarios: It can detect the delay, fitting degree, and stability of virtual image data in complex and changing driving scenarios in real time, covering a wide range of test scenarios.

[0094] 3. Improve accuracy and consistency: The introduction of image recognition and processing technology ensures high accuracy and consistency of test results, eliminating the bias of manual subjective judgment.

[0095] 4. Optimize the fast feedback system: By automatically generating detailed performance reports, it helps developers quickly locate problems and optimize the design and debugging of the AR HUD system.

[0096] The following will combine with the attached Figure 2 ... to introduce in detail a performance detection device for AR HUD based on image recognition provided by the embodiments of the present application. It should be noted that the attached Figure 2 ... shows a performance detection device for AR HUD based on image recognition, which is used to execute the method of the embodiments of the present application. For the convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the embodiments shown in the present application Figure 1 ... Figure 1 ...

[0097] Please refer to Figure 2 ... Figure 2 ... is a schematic structural diagram of a performance detection device for AR HUD based on image recognition provided by the embodiments of the present application. As shown in Figure 2 ... the device includes

[0098] A data acquisition module 201, used to collect superimposed image data within a preset range of the driver's perspective. The superimposed image data includes driving image data reflecting the driving environment and virtual image data corresponding to the driving image data displayed by the AR HUD system;

[0099] A data processing module 202, used to identify the superimposed image data through an image recognition algorithm to respectively obtain a first target object in the driving image data and a second target object in the virtual image data;

[0100] A performance index calculation module 203, used to calculate the first target object and the second target object to obtain key performance indexes in the AR HUD system. The key performance indexes include AR image recognition delay time, AR image following delay time, AR image recognition target fitting degree, and AR image recognition target display stability.

[0101] In the embodiments of the present application, the data processing module 202 includes

[0102] A preprocessing unit, used to preprocess the superimposed image data to obtain preprocessed data. The preprocessing at least includes image denoising;

[0103] A feature extraction unit, used to extract features from the preprocessed data to respectively obtain the first target object and the second target object.

[0104] In the embodiment of the present application, the performance index calculation module 203 includes an identification delay time acquisition unit, which is used to respectively acquire a first acquisition timestamp corresponding to the first target object when it is first collected and a first display timestamp corresponding to the second target object when it is first displayed when the vehicle is in a static scenario; calculate and obtain a first time difference between the first acquisition timestamp and the first display timestamp, and use this first time difference as the AR image recognition delay time.

[0105] In the embodiment of the present application, the performance index calculation module 203 further includes a following delay time acquisition unit, which is used to acquire a first movement time when the first target object moves to a first preset position when the vehicle is in a dynamic scenario; acquire a first update time when the second target object is updated to be correspondingly set with the first target object at the first preset position in the superimposed image data; calculate and obtain a second time difference between the first movement time and the first update time, and use this second time difference as the AR image following delay time.

[0106] In the embodiment of the present application, the performance index calculation module 203 further includes a fitting degree acquisition unit, which is used to calculate and obtain the geometric deviation between the first target object and the second target object through a geometric matching algorithm; calculate and obtain the AR image recognition target fitting degree based on the geometric deviation.

[0107] In the embodiment of the present application, the performance index calculation module 203 further includes a display stability acquisition unit, which is used to acquire multiple first adjacent images adjacent to the second target object at the current moment in the superimposed image data when the vehicle is in a dynamic scenario; acquire multiple display positions of the first target object in the multiple first adjacent images; calculate and obtain the jitter frequency of the second target object based on the multiple display positions; calculate and obtain the display stability score based on the jitter frequency.

[0108] In the embodiment of the present application, there is also a score acquisition module, which is used to calculate and obtain a performance index calculation value by calculating the AR image recognition delay time, the AR image following delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability according to a preset weighting value, and compare the performance index calculation value with a preset threshold to obtain the comprehensive performance score of the AR HUD system.

[0109] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented by means of software and / or hardware. The "units", "modules", and "parts" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0110] Each processing unit and / or module of the embodiments of the present application can be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or can be implemented by software that executes the functions described in the embodiments of the present application.

[0111] See Figure 3 , which shows a schematic structural diagram of an electronic device related to the embodiments of the present application. This electronic device can be used to implement Figure 1 the method in the illustrated embodiment. As Figure 3 shown, the electronic device 300 may include: at least one central processing unit 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0112] Among them, the communication bus 302 is used to implement connection communication between these components.

[0113] Among them, the user interface 303 may include a display screen (Display), a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0114] Among them, the network interface 304 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0115] Among them, the central processing unit 301 may include one or more processing cores. The central processing unit 301 connects various parts within the entire electronic device 300 through various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling the data stored in the memory 305, it executes various functions of the terminal 300 and processes data. Optionally, the central processing unit 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The central processing unit 301 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the central processing unit 301 and may be implemented separately by a single chip.

[0116] Among them, the memory 305 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store the data involved in the above-mentioned method embodiments. Optionally, the memory 305 may also be at least one storage device located far from the aforementioned central processing unit 301. As Figure 3 shown, the memory 305, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0117] In Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user to obtain the data input by the user; and the central processing unit 301 can be used to call an application program of an AR HUD performance detection method stored in the memory 305 and specifically perform the following operations:

[0118] S1: Collect superimposed image data within a preset range of the driver's perspective. The superimposed image data includes driving image data for reflecting the driving environment and virtual image data corresponding to the driving image data displayed by the AR HUD system.

[0119] S2: Identify the superimposed image data through an image recognition algorithm to respectively obtain a first target object in the driving image data and a second target object in the virtual image data.

[0120] S3: Calculate the first target object and the second target object to obtain key performance indicators in the AR HUD system. The key performance indicators include AR image recognition latency, AR image following latency, AR image recognition target fitting degree, and AR image recognition target display stability.

[0121] This application also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are implemented. Among them, the computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, micro drives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nano-systems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0122] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0123] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0125] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0127] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned memory includes: USB flash drives, read-only memory (ROM), random access memory (RAM), mobile hard disks, magnetic disks, or optical discs, etc., which can store program codes.

[0128] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable memory. The memory can include: flash drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs, etc.

[0129] The foregoing are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereby. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not described in the present disclosure. The specification and embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An AR HUD performance detection method based on image recognition, characterized in that, The method includes the following steps: S1: Collect superimposed image data within a preset range of the driver's perspective. The superimposed image data includes driving image data for reflecting the driving environment and virtual image data corresponding to the driving image data displayed by the AR HUD system. S2: Identify the superimposed image data through an image recognition algorithm to respectively obtain a first target object in the driving image data and a second target object in the virtual image data. S3: Calculate the first target object and the second target object to obtain key performance indicators in the AR HUD system. The key performance indicators include AR image recognition latency, AR image following latency, AR image recognition target fitting degree, and AR image recognition target display stability.

2. The method for detecting the performance of an AR HUD based on image recognition according to claim 1, wherein Specifically, step S2 includes: Preprocess the superimposed image data to obtain preprocessed data. The preprocessing at least includes image denoising. Extract features from the preprocessed data to respectively obtain the first target object and the second target object.

3. A method for detecting the performance of an AR HUD based on image recognition according to claim 1 or 2, characterized in that, Specifically, for obtaining the AR image recognition latency in the key performance indicators in step S3: When the vehicle is in a static scenario, respectively obtain a first acquisition timestamp corresponding to the first time the first target object is collected and a first display timestamp corresponding to the first time the second target object is displayed. Calculate the first time difference between the first acquisition timestamp and the first display timestamp and use this first time difference as the AR image recognition latency.

4. The method for detecting the performance of an AR HUD based on image recognition according to claim 1 or 2, characterized in that, Specifically, for obtaining the AR image following latency in the key performance indicators in step S3: When the vehicle is in a dynamic scenario, obtain a first movement time when the first target object moves to a first preset position. Obtain a first update time when the second target object in the virtual image data is updated to be set corresponding to the first target object at the first preset position. Calculate the second time difference between the first movement time and the first update time and use this second time difference as the AR image following latency.

5. The AR HUD performance detection method based on image recognition according to claim 1 or 2, characterized in that, Specifically, for obtaining the AR image recognition target fitting degree in the key performance indicators in step S3: Calculate the geometric deviation between the first target object and the second target object through a geometric matching algorithm. Calculate the AR image recognition target fitting degree based on the geometric deviation.

6. The AR HUD performance detection method based on image recognition according to claim 1 or 2, characterized in that Specifically, for obtaining the AR image recognition target display stability in the key performance indicators in step S3: When the vehicle is in a dynamic scenario, obtain a first detection image including the second target object at the current moment in the superimposed image data, and obtain multiple first adjacent images adjacent to the first detection image in the superimposed image data. Obtain multiple display positions of the second target object in the multiple first adjacent images and the first detection image respectively. Calculate the jitter frequency of the second target object respectively based on the multiple display positions. Calculate the display stability score based on the jitter frequency.

7. The method for detecting the performance of an AR HUD based on image recognition according to claim 1 or 2, characterized in that, After step S3, it further includes: S4: Calculate the calculated value of the performance index by calculating the AR image recognition delay time, the AR image following delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability according to the preset weighting value, and compare the calculated value of the performance index with the preset threshold to obtain the comprehensive performance score of the AR HUD system.

8. An AR HUD performance detection device based on image recognition, characterized in that: including a data acquisition module for collecting superimposed image data within a preset range of the driver's perspective, the superimposed image data including driving image data for reflecting the driving environment and virtual image data corresponding to the driving image data displayed by the AR HUD system; a data processing module for identifying the superimposed image data through an image recognition algorithm to respectively obtain a first target object in the driving image data and a second target object in the virtual image data; a performance index calculation module for calculating the first target object and the second target object to obtain key performance indexes in the AR HUD system, the key performance indexes including the AR image recognition delay time, the AR image following delay time, the AR image recognition target fitting degree, and the AR image recognition target display stability.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1-7 are implemented.

Images