Method for determining display failure of intelligent cabin and failure detection device

Abstract

The present application relates to the technical field of display fault detection, and more particularly to a method for determining display faults of an intelligent cockpit, a fault detection device, an intelligent cockpit test bench and a computer storage medium. The method comprises: A, receiving real-time images collected by a camera, wherein the real-time images comprise original display images of one or more displays in an intelligent cockpit test bench; B, simultaneously starting multiple image processing algorithms through multi-threading calls to perform fault identification on each original display image to determine whether each display has a display fault and the type of the display fault, wherein each image processing algorithm in the multiple image processing algorithms is used to identify a different type of display fault; and C, determining whether to store the real-time images based on the fault identification results.

Description

Technical Field

[0001] The present invention relates to the technical field of display fault detection, and more particularly to a method for determining a display fault of an intelligent cockpit, a fault detection device, an intelligent cockpit test bench, and a computer storage medium. Background Art

[0002] With the development of automotive smart cockpit technology, smart cockpits are gradually becoming the carrier of user experience. Since the functions of smart cockpits are large and complex, it is usually necessary to control the rendering of multiple screens in the car at the same time, so there is a possibility of display failures (for example, black screens). According to the probability of occurrence, display failures can be divided into inevitable problems and occasional problems. Among them, occasional problems have uncertain timing of occurrence, are easy to miss the first scene after occurrence, and even after repair, it is difficult to accurately judge the effectiveness of the repair. Therefore, it is necessary to verify the display failure through automated stress testing. However, the currently disclosed automated stress testing methods are difficult to cover a variety of display problems while ensuring real-time performance. Summary of the Invention

[0003] To address or at least alleviate one or more of the above issues, the following technical solutions are provided. Embodiments of the present invention provide a method, fault detection device, smart cockpit test bench, and computer storage medium for identifying display faults in a smart cockpit. These methods can process camera-captured images in real time during automated testing of the smart cockpit to accurately locate various display faults and promptly preserve the initial scene, providing important data analysis support for further diagnosis.

[0004] According to a first aspect of the present invention, a method for determining display faults in a smart cockpit is provided, comprising: A. receiving a real-time image captured by a camera, wherein the real-time image includes original display images of one or more displays within a smart cockpit test bench; B. simultaneously starting multiple image processing algorithms through multi-threaded calls to perform fault identification on each original display image to determine whether there is a display fault on each display and the type of display fault, wherein each of the multiple image processing algorithms is used to identify different types of display faults; and C. determining whether to store the real-time image based on the fault identification result.

[0005] As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, step A further includes one or more of the following: waking up the smart cockpit test bench in response to receiving a wake-up signal; loading a baseline image as a reference benchmark for the one or more displays; denoising the real-time image using a Gaussian low-pass filtering algorithm; and extracting the original display image of one or more displays from the real-time image using an image capture algorithm.

[0006] As an alternative or supplement to the above scheme, in a method according to one embodiment of the present invention, the multiple image processing algorithms include one or more of the following: a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snow screen type faults, a third image processing algorithm for identifying screen stuck type faults, and a fourth image processing algorithm for identifying screen jitter type faults.

[0007] As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the multiple image processing algorithms include a first image processing algorithm for identifying black screen type faults, and step B includes: performing grayscale processing on the original display image using a grayscale processing function; binarizing the grayscale processed image using a binarization function; counting the number of non-zero pixels in the binarized image using a non-zero counting function; comparing the number of non-zero pixels with a first threshold, and determining whether the display corresponding to the original display image has a black screen type fault based on the comparison result.

[0008] As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the multiple image processing algorithms include a second image processing algorithm for identifying snow screen type faults, and step B includes: using a feature extraction function to detect feature points of the original display image and the benchmark image serving as a reference benchmark and calculate their feature descriptors; using an image matching function to match the feature descriptors of the original display image and the feature descriptors of the benchmark image to generate matched feature point pairs; calculating the number of feature point pairs in the matched feature point pairs whose feature point similarity is greater than or equal to a second threshold; comparing the number with a third threshold, and determining whether the display corresponding to the original display image has a snow screen type fault based on the comparison result.

[0009] As an alternative or supplement to the above scheme, in a method according to one embodiment of the present invention, step A includes changing the CAN bus signal input to the smart cockpit test bench after loading a benchmark image as a reference benchmark, and receiving the real-time image captured by the camera after a first time period, and the multiple image processing algorithms include a third image processing algorithm for identifying screen jamming type faults.

[0010] As an alternative or supplement to the above solutions, in the method according to an embodiment of the present application, step B comprises: determining whether the display corresponding to the original display image is at risk of sticking based on the number of feature point pairs in the feature point pairs of the original display image and the reference image that have a similarity greater than or equal to a fourth threshold value; if it is determined that there is a risk of sticking, performing grayscale processing and binarization processing on the original display image and the reference image, and counting the number of non-zero pixels in the images after grayscale and binarization processing; and if the absolute value of the difference between the number of non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold value, determining that the display has a screen sticking type fault.

[0011] As an alternative or supplement to the above solutions, in the method according to an embodiment of the present application, the plurality of image processing algorithms comprises a fourth image processing algorithm for identifying a screen shaking type fault, and step B comprises: storing original display images acquired during a second time period and performing frame sequencing thereon; performing shaking comparison on original display images with adjacent frame numbers, and acquiring the total number of shaking images; comparing the total number of shaking images with a sixth threshold value, and determining whether the display corresponding to the original display image has a screen shaking type fault based on the comparison result.

[0012] As an alternative or supplement to the above solutions, in the method according to an embodiment of the present application, step B further comprises: monitoring the running state of each image processing thread in real time; and adjusting the priority of the plurality of image processing threads based on the running state.

[0013] As an alternative or supplement to the above solutions, in the method according to an embodiment of the present application, step C comprises: if it is determined that one or more of the displays have a display fault, storing the real-time images; and if it is determined that the displays do not have a display fault, controlling the intelligent cockpit test bench to power off, and sending a wake-up signal for the intelligent cockpit test bench after a preset time interval.

[0014] According to a second aspect of the present application, there is provided a fault detection device, comprising: a memory configured to store instructions; and a processor configured to execute the instructions to perform any one of the methods according to the first aspect of the present application.

[0015] According to a third aspect of the present application, there is provided an intelligent cockpit test bench, comprising: a camera for acquiring original display images of one or more displays in the intelligent cockpit test bench; and any one of the fault detection devices according to the second aspect of the present application.

[0016] According to a fourth aspect of the present invention, there is provided a computer storage medium, the computer storage medium comprising instructions, the instructions being used to execute any one of the methods according to the first aspect of the present invention when run.

[0017] According to one or more embodiments of the present invention, a solution for identifying display faults in a smart cockpit captures raw display images from each display in real time and uses multiple image processing algorithms to identify faults in each raw display image. This ensures real-time image recognition and processing while accurately locating various display faults and promptly preserving the initial scene, thereby providing important data analysis support for further diagnosis. Furthermore, according to one or more embodiments of the present invention, a solution for identifying display faults in a smart cockpit utilizes multiple image processing algorithms simultaneously through multi-threaded invocation, avoiding the inefficiency of single-threaded processing and enabling targeted identification and location of various display faults, thereby improving the accuracy of identifying and repairing smart cockpit display faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or other aspects and advantages of the present invention will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar elements are represented by the same reference numerals. In the accompanying drawings:

[0019] Figure 1 is a schematic flow chart of a method 10 for determining a display fault of a smart cockpit according to one or more embodiments of the present invention; and

[0020] Figure 2 FIG2 is a schematic flow chart of a method 20 for determining a display fault of a smart cockpit according to one or more embodiments of the present invention. DETAILED DESCRIPTION

[0021] The description of the following specific embodiments is merely exemplary in nature and is not intended to limit the disclosed technology or the application and use of the disclosed technology. In addition, there is no intention to be bound by any express or implied theory presented in the foregoing technical field, background technology or the following specific embodiments.

[0022] In the following detailed description of the embodiments, numerous specific details are set forth to provide a more thorough understanding of the disclosed technology. However, it will be apparent to one of ordinary skill in the art that the disclosed technology can be practiced without these specific details. In other instances, well-known features are not described in detail to avoid unnecessarily complicating the description.

[0023] Terms such as "comprising" and "including" indicate that in addition to the units and steps directly and clearly stated in the specification, the technical solution of the present invention does not exclude the situation where there are other units and steps that are not directly or clearly stated. Terms such as "first" and "second" do not indicate the order of units in terms of time, space, size, etc., but are only used to distinguish between the units. In this specification, the term "vehicle" or other similar terms include general motor vehicles, such as passenger cars (including sports utility vehicles, buses, trucks, etc.), various commercial vehicles, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, etc. A hybrid vehicle is a vehicle with two or more power sources, such as gasoline-powered and electric vehicles.

[0024] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[0025] Figure 1 This is a schematic flowchart of a method 10 for determining a display fault in a smart cockpit according to one or more embodiments of the present invention. It should be noted that the step names mentioned above (and those mentioned below) are merely used to distinguish and facilitate step reference and do not represent a sequential order between the steps. The flowcharts, including those in the accompanying figures, are merely examples of how to execute this method. Unless there is an obvious conflict, the steps may be executed in various orders or simultaneously.

[0026] like Figure 1 As shown, in step S110 , a real-time image captured by a camera is received, wherein the real-time image includes original display images of one or more displays in the smart cockpit test bench.

[0027] As the functions of the smart cockpit become richer, one or more displays may be arranged in the smart cockpit, such as the dashboard in front of the driver, the head-up display (HUD) and the center console screen, as well as the entertainment screen in front of the co-pilot and the rear row. The smart cockpit host is connected to various buses and information transmission components on the vehicle, completes the reception, transmission and preprocessing of various communication signals (for example, controller area network (CAN) bus signals), and renders the display images of one or more displays based on the above communication signals. It is understandable that one or more displays are correspondingly provided in the test bench for functional testing of the smart cockpit, and a camera (for example, an industrial camera) is also provided for capturing the original display images of one or more displays. It should be understood that in the embodiments of the present invention, cameras, video cameras, cameras, etc. all refer to devices that can obtain images or images within the coverage area, and their meanings are similar and interchangeable, and the present invention does not limit this.

[0028] Exemplarily, method 10 is implemented based on OpenCV. OpenCV is an open source software library that can run on a variety of operating systems and supports Python language interface. It has the advantages of lightweight and efficient functions and provides support for a large number of image recognition library functions. The intelligent cockpit test bench uses Python scripts to automatically control the operation, and Python calls the library functions through the OpenCV compatible interface cv2. Exemplarily, the following core library functions can be called through OpenCV, such as Gaussian filtering function gaussian(), image capture function VideoCapture(), image loading function imread(), grayscale processing function cvColor(), binarization function threshold(), non-zero count function countNonZero(), object creation function xfeatures2d.SIFT_create(), feature extraction function detectAndCompute(), image matching functions FlannBasedMatcher() and knnMatch(), etc.

[0029] Optionally, in method 10, a power-on cycle interval is set for the smart cockpit test bench, for example, a wake-up signal is sent to the smart cockpit test bench every 5 minutes. In response to receiving the wake-up signal, the smart cockpit test bench turns on one or more displays and cameras within the smart cockpit test bench. Optionally, after the smart cockpit test bench is powered on, an image capture algorithm is used (for example, by calling the image capture function VideoCapture() in the OpenCV library function) to capture the original display image of one or more displays from the real-time image, and / or a reference image is loaded as a reference benchmark for one or more displays (for example, by calling the image capture function VideoCapture() in the OpenCV library function).

[0030] Optionally, after receiving the real-time image captured by the camera, a Gaussian low-pass filtering algorithm may be used to perform denoising on the real-time image. For example, the real-time image is first converted from the spatial domain to the frequency domain using a two-dimensional discrete Fourier transform F(u, v) as shown below to obtain pixels (u, v) on a two-dimensional spectrum:

[0031]

[0032] Wherein, x and y are the horizontal and vertical coordinates on the real-time image, M and N are the height and width of the real-time image, and f(x, y) is the grayscale value at the coordinate point (x, y) on the real-time image.

[0033] Next, a low-pass filtering operation is performed on the two-dimensional spectrum using a Gaussian low-pass filter H(u, v) as shown in the following equation:

[0034]

[0035] Where D(u, v) is the distance from the pixel (u, v) to the center of the spectrum on the two-dimensional spectrum.

[0036] Finally, a two-dimensional inverse discrete Fourier transform is performed on the filtered spectrum to obtain the filtered image f(x, y) as shown below:

[0037]

[0038] like Figure 1 As shown, in step S120, multiple image processing algorithms are simultaneously started through multi-threaded calls to perform fault identification on each original display image to determine whether there is a display fault on each display and the type of display fault, wherein each image processing algorithm in the multiple image processing algorithms is used to identify different types of display faults.

[0039] Multithreading refers to the technology that enables the concurrent execution of multiple threads through software or hardware via program calls. Multithreaded devices, with hardware support, can execute multiple threads simultaneously, thereby improving overall processing performance. Simultaneously activating multiple image processing algorithms through multithreading improves fault identification accuracy and repair efficiency, while also enabling targeted identification and location of various display faults. For example, identifying which display fault is occurring on which display.

[0040] Optionally, the plurality of image processing algorithms include one or more of the following: a first image processing algorithm for identifying black screen faults, a second image processing algorithm for identifying snow screen faults, a third image processing algorithm for identifying screen lag faults, and a fourth image processing algorithm for identifying screen jitter faults. In other words, for the original display image shown on each display, black screen faults, snow screen faults, screen lag faults, and screen jitter faults can be identified and located simultaneously through multi-threaded calls.

[0041] Optionally, in an embodiment where the plurality of image processing algorithms include a first image processing algorithm for identifying black screen faults, step S120 includes: performing grayscale processing on the original display image using a grayscale processing function; performing binarization processing on the grayscale-processed image using a binarization function; counting the number of non-zero pixels in the binarized image using a non-zero count function; comparing the number of non-zero pixels with a first threshold, and determining whether the display corresponding to the original display image has a black screen fault based on the comparison result. Exemplarily, the grayscale processing function cvColor() and the binarization function threshold() in the OpenCV library function are called to perform grayscale processing and binarization threshold processing on the original display image of each display. For example, if the grayscale value of a pixel is greater than or equal to 127, the pixel is processed as a white point (i.e., the grayscale value of the pixel is set to 255); if the grayscale value of the pixel is less than 127, the pixel is processed as a black point (i.e., the grayscale value of the pixel is set to 0). Next, the number of non-zero pixels (i.e., white dots) in the binarized image is counted by calling the non-zero counting function countNonZero() in the OpenCV library function. If the total number of non-zero pixels is greater than or equal to a preset first threshold, the original display image is determined to be normally lit (i.e., the display does not have a black screen failure). Otherwise, the original display image is determined to be not normally lit (i.e., the display has a black screen failure). Optionally, although most laboratories where smart cockpit test benches are located use light sources with stable brightness, there may still be certain brightness fluctuations at different time periods throughout the day. Therefore, the brightness of the laboratory environment can be calibrated in advance to determine the first threshold for different time periods.

[0042] Optionally, in the embodiment in which the plurality of image processing algorithms comprises the second image processing algorithm for identifying the snowflake screen type failure, step S120 comprises: performing feature point detection and calculating feature descriptors of the original display image and the reference image as the reference benchmark respectively by using a feature extraction function; matching the feature descriptors of the original display image and the feature descriptors of the reference image by using a picture matching function to generate matched feature point pairs; calculating the number of feature point pairs in which the feature point similarity is greater than or equal to the second threshold value; comparing the number with the third threshold value, and determining whether the display corresponding to the original display image has the snowflake screen type failure based on the comparison result. Illustratively, an object is created by calling the object creation function xfeatures2d.SIFT_create() in the OpenCV library function, and the feature points (e.g., contour corners, vertices) of the original display image and the reference image are extracted and their feature descriptors are calculated by calling the feature extraction function detectAndCompute() respectively. The feature descriptor is a representation of an image (e.g., it can represent the local image gradient calculated at a selected scale within the region around each feature point), and the commonalities of two pictures can be found by comparing the feature descriptors of the two pictures. Then, the feature descriptors of the two images are matched by calling the picture matching functions FlannBasedMatcher() and knnMatch() in the OpenCV library function to obtain the matched feature point pairs. Next, similarity screening is performed on the matched feature point pairs, that is, the feature point pairs with a similarity greater than or equal to the second threshold value are retained and their number is counted, and if the number is greater than or equal to the preset third threshold value, it is determined that the display does not have the snowflake screen type failure, otherwise, it is determined that the display has the snowflake screen type failure.

[0043] Optionally, in the embodiment in which the plurality of image processing algorithms comprises the second image processing algorithm for identifying the snowflake screen type failure, step S110 comprises: changing the CAN bus signal input to the intelligent cockpit test bench after loading the reference image as the reference benchmark, and receiving the real-time image collected by the camera after the first time period. Illustratively, the normally displayed reference image is first loaded by calling the image loading function imread() in the OpenCV library function, then the change of the CAN bus signal, such as vehicle speed, gear position, etc., is input to the test bench, and the original display image of the display is captured and loaded after the first time period. Local matting (e.g., by calling the image capture function VideoCapture()) can also be performed on the two pictures to extract the interested part.

[0044] Further, optionally, in the embodiment in which the plurality of image processing algorithms comprises a third image processing algorithm for identifying the screen sticking type fault, step S120 comprises: determining whether the display corresponding to the original display image is at risk of sticking based on the number of feature point pairs with similarity greater than or equal to a fourth threshold value among the feature point pairs of the original display image and the reference image; if it is determined that there is a risk of sticking, performing grayscale processing and binaryzation processing on the original display image and the reference image, and counting the number of non-zero pixels in the images after grayscale and binaryzation processing; if the absolute value of the difference between the number of non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold value, it is determined that the display has a screen sticking type fault. Exemplarily, the reference image and the original display image acquired before and after the CAN bus signal input to the intelligent cockpit test bench is changed are sequentially subjected to feature point detection, feature descriptor calculation, feature point pair matching, and similarity screening operations. The above operations have the same or similar steps as the feature point detection, feature descriptor calculation, feature point pair matching, and similarity screening in the snowflake screen fault identification, which will not be described here. Then, the number counted in the similarity screening is compared with the fourth threshold value, if the number is greater than or equal to the preset fourth threshold value, it is determined that the display is at risk of sticking, otherwise, it is determined that the display is not at risk of sticking (i.e., no screen sticking type fault has occurred). If it is determined that the display is at risk of sticking, in order to avoid misjudgment scenarios, next, the original display image and the reference image are subjected to grayscale processing and binaryzation threshold processing by calling the grayscale processing function cvColor() and the binaryzation function threshold() in the OpenCV library function, and the number of non-zero pixels in the two binaryzation processed images is counted by calling the non-zero counting function countNonZero() in the OpenCV library function, if the absolute value of the difference between the number of non-zero pixels of the original display image and the reference image is greater than or equal to the fifth threshold value, it is determined that the display has a screen sticking type fault, otherwise, it is determined that the display does not have a screen sticking type fault.

[0045] Optionally, in an embodiment where the multiple image processing algorithms include a fourth image processing algorithm for identifying screen jitter faults, step S120 includes: storing raw display images acquired during the second time period and performing frame sorting on them; performing a jitter comparison on raw display images with adjacent frame numbers and obtaining a total number of jittered images; comparing the total number of frames with a sixth threshold, and determining whether the display corresponding to the raw display image has a screen jitter fault based on the comparison result. For example, in an embodiment where a camera captures real-time video, the video acquired during the second time period after the camera is powered on can be directly stored and frame sorted. It should be noted that it is necessary to ensure that there are no changes in the CAN bus signal input during this second time period, and thus no display updates occur. Next, for each image frame, a jitter comparison is performed on the image frame with adjacent frames (e.g., the previous frame and the next frame), and the total number of jittered images (e.g., images with larger jitter values ​​between adjacent frames) is counted. If the total number of frames is greater than or equal to the sixth threshold, the display is determined to have a screen jitter fault; otherwise, it is determined that the display does not have a screen jitter fault.

[0046] Optionally, step S120 further includes: monitoring the operating status of each image processing thread in real time; and adjusting the priorities of the multiple image processing threads based on the operating status. It is understood that to prevent thread conflicts, the priorities of multiple threads can be pre-set, for example, a first thread for executing a first image processing algorithm, a second thread for executing a second image processing algorithm, a third thread for executing a third image processing algorithm, and a fourth thread for executing a fourth image processing algorithm. For example, priorities can be set based on the severity of each display fault. For example, a first thread for a black screen fault may have a higher priority than other threads. However, thread starvation is prone to occur under priority scheduling. That is, a higher-priority thread is always waiting to execute before a lower-priority thread is executed, so the lower-priority thread is never executed. To prevent thread starvation, dynamic thread priority setting can be employed. That is, the priority of each thread can be dynamically adjusted based on its real-time operating status. For example, the priority can be adjusted based on how frequently a thread enters a waiting state.

[0047] In step S130, based on the fault identification result, it is determined whether to store the real-time image. Optionally, if it is determined that one or more of the displays has a display fault, the real-time image is stored; if it is determined that no display fault exists, the smart cockpit test bench is controlled to power off, and a wake-up signal is sent to the smart cockpit test bench after a preset time interval. For example, if it is determined in step S120 that any display fault occurs (such as a black screen fault, a snow screen fault, a screen lag fault, or a screen jitter fault), in order to preserve the first scene to provide data analysis support for further diagnosis, the real-time image captured by the camera during the power-on cycle is stored. If no display has a display fault, a sleep signal is sent to the smart cockpit test bench, and a wake-up signal is sent to the smart cockpit test bench after a preset power-on cycle interval (for example, 5 minutes) to restart execution of steps S110-S130.

[0048] Method 10 according to one or more embodiments of the present invention captures raw display images from each display in real time and uses multiple image processing algorithms to identify faults in each raw display image. This ensures real-time image recognition and processing while accurately locating various display faults and promptly preserving the initial scene, providing important data analysis support for further diagnosis. Furthermore, by simultaneously invoking multiple image processing algorithms through multi-threaded calls, method 10 avoids the inefficiency of single-threaded processing and enables targeted identification and location of various display faults, thereby improving the accuracy of identifying and repairing smart cockpit display faults.

[0049] In order to explain the principle of this application more clearly, Figure 2 A method 20 for determining a display failure of a smart cockpit is shown in a more complete form. It should be understood that Figure 2 The examples should not be considered as an endorsement of other examples in this document (e.g., Figure 1 The corresponding embodiments) constitute additional limitations.

[0050] In step S210, in response to receiving a wake-up signal, the smart cockpit test bench is awakened, and then one or more displays and cameras in the smart cockpit test bench are turned on. In step S220, a reference image is loaded as a reference benchmark for one or more displays. In step S230, the main thread is started, a real-time image captured by the camera is received (wherein the real-time image includes the original display image of one or more displays in the smart cockpit test bench) and the original display image of one or more displays is cut out from the real-time image using an image capture algorithm. In step S240, the original display image of one or more displays is loaded. In step S250, the real-time image is denoised using a Gaussian low-pass filtering algorithm. In step S260, multiple sub-threads are simultaneously launched. The first sub-thread executes a first image processing algorithm for identifying black screen faults, the second sub-thread executes a second image processing algorithm for identifying snow screen faults, the third sub-thread executes a third image processing algorithm for identifying screen lag faults, the fourth sub-thread executes a fourth image processing algorithm for identifying screen jitter faults, and the fifth sub-thread monitors the operating status of each image processing thread in real time and adjusts the priority of the multiple image processing threads based on the operating status. In step S270, based on the execution results of the first through fourth sub-threads, a determination is made as to whether the display has a display fault. If so, method 20 proceeds to step S280; otherwise, method 20 proceeds to step S290. In step S280, the program stops looping and stores the real-time image so that the test bench can retain the fault scene. In step S290, a timer is started to send a wake-up signal to the smart cockpit test bench after a preset power-on cycle interval, restarting steps S210-S290.

[0051] According to another aspect of the present invention, there is provided a fault detection device, comprising: a memory configured to store instructions; and a processor configured to execute the instructions to perform the following steps: Figure 1 Method 10 shown or as Figure 2 Method 20 shown.

[0052] According to another aspect of the present invention, a smart cockpit test bench is provided, comprising: a camera for capturing original display images of one or more displays in the smart cockpit test bench; and the fault detection device according to one aspect of the present invention.

[0053] In addition, the present invention can also be implemented as a computer storage medium, in which a program for causing a computer to execute the following steps is stored. Figure 1 Method 10 shown or as Figure 2Method 20 shown. Here, as the computer storage medium, various computer storage media can be used, such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memories (e.g., ROMs, non-volatile memories, etc.), and tapes (e.g., magnetic tapes, cassettes, etc.).

[0054] In the case of applicable, hardware, software or a combination of hardware and software can be used to realize the various embodiments provided by the present invention. Moreover, in the case of applicable, without departing from the scope of the present invention, the various hardware components and / or software components set forth herein can be combined into composite components comprising software, hardware and / or both. In the case of applicable, without departing from the scope of the present invention, the various hardware components and / or software components set forth herein can be divided into subcomponents comprising software, hardware or both. In addition, in the case of applicable, it is contemplated that software components can be implemented as hardware components, and vice versa.

[0055] Software according to the present invention (such as program code and / or data) can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special computers and / or computer systems that are networked and / or otherwise. The embodiments and examples proposed herein are provided to best illustrate the embodiments according to the present invention and its specific applications, and to enable those skilled in the art to implement and use the present invention. However, it will be appreciated by those skilled in the art that the above description and examples are provided only for the convenience of explanation and giving an example. The description proposed is not intended to encompass all aspects of the present invention or to limit the present invention to disclosed precise forms.

Claims

1. A method for determining a display failure in a smart cockpit, characterized in that: The method comprises the following steps: A. loading a baseline image serving as a reference for one or more displays, then changing a CAN bus signal input to the smart cockpit test bench, and receiving a real-time image captured by a camera after a first period of time, wherein the real-time image includes an original display image of the one or more displays within the smart cockpit test bench; B. simultaneously starting multiple image processing threads through multi-threaded calls to perform fault identification on each original display image to determine whether each display has a display fault and the type of the display fault, and during the fault identification process, monitoring the operating status of each image processing thread in real time and adjusting the priority of the multiple image processing threads based on the operating status, wherein each image processing thread corresponds to a different image processing algorithm and each image processing algorithm is used to identify a different type of display fault; and C. Determine whether to store the real-time image based on the fault identification result, The image processing algorithm includes a third image processing algorithm for identifying screen stuck faults, and step B includes: determining, based on the number of feature point pairs between the original display image and the reference image whose similarity is greater than or equal to a fourth threshold, whether a display corresponding to the original display image has a risk of sticking; If it is determined that there is a risk of jamming, grayscale processing and binarization processing are performed on the original display image and the reference image, and the number of non-zero pixels in the grayscale and binarization processed images is counted; If the absolute value of the difference between the number of non-zero pixels of the original display image and the number of non-zero pixels of the reference image is greater than or equal to a fifth threshold, it is determined that a screen lag fault occurs on the display.

2. The method according to claim 1, wherein Step A further includes one or more of the following: In response to receiving a wake-up signal, waking up the smart cockpit test bench; Performing denoising on the real-time image using a Gaussian low-pass filtering algorithm; and An image capture algorithm is used to extract original display images of one or more displays from the real-time image.

3. The method according to claim 1, wherein The image processing algorithm includes one or more of the following: a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snow screen type faults, and a fourth image processing algorithm for identifying screen shaking type faults.

4. The method according to claim 1, wherein The image processing algorithm includes a first image processing algorithm for identifying black screen type faults, and step B includes: Performing grayscale processing on the original display image using a grayscale processing function; Use the binarization function to perform binarization on the grayscale processed image; The number of non-zero pixels in the binary image is counted using a non-zero counting function; The number of non-zero pixels is compared with a first threshold, and based on the comparison result, it is determined whether a display corresponding to the original display image has a black screen fault.

5. The method according to claim 1, wherein The image processing algorithm includes a second image processing algorithm for identifying snow screen type faults, and step B includes: Using a feature extraction function, feature point detection is performed on the original display image and the reference image as a reference, and feature descriptors are calculated; Matching the feature descriptors of the original display image and the feature descriptors of the reference image using an image matching function to generate matched feature point pairs; Calculating the number of feature point pairs whose feature point similarity is greater than or equal to a second threshold value among the matched feature point pairs; The number is compared with a third threshold, and based on the comparison result, it is determined whether the display corresponding to the original display image has a snow screen type fault.

6. The method according to claim 1, wherein The image processing algorithm includes a fourth image processing algorithm for identifying screen shaking faults, and step B includes: storing and frame-sorting the original display images acquired during the second time period; Performing jitter comparison on original display images with adjacent frame numbers and obtaining the total number of frames of the jittered images; The total number of frames is compared with a sixth threshold, and based on the comparison result, it is determined whether a display corresponding to the original display image has a screen jitter fault.

7. The method according to claim 1, wherein Step C includes: If it is determined that one or more of the displays has a display failure, storing the real-time image; If it is determined that there is no display fault on the display, the smart cockpit test bench is controlled to be powered off, and a wake-up signal for the smart cockpit test bench is sent after a preset time interval.

8. A fault detection device, characterized in that: include: a memory configured to store instructions; as well as A processor configured to execute the instructions to perform the method according to any one of claims 1 to 7.

9. An intelligent cockpit test bench, characterized in that: include: A camera, used to capture raw display images of one or more displays in the smart cockpit test bench; as well as The fault detection device as claimed in claim 8.

10. A computer storage medium, characterized in that The computer storage medium includes instructions that, when executed, perform the method of any one of claims 1-7.

Images