Intelligent cabin instrument self-checking method and device, and electronic equipment

Abstract

The application relates to the technical field of intelligent vehicles, and provides an intelligent cabin instrument self-checking method and device and electronic equipment. The method starts a target detection model to detect the state of a target indicator light in the intelligent cabin under the condition that the intelligent cabin starts instrument self-checking, obtains a first time when the target indicator light is first lighted and a second time when the target indicator light is first extinguished after being lighted, and determines that instrument self-checking of the target indicator light is passed when the absolute value of the time difference between the second time and the first time is less than a preset threshold. The method realizes intelligent cabin instrument self-checking verification based on interface detection, improves the accuracy of self-checking result judgment, and improves the judgment efficiency.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle technology, and in particular to a method, device and electronic equipment for self-testing intelligent cockpit instruments. Background Technology

[0002] With the development of smart cockpits, automotive instrument panels offer rich graphical interfaces and display technologies, helping drivers to understand various vehicle parameters in a timely manner, ensuring normal vehicle operation and driving safety. The instrument panel indicator self-test process provides drivers with crucial system information, ensuring driving safety and driver oversight. During the self-test, the instrument panel displays various vehicle information, allowing drivers to better understand the vehicle's operating status and improve driving safety and comfort. Therefore, verifying the phenomena observed during the instrument panel self-test process is particularly important.

[0003] In related technologies, the self-testing and verification of instruments in smart cockpits is usually done manually by the naked eye, which is inefficient, prone to errors, and has poor versatility. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, and electronic device for self-testing smart cockpit instruments to solve the problems of low efficiency and poor versatility in the self-testing of smart cockpit instruments in the prior art.

[0005] A first aspect of this application provides a self-testing method for intelligent cockpit instruments, the method being executed by a testing device, including:

[0006] In response to the determination that the smart cockpit has started instrument self-test, the target detection model is started to detect the status of the target indicator lights in the smart cockpit. The target indicator lights are the self-test indicator lights of any N instruments in the smart cockpit, where N is a positive integer.

[0007] Receive the first and second times returned by the target detection model, where the first time is the time when the target indicator light first turns on and the second time is the time when the target indicator light first turns off after turning on.

[0008] The time difference between the second time and the first time is determined as the detection duration of the target indicator light;

[0009] In response to the absolute value of the difference between the detection time and the preset display time being less than a preset threshold, the instrument self-test corresponding to the target indicator light is determined to have passed.

[0010] A second aspect of this application provides a smart cockpit instrument self-testing device, comprising:

[0011] The startup module is configured to, in response to determining that the smart cockpit is activating instrument self-test, start the target detection model to detect the status of target indicator lights in the smart cockpit. The target indicator lights are the self-test indicator lights of any N instruments in the smart cockpit, where N is a positive integer.

[0012] The receiving module is configured to receive the first time and the second time returned by the target detection model, wherein the first time is the time when the target indicator light first lights up, and the second time is the time when the target indicator light first turns off after it lights up;

[0013] The determination module is configured to determine the time difference between the second time and the first time as the detection duration of the target indicator light;

[0014] The detection module is configured to determine that the instrument self-test of the target indicator light has passed if the absolute value of the difference between the detection duration and the preset display duration is less than a preset threshold.

[0015] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0016] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment starts the target detection model to detect the status of the target indicator light in the smart cockpit under the condition that the smart cockpit has started instrument self-test. It obtains the first time when the target indicator light first lights up and the second time when it first turns off. When the absolute value of the time difference between the second time and the first time is less than a preset threshold, it is determined that the instrument self-test of the target indicator light has passed. This realizes the smart cockpit instrument self-test verification based on interface detection, improves the accuracy of the self-test result judgment, and improves the judgment efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a self-testing method for intelligent cockpit instruments provided in an embodiment of this application.

[0019] Figure 2 This is a flowchart illustrating a method for detecting the status of target indicator lights in a smart cockpit using a target detection model provided in this application embodiment.

[0020] Figure 3This is a flowchart of a method provided in this application embodiment for calling a target detection model in a processing thread to detect video data of a target interface and obtain the status of a target indicator light.

[0021] Figure 4 This is a flowchart of another method for detecting the status of target indicator lights in a smart cockpit using a target detection model provided in this application embodiment.

[0022] Figure 5 This is the overall flowchart of the self-test verification of the intelligent cockpit instrument provided in the embodiments of this application.

[0023] Figure 6 This is a schematic diagram of the structure of the self-testing verification system provided in the embodiments of this application.

[0024] Figure 7 This is a schematic diagram of the workflow of the video thread provided in the embodiments of this application.

[0025] Figure 8 This is a schematic diagram of the workflow of the processing thread provided in the embodiments of this application.

[0026] Figure 9 This is a schematic diagram of the workflow of the self-test verification display method provided in the embodiments of this application.

[0027] Figure 10 This is a schematic diagram of the AI ​​detection and processing provided in the embodiments of this application.

[0028] Figure 11 This is a schematic diagram of the process for implementing instrument self-test verification and test result display based on the QT framework provided in the embodiments of this application.

[0029] Figure 12 This is a detailed flowchart of the self-test verification of the intelligent cockpit instrument provided in the embodiments of this application.

[0030] Figure 13 This is a schematic diagram of a smart cockpit instrument self-testing device provided in an embodiment of this application.

[0031] Figure 14 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0032] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0033] The following will describe in detail, with reference to the accompanying drawings, a method and apparatus for self-testing instruments in a smart cockpit according to an embodiment of this application.

[0034] As mentioned above, in related technologies, the self-test verification of instruments in smart cockpits is usually performed manually. During manual testing, the time it takes for the instrument's self-test indicator light to turn off after continuous display matches the nominal display duration, thereby determining whether the instrument's self-test has passed. Table 1 shows the nominal continuous display duration of some instrument indicator lights in smart cockpits during self-test.

[0035] Table 1. Nominal display duration of some instrument indicator lights during self-test in the smart cockpit.

[0036]

[0037]

[0038] During self-testing in the intelligent cockpit, the indicator lights of each instrument need to illuminate and remain lit for a preset duration before turning off, indicating that the instrument self-test is normal. As shown in Table 1, the continuous display duration of the indicator lights is usually short, for example, 4 seconds. It is difficult for the human eye to accurately determine whether the actual continuous display duration of the indicator light is 4 seconds. Furthermore, when performing a large number of test tasks, the continuous display duration of multiple indicator lights needs to be judged simultaneously, which further increases the difficulty of manual testing.

[0039] In other words, there is no solution in the relevant technologies for accurately detecting the duration of the indicator light during the self-test process. Existing detection methods are inefficient, prone to errors, have poor versatility, and are costly, so the problem of abnormal duration of the instrument's self-test indicator light is usually overlooked.

[0040] In view of this, this application provides a method for self-testing intelligent cockpit instruments. The method uses a target detection model to detect the interface of the intelligent cockpit instruments during self-testing. The method records the first time the indicator lights of each instrument first turn on and the second time they first turn off after turning on. This automatically detects the duration of the indicator lights of each instrument during self-testing. Then, based on the matching relationship between the detected duration of the ...

[0041] Furthermore, the smart cockpit instrument self-testing method provided in this application embodiment, by activating a target detection model to detect the status of target indicator lights in the smart cockpit under the condition that the smart cockpit has started instrument self-testing, obtains the first time when the target indicator light first lights up and the second time when it first turns off after lighting up. When the absolute value of the time difference between the second time and the first time is less than a preset threshold, it is determined that the instrument self-testing of the target indicator light has passed. This realizes smart cockpit instrument self-testing verification based on interface detection, improves the accuracy of self-testing result judgment, and enhances judgment efficiency.

[0042] Figure 1 This is a flowchart illustrating a self-testing method for intelligent cockpit instruments provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0043] In step S101, in response to determining that the smart cockpit has started instrument self-test, the target detection model is activated to detect the status of the target indicator lights in the smart cockpit.

[0044] Among them, the target indicator light is the self-test indicator light of any N instruments in the smart cockpit, where N is a positive integer.

[0045] In step S102, the first time and the second time returned by the target detection model are received.

[0046] The first time is the time when the target indicator light first turns on, and the second time is the time when the target indicator light first turns off after turning on.

[0047] In step S103, the time difference between the second time and the first time is determined as the detection duration of the target indicator light.

[0048] In step S104, in response to the absolute value of the difference between the detection time and the preset display time being less than a preset threshold, it is determined that the instrument self-test corresponding to the target indicator light has passed.

[0049] In some embodiments of this application, the method can be executed by a testing device. This testing device may be, for example, a testing terminal that has communication capabilities and can communicate with the smart cockpit under test to send test-related data to the smart cockpit, or receive relevant data sent by the smart cockpit or other auxiliary devices.

[0050] Furthermore, the testing equipment can also run a test program, which controls the smart cockpit or other auxiliary equipment to execute target events, and acquire various response data when the smart cockpit executes the target events. The acquired response data is then analyzed and processed to obtain test results.

[0051] Furthermore, the testing equipment may also include a display interface that can be used to display relevant information and test results during the testing process.

[0052] In some implementations, after determining that the smart cockpit has initiated its instrument self-test, the testing equipment can activate a target detection model to detect the status of target indicator lights within the smart cockpit. In one example, the smart cockpit can receive a test command sent by the testing equipment and initiate its instrument self-test accordingly. In another example, the smart cockpit can also initiate its instrument self-test automatically; this is not a limitation.

[0053] The object detection model can be any AI (Artificial Intelligence) model capable of object detection, such as the YOLOv5 model. The detection target of the object detection model can be the indicator lights of various instruments; further, the detection target can be the indicator lights indicating the lighting status of various instruments.

[0054] The target detection model detects the status of indicator lights on various instruments in the smart cockpit to obtain the first time each indicator light illuminates, and records this as the first time the indicator light illuminates. Furthermore, the target detection model can also obtain the first time each indicator light turns off after its initial illumination, and record this as the second time the indicator light illuminates.

[0055] The testing equipment receives the first and second times returned by the target detection model and determines the time difference between the second and first times of the target indicator light as the detection duration of the target indicator light. Further, the testing equipment compares this detection duration with a preset display duration. If the absolute value of the difference between the detection duration and the preset display duration is less than a preset threshold, then the instrument corresponding to the target indicator light can be determined to have passed its self-test.

[0056] Understandably, the number of target indicator lights can be one or more. That is, multiple instrument self-tests can be activated simultaneously in the smart cockpit. In this case, the target detection model uses the illuminated indicator lights of multiple instruments as detection targets, recording the first and second timestamps of each indicator light. The testing equipment determines whether the corresponding instrument self-test passes based on the received first and second timestamps of each indicator light.

[0057] According to the technical solution provided in the embodiments of this application, by starting the target detection model to detect the status of the target indicator light in the smart cockpit under the condition that the smart cockpit has started instrument self-test, the first time when the target indicator light first lights up and the second time when it first turns off after lighting up are obtained. When the absolute value of the time difference between the second time and the first time is less than a preset threshold, it is determined that the instrument self-test of the target indicator light has passed. This realizes the smart cockpit instrument self-test verification based on interface detection, improves the accuracy of self-test result judgment, and enhances judgment efficiency.

[0058] Figure 2 This is a flowchart illustrating a method for detecting the status of target indicator lights in a smart cockpit using a target detection model provided in this application embodiment. Figure 2 As shown, the method includes the following steps:

[0059] In step S201, a start message is sent to the video recording application so that the video recording application can record the video of the target interface in real time.

[0060] The target interface should include at least a target indicator light.

[0061] In step S202, a processing thread is started, and video data of the target interface is obtained in the processing thread. The target detection model is called to detect the video data of the target interface and obtain the status of the target indicator light.

[0062] In some embodiments of this application, the testing device can send a start message to a video recording application after determining that the smart cockpit has initiated instrument self-testing, so that the video recording application can record video of the target interface in real time. The video recording application can be a screen recording application built into the smart cockpit, or it can be a video recording application corresponding to an external camera device. The target interface includes at least indicator lights showing instruments currently undergoing self-testing. Furthermore, the target interface may also include indicator lights for instruments that will subsequently require self-testing.

[0063] Furthermore, the testing equipment can also start a processing thread, in which video data of the target interface is acquired, and the target detection model is called to detect the video data of the target interface, thereby obtaining the status of the target indicator light.

[0064] Figure 3 This is a flowchart illustrating a method provided in this application embodiment for detecting video data of a target interface using a target detection model within a processing thread to obtain the status of a target indicator light. Figure 3 As shown, the method includes the following steps:

[0065] In step S301, the video data of the target interface is written to the first object of the processing thread.

[0066] In step S302, in response to receiving the target detection activation signal, the first object is instantiated.

[0067] In step S303, the target detection model is invoked to detect the confidence level of each video data frame in the first object instance frame by frame.

[0068] In step S304, in response to determining that the confidence level of the first target video data frame is greater than a preset confidence threshold, and that the confidence levels of the preceding video data frames of the first target video data frame are all less than or equal to the preset confidence threshold, the timestamp corresponding to the first target video data frame is determined to be the first time.

[0069] In step S305, in response to determining that the confidence level of the second target video data frame is less than or equal to a preset confidence threshold, and that the confidence level of the previous video data frame of the second target video data frame is greater than the preset confidence threshold, the timestamp corresponding to the second target video data frame is determined to be the second time.

[0070] In some embodiments, the video data of the target interface can be first written to a first object of the processing thread. This first object can be the video parameters of the processing process, for example, denoted as cap1. After receiving the signal to enable target detection, the processing thread can instantiate the first object to obtain a first object instance.

[0071] Furthermore, the object detection model can be invoked in the processing thread to detect video data frames one by one in the first object instance and calculate the confidence score of each video data frame. The confidence score of a video data frame refers to the confidence that the video data frame contains the detected target.

[0072] If the target detection model determines that the confidence level of the first target video data frame is greater than a preset confidence threshold, and the confidence levels of all preceding video data frames of the first target video data frame are less than or equal to the preset confidence threshold, then the timestamp corresponding to the first target video data frame is determined as the first time. That is, when the target detection model detects the target indicator light to light up for the first time, it records the timestamp of the currently detected video data frame as the first time for that target indicator light.

[0073] If the target detection model determines that the confidence level of the second target video data frame is less than or equal to a preset confidence threshold, and the confidence level of the previous video data frame of the second target video data frame is greater than the preset confidence threshold, then the timestamp corresponding to the second target video data frame is determined as the second time. That is, when the target detection model detects that the target indicator light is turned off for the first time after it is first lit, it records the timestamp of the currently detected video data frame as the second time of the target indicator light.

[0074] Understandably, when there are multiple detection targets, that is, when the target detection model needs to detect the indicator lights of multiple instruments, the target detection model can output a confidence level for each detection target and record the first time and / or the second time for the detection targets that meet the recording timestamp.

[0075] In some implementations, determining that the confidence level of the target video data frame is greater than a preset confidence threshold can be achieved by inputting the target video data frame into the target detection model, wherein the target video data frame is any video data frame in the first object instance; and determining that the confidence level of the target video data frame is greater than the preset confidence threshold in response to detecting a lit target indicator light in the target video data frame.

[0076] Determining that the confidence level of a target video data frame is less than or equal to a preset confidence threshold can be achieved by inputting a target video data frame into the target detection model. The target video data frame can be any video data frame from the first object instance. In response to the absence of a lit target indicator light in the target video data frame, the confidence level of the target video data frame is determined to be less than or equal to the preset confidence threshold. The preset confidence threshold can be set according to actual needs, for example, to 0.99.

[0077] To facilitate the observation of the instrument's self-test process by testers, the indicator lights of the targets detected by the target detection model after they turn on can be marked, and the marked images can be displayed in the test equipment.

[0078] Figure 4 This is a flowchart illustrating another method for detecting the status of target indicator lights in a smart cockpit using a target detection model, as provided in this application embodiment. Figure 4 Steps S401 to S402 in the illustrated embodiment are Figure 2 Steps S201 to S202 in the illustrated embodiment are basically the same and will not be repeated here. Figure 4 As shown, the method also includes the following steps:

[0079] In step S403, a first video stream signal is defined in the processing thread and bound to a first custom signal slot.

[0080] In step S404, in response to determining that the confidence level of the third target video data frame is greater than a preset confidence threshold, the target indicator light in the third target video data frame is marked to obtain the processed third target video data frame.

[0081] In step S405, the processed third target video data frame is transmitted to the first video stream signal.

[0082] In step S406, the display method is invoked to display the processed third target video data frame on the test device interface through the first custom signal slot.

[0083] In some embodiments of this application, a first video stream signal, such as handle_frame_signal, can be defined in the processing thread and bound to a first custom signal slot. When the processing thread calls the target detection model to determine that the confidence level of the third target video data frame in the first object instance is greater than a preset confidence threshold, the target indicator light in the third target video data frame can be marked to obtain the processed third target video data frame. The marking method can be selected according to actual needs, such as adding region outlines to the target indicator light; no limitation is made here.

[0084] The processed third target video data frame is transmitted to the first video stream signal. The test device calls the display method and displays the processed third target video data frame on the test device interface through the first custom signal slot. This enables the display of an image marked with the target indicator light in the test device.

[0085] On the other hand, the process of calling the object detection model while transmitting video frames to be displayed may cause stuttering issues, because the data frames need to be inferred and detected by the object detection model before being sent back to the system for display.

[0086] To address this issue, a video thread can be started in the test device, within which a second video stream signal can be defined and bound to a second custom signal slot. Upon receiving the signal to start the video stream, the test device instantiates a second object instance and then saves the video data of the target interface to this second object instance. Furthermore, video data frames from the second object instance can be sequentially passed to the second video stream signal.

[0087] At this time, the test equipment calls the display method and displays the third target video data frame on the test equipment interface through the first custom signal slot and the second custom signal slot. The third target video data frame is a superimposed data frame of the original third target video data frame and the processed third target video data frame.

[0088] Understandably, if there is no associated processed video data frame in the first custom signal slot, the test device will only display the original video data frame associated with the second custom signal slot. Furthermore, the first target video data frame, the second target video data frame, and the third target video data frame can be the same video data frame or different video data frames.

[0089] Figure 5 This is a flowchart illustrating the overall process for self-testing and verifying intelligent cockpit instruments provided in this application's embodiments. For example... Figure 5As shown, the process includes first setting up a self-test verification system, then starting a video stream and enabling AI detection when instrument self-test verification is required. This AI detection involves using a target detection model to detect the illumination status of the instrument's indicator lights. Next, the instrument powers on and begins its self-test, using the AI ​​detection results to verify the self-test status, and finally reporting the verification results.

[0090] Figure 6 This is a schematic diagram of the self-testing verification system provided in an embodiment of this application. For example... Figure 6 As shown, this self-testing verification system includes external devices, a client, a test device with a graphical interface, and a model service. The external devices may include a camera and a video stream acquisition application; the video stream acquisition device has a built-in screen recording application. The client includes a video stream display module, an AI detection feedback module, an AI detection real-time log module, an anomaly alert module, a visualization report module, a video storage module, and an instrument log acquisition module. The model service module includes an instrument indicator image set acquisition module, an image annotation module, and a YOLOv5 model service module. The external devices and the client communicate with each other, and the test device with the graphical interface communicates with the model service module.

[0091] In some implementations, the testing equipment can use the QT graphical user interface application development framework to achieve video stream acquisition, target detection, and video stream display. The following example, using the QT framework to implement instrument self-test verification and test result display, details the entire verification process.

[0092] Figure 7 This is a schematic diagram of the video thread workflow provided in an embodiment of this application. For example... Figure 7 As shown, this video thread processes the raw video stream, including a first parameter `frame_signal` and a second parameter `cap`. `frame_signal` is a custom signal, and `cap` is a thread parameter referring to the camera or video stream object. Within the video thread, you can first define the `frame_signal` signal to transmit each frame of the video stream, and then initialize the `cap` object. Next, iterate through the `cap` object information, returning each frame of data. Finally, pass the frame to the `frame_signal` signal.

[0093] In other words, within CameraThread, you can first define a signal `frame_signal` to transmit each frame of the video stream, binding each frame's data for communication with the video stream object. Then, initialize the `cap` object and define the `cap` video stream object; instantiation is not involved at this stage, but will be demonstrated in the self-test verification system's display process. Next, loop through the `cap` object information, returning each frame's data (`frame`), thus continuously acquiring each frame of the video stream data. Finally, pass the `frame` to the signal `frame_signal`, effectively passing each frame's data into a custom signal slot.

[0094] Figure 8 This is a schematic diagram of the workflow of the processing thread provided in an embodiment of this application. For example... Figure 8 As shown, this processing thread handles instrument self-test verification, including a first parameter `handle_frame_signal` and a second parameter `cap`. `handle_frame_signal` is a custom signal, and `cap` is a thread parameter referring to the camera or video stream object. Within the processing thread, a signal `handle_frame_signal` for transmitting each frame of the video stream can be defined, and then the `cap` object is initialized. Next, the information from the `cap` object is read in a loop, returning each frame of data. Each frame of data is fed into the YOLOv5 model for inference, and after the model identifies the target, the target's appearance area is drawn on the frame. Finally, the frame with the target's appearance area drawn is passed to `handle_frame_signal`.

[0095] In other words, within HandleThread, a signal `handle_frame_signal` is first defined to transmit each frame of the video stream, binding each frame's data for communication with the video stream object. Then, the `cap` object is initialized, defining the `cap` video stream object. At this stage, no instance is involved; instantiation will occur during the self-testing and verification system's display process. Next, the `cap` object information is read in a loop, returning each frame of data (`frame`), thus continuously acquiring each frame of the video stream data. Then, Yolov5 is used for model inference, and after target recognition, the target's appearance area is drawn on the `frame`. In one example, OpenCV can be used to draw bounding boxes for detected target images, facilitating system feedback of AI detection results. Finally, the `frame` is passed to `handle_frame_signal`, i.e., the video frame is passed to a custom signal slot.

[0096] Figure 9 This is a schematic diagram illustrating the workflow of the self-test verification display method provided in this application embodiment. For example... Figure 9As shown, the self-test verification display method can include the parameter show_frame. This method uses OpenCV to process the image frame. Once the image frame is passed in, it is captured and displayed on the client (i.e., test device) interface.

[0097] In the self-test verification display method, a CameraThread instance can be used to bind the raw video stream thread to the system UI display screen via a custom signal slot (CameraThread.frame_signal.connect(show_frame)), thereby displaying the raw video stream on the test device interface. Alternatively, a HandleThread instance can be used to bind the video stream processing thread to the system UI display screen via a custom signal slot (HandleThread.handle_frame_signal.connect(show_frame)), thereby drawing the target area on the test device interface after the AI ​​detects the target.

[0098] In other words, within the `show_frame` method, the `cap` object can be passed to the `CameraThread` thread instance. In this case, the original video stream thread binds to the system UI display screen via a custom signal slot. Simultaneously, the `cap` object can also be passed to the `CameraThread` thread instance. In this case, the AI ​​detection and processing video stream thread binds to the system UI display screen via a custom signal slot. The two signals can be displayed overlay. When there is no AI-processed display screen, the original image is displayed. When an AI-processed display screen exists, since AI processing involves adding markers to the original image frames, overlaying the original image signal will not affect the display of the processed image.

[0099] Figure 10 This is a schematic diagram of the AI ​​detection processing provided in an embodiment of this application. Figure 10 As shown, if the AI ​​model is a YOLOv5 model, it can be started first, and an image stream frame can be input into the model. Inference and recognition are then performed on the input image stream frame. If the confidence score of an image frame in the image stream is greater than 0.99, the current timestamp is recorded as the start time from the first detected frame, and subsequent detected targets are not included in the timing. Otherwise, the current timestamp is recorded as the end time from the last undetected frame, and subsequent undetected targets are not included in the timing. The difference between the end time and the start time is then used as the continuous display time obtained from the detection, and the detection process is exited.

[0100] In other words, the YOLOv5 inference frame method in a video stream can infer one or more targets. If the inference confidence of an image frame is greater than 0.99, the current timestamp is recorded from the first detected frame as the start time. Subsequent detected targets are not included in the timing until continuous detection no longer satisfies a confidence level greater than 0.99, at which point the end time is obtained. The difference between the detection end time and the detection start time is taken as the continuous display time obtained from the detection.

[0101] Figure 11 This is a schematic diagram illustrating the process of implementing instrument self-test verification and test result display based on the QT framework provided in this application embodiment. For example... Figure 11 As shown, you can first start the video stream, for example, by using a button control on the interface. Then, instantiate the cap object, for example, by opening the video stream object for instance processing, such as with a camera, cv2.VideoCapture. Next, start the CameraThread thread to implement the transmission of image frame data, and pass the frame_signal to the image frame. The image frame from the original video stream can be transmitted through a custom signal slot.

[0102] Next, AI detection can be enabled, for example, YOLOv5 model detection. After enabling YOLOv5 model detection, a HandleThread can be started, with the processed image frame passed to handle_frame_signal. The processed video stream image frame can be passed through a custom signal slot. Finally, the real-time AI detection display is presented in the self-test verification system of the test equipment, and a report is generated.

[0103] The self-test verification system can capture each frame of the raw video stream and directly bind it to the self-test system's display method (show_frame). The CameraThread method takes a cap object parameter. When the "Start Video Stream" button is triggered in the self-test verification system interface, the specific camera or video stream is instantiated, and the instantiated cap object is passed to the CameraThread method. At this time, CameraThread is started, and the self-test system displays the camera device or video stream image.

[0104] Simultaneously, the self-testing verification system can also process the image frames accessed by the YOLOv5 model inference. After processing, it acquires each frame and directly binds it to the self-testing system's display method (show_frame), with the HandleThread method taking a cap object parameter. After "Starting Video Stream" in the self-testing verification system interface, followed by "Starting AI Detection," the instantiated cap object is passed to the HandleThread method. At this point, HandleThread is enabled, and the self-testing system displays the detection effect on the interface after the YOLOv5 model inference detects the target. This effect display could include, for example, interface prompts or drawing the target's outline.

[0105] Furthermore, the self-testing verification system uses CameraThread and HandleThread thread methods to bind custom signal slots, thereby passing image frames to the show_frame(frame) method, which can solve the stuttering problem.

[0106] Figure 12 This is a detailed flowchart of the self-test verification of intelligent cockpit instruments provided in the embodiments of this application. For example... Figure 12 As shown, when performing instrument self-test verification, you can first enable AI detection, such as using the YOLOv5 model. After enabling it, monitor the instrument screen in real time and capture the instrument self-test process.

[0107] In some embodiments, the vehicle's power cord can be closed using a program on the test equipment. For example, a relay-like device can be controlled via the device's COM port (Communication Port) to simulate the vehicle's startup, thereby simulating the instrument cluster startup process. Next, system client testing is initiated. The instrument cluster's self-test screen is used to continuously monitor the duration of the indicator lights. It is then determined whether this duration matches the expected result, i.e., the system-detected duration of the instrument cluster self-test indicator lights is compared with the expected duration. If it meets the requirements, verification is successful, and a report is generated; otherwise, the vehicle's logs and video are retrieved for the instrument cluster developers' reference, and a report is generated.

[0108] In some embodiments of this application, if the test device does not receive at least one of the first time and the second time of the target indicator light, or determines that the absolute value of the difference between the detection duration of the target indicator light and the preset display duration is less than a preset threshold, it can be determined that the instrument self-test corresponding to the target indicator light has failed.

[0109] In some implementations, the target detection model can be a trained target detection model. The trained detection model is obtained by: acquiring training samples, which include at least images of self-test indicator lights of different instruments in different types of smart cockpits when they are lit; acquiring a pre-trained target detection model; and training the target detection model using the training samples until the target detection model's recognition accuracy for any lit indicator light is greater than a preset recognition threshold.

[0110] Table 2 shows examples of some training samples:

[0111] Table 2 shows some examples of training samples.

[0112]

[0113]

[0114] In one example, during training, 70% of the data can be used as the training set, 20% as the validation set, and 10% as the test set. During training, a comprehensive evaluation is performed, considering factors such as precision and recall curves, mean precision, and confusion matrix, to determine if the model has converged.

[0115] The technical solution adopted in this application uses a target detection model to accurately detect the display duration of the instrument self-test indicator light, solving the problem of mistesting caused by subjective human factors in the instrument self-test verification process. This achieves full automation of the instrument self-test indicator light verification process, resolving serious issues of low efficiency and high labor costs. Furthermore, this instrument self-test indicator light verification method can be reused across various vehicle models. The fully automated instrument self-test verification process improves implementation efficiency and addresses the problem of poor versatility.

[0116] Furthermore, the technical solution of this application embodiment uses an AI model combined with program-controlled instrument startup to obtain the continuous display time of indicator lights during the instrument self-test process, reducing the accuracy issues of manual verification. This forms a set of automated program control combined with AI technology for verifying the instrument self-test process. It is simple to operate; only the self-test verification system / program needs to be operated to automatically verify the instrument self-test process and generate a report, greatly reducing human resource costs. By program-controlled vehicle startup, the instrument screen is indirectly displayed. Running AI technology to pre-train target indicator lights allows for rapid implementation in the instrument self-test verification process, improving verification efficiency. Moreover, the trained indicator light content is universal across various vehicle models during instrument self-test verification. Customized verification reports are automatically generated, and the entire verification process is recorded in real-time.

[0117] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0118] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0119] Figure 13 This is a schematic diagram of a smart cockpit instrument self-testing device provided in an embodiment of this application. Figure 13 As shown, the device includes:

[0120] The startup module 1301 is configured to, in response to determining that the smart cockpit is performing instrument self-test, start the target detection model to detect the status of the target indicator lights in the smart cockpit. The target indicator lights are the self-test indicator lights of any N instruments in the smart cockpit, where N is a positive integer.

[0121] The receiving module 1302 is configured to receive the first time and the second time returned by the target detection model, wherein the first time is the time when the target indicator light first lights up, and the second time is the time when the target indicator light first turns off after it lights up.

[0122] The determination module 1303 is configured to determine the time difference between the second time and the first time as the detection duration of the target indicator light.

[0123] The detection module 1304 is configured to determine that the instrument self-test of the target indicator light has passed when the absolute value of the difference between the detection duration and the preset display duration is less than a preset threshold.

[0124] According to the technical solution provided in the embodiments of this application, by starting the target detection model to detect the status of the target indicator light in the smart cockpit under the condition that the smart cockpit has started instrument self-test, the first time when the target indicator light first lights up and the second time when it first turns off after lighting up are obtained. When the absolute value of the time difference between the second time and the first time is less than a preset threshold, it is determined that the instrument self-test of the target indicator light has passed. This realizes the smart cockpit instrument self-test verification based on interface detection, improves the accuracy of self-test result judgment, and enhances judgment efficiency.

[0125] In some implementations, the target detection model is activated to detect the status of target indicator lights in the smart cockpit, including: sending a start message to a video recording application to enable the video recording application to record video of the target interface in real time, the target interface including at least the target indicator lights; starting a processing thread, acquiring video data of the target interface in the processing thread, calling the target detection model to detect the video data of the target interface, and obtaining the status of the target indicator lights.

[0126] In some implementations, the process involves calling a target detection model in a processing thread to detect video data of the target interface and obtain the status of the target indicator light. This includes: writing the video data of the target interface into a first object of the processing thread; instantiating the first object in response to receiving a signal to enable target detection; calling the target detection model to detect the confidence level of each video data frame in the first object instance frame by frame; determining the timestamp corresponding to the first target video data frame as a first time in response to determining that the confidence level of the first target video data frame is greater than a preset confidence threshold, and that the confidence levels of the preceding video data frames of the first target video data frame are all less than or equal to the preset confidence threshold; and determining the timestamp corresponding to the second target video data frame as a second time in response to determining that the confidence level of the second target video data frame is less than or equal to the preset confidence threshold, and that the confidence level of the previous video data frame of the second target video data frame is greater than the preset confidence threshold.

[0127] In some implementations, determining that the confidence level of a target video data frame is greater than a preset confidence threshold includes: inputting a target video data frame into a target detection model, wherein the target video data frame is any video data frame in the first object instance; and determining that the confidence level of the target video data frame is greater than the preset confidence threshold in response to detecting a lit target indicator light in the target video data frame. Determining that the confidence level of a target video data frame is less than or equal to the preset confidence threshold includes: inputting a target video data frame into a target detection model, wherein the target video data frame is any video data frame in the first object instance; and determining that the confidence level of the target video data frame is less than or equal to the preset confidence threshold in response to not detecting a lit target indicator light in the target video data frame.

[0128] In some implementations, the method further includes: defining a first video stream signal in a processing thread and binding the first video stream signal to a first custom signal slot; in response to determining that the confidence level of the third target video data frame is greater than a preset confidence threshold, marking the target indicator light in the third target video data frame to obtain the processed third target video data frame; passing the processed third target video data frame to the first video stream signal; and calling a display method to display the processed third target video data frame on the test device interface through the first custom signal slot.

[0129] In some implementations, the method further includes: starting a video thread, defining a second video stream signal in the video thread, and binding the second video stream signal to a second custom signal slot; in response to receiving the start video stream signal, instantiating a second object and saving the video data of the target interface to the second object instance; sequentially passing the video data frames in the second object instance to the second video stream signal; and calling a display method to display a third target video data frame on the test device interface through the first custom signal slot and the second custom signal slot, wherein the third target video data frame is a superimposed data frame of the original third target video data frame and the processed third target video data frame.

[0130] In some implementations, the method further includes: determining that the instrument self-test corresponding to the target indicator light has failed in response to not receiving at least one of the first time and the second time, or in response to determining that the absolute value of the difference between the detection duration and the preset display duration is less than a preset threshold.

[0131] In some implementations, the target detection model is a trained target detection model; the trained detection model is trained in the following way: acquiring training samples, which include at least images of self-test indicator lights of different instruments in different types of smart cockpits when they are lit; acquiring a pre-trained target detection model; and training the target detection model using the training samples until the target detection model's recognition accuracy for any lit indicator light is greater than a preset recognition threshold.

[0132] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0133] Figure 14 This is a schematic diagram of the electronic device provided in an embodiment of this application. For example... Figure 14 As shown, the electronic device 14 of this embodiment includes: a processor 1401, a memory 1402, and a computer program 1403 stored in the memory 1402 and executable on the processor 1401. When the processor 1401 executes the computer program 1403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 1401 executes the computer program 1403, it implements the functions of each module / unit in the various device embodiments described above.

[0134] Electronic device 14 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 14 may include, but is not limited to, processor 1401 and memory 1402. Those skilled in the art will understand that... Figure 14 This is merely an example of electronic device 14 and does not constitute a limitation on electronic device 14. It may include more or fewer components than shown, or different components.

[0135] The processor 1401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0136] The memory 1402 can be an internal storage unit of the electronic device 14, such as a hard disk or RAM of the electronic device 14. The memory 1402 can also be an external storage device of the electronic device 14, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc., equipped on the electronic device 14. The memory 1402 can also include both internal and external storage units of the electronic device 14. The memory 1402 is used to store computer programs and other programs and data required by the electronic device.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0139] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An intelligent cockpit instrument self-checking method, characterized in that, The method is executed by a test device, and comprises: In response to determining that the intelligent cockpit starts an instrument self-check, a start message is sent to a video recording application to enable the video recording application to record a video of a target interface in real time, the target interface comprising at least the target indicator light; A processing thread is started, in which video data of the target interface is obtained, a target detection model is called to detect the video data of the target interface to detect a state of a target indicator light in the intelligent cockpit, and the state of the target indicator light is obtained, the target indicator light being a self-check indicator light of any N instruments in the intelligent cockpit, N being a positive integer; A first time and a second time returned by the target detection model are received, wherein the first time is a time at which the target indicator light is first lit, and the second time is a time at which the target indicator light is first extinguished after being lit; A time difference between the second time and the first time is determined as a detection duration of the target indicator light; In response to determining that an absolute value of a difference between the detection duration and a preset display duration is less than a preset threshold, it is determined that a self-check of the instrument corresponding to the target indicator light is passed; In the processing thread, a first video stream signal is defined, the first video stream signal is bound to a first custom signal slot, and third target video data frames in which the target indicator light is marked are transmitted into the first video stream signal; A video thread is started, in which a second video stream signal is defined, the second video stream signal is bound to a second custom signal slot, and data frames of the video data of the target interface are sequentially transmitted into the second video stream signal; A display method is called, and the third target video data frames are displayed on the test device interface, the third target video data frames being superimposed data frames of original third target video data frames and the third target video data frames in which the target indicator light is marked.

2. The method of claim 1, wherein, In the processing thread, the target detection model is called to detect the video data of the target interface, and the state of the target indicator light is obtained, comprising: The video data of the target interface is written into a first object of the processing thread; In response to receiving a start target detection signal, the first object is instantiated; The target detection model is called to detect confidence of each video data frame in the first object instance frame by frame; In response to determining that confidence of a first target video data frame is greater than a preset confidence threshold, and confidence of a previous video data frame of the first target video data frame is less than or equal to the preset confidence threshold, a timestamp corresponding to the first target video data frame is determined as the first time; In response to determining that confidence of a second target video data frame is less than or equal to the preset confidence threshold, and confidence of a previous frame of video data frame of the second target video data frame is greater than the preset confidence threshold, a timestamp corresponding to the second target video data frame is determined as the second time.

3. The method of claim 2, wherein, Determining that confidence of a target video data frame is greater than a preset confidence threshold comprises: The target video data frame is input into the target detection model, the target video data frame being any one of video data frames in the first object instance; in response to detecting the target indicator light in the target video data frame in the light-on state, determining that the confidence of the target video data frame is greater than a preset confidence threshold; or determining that the confidence of the target video data frame is less than or equal to a preset confidence threshold, comprising: inputting the target video data frame into the target detection model, the target video data frame being any one of the video data frames in the first object instance; in response to not detecting the target indicator light in the target video data frame in the light-on state, determining that the confidence of the target video data frame is less than or equal to a preset confidence threshold.

4. The method of claim 3, wherein, the third target video data frame in which the target indicator light is marked is input into the first video stream signal, comprising: in response to determining that the confidence of the third target video data frame is greater than a preset confidence threshold, marking the target indicator light in the third target video data frame to obtain a processed third target video data frame; the processed third target video data frame is input into the first video stream signal.

5. The method of claim 1, wherein, the data frames of the video data of the target interface are sequentially input into the second video stream signal, comprising: in response to receiving the start video stream signal, instantiating a second object of the video thread, and saving the video data of the target interface to the second object instance; the video data frames in the second object instance are sequentially input into the second video stream signal.

6. The method of claim 1, wherein, The method further comprises: in response to not receiving at least one of the first time and the second time, or in response to determining that the absolute value of the difference between the detection duration and the preset display duration is less than a preset threshold, determining that the self-check of the instrument corresponding to the target indicator light fails.

7. The method according to any one of claims 1 to 6, characterized in that, The target detection model is a trained target detection model; The trained detection model is trained in the following manner: obtain training samples, the training samples at least including images of self-check indicator lights of different instruments in different types of intelligent cockpit in light-on state; obtain a pre-trained target detection model; train the target detection model using the training samples until the identification accuracy of the target detection model for any indicator light in the light-on state is greater than a preset identification threshold.

8. An intelligent cabin instrument self-checking device, characterized in that, comprising: a starting module configured to, in response to determining that the intelligent cockpit starts instrument self-check, send a starting message to a video recording application to enable the video recording application to record the video of the target interface in real time, the target interface at least including the target indicator light; start a processing thread, in which the video data of the target interface is obtained, and a target detection model is called to detect the video data of the target interface to detect the state of the target indicator light in the intelligent cockpit, obtaining the state of the target indicator light, the target indicator light being a self-check indicator light of any N instruments in the intelligent cockpit, N being a positive integer; a receiving module configured to receive a first time and a second time returned by the target detection model, wherein the first time is the time when the target indicator light is first turned on, and the second time is the time when the target indicator light is first turned off after being turned on; The determining module is configured to determine a time difference between the second time and the first time as a detection duration of the target indicator light; The detecting module is configured to determine that the target indicator light corresponds to a self-check pass of the instrument in response to determining that an absolute value of a difference between the detection duration and a preset display duration is less than a preset threshold. The starting module is further configured to define a first video stream signal in the processing thread, bind the first video stream signal with a first custom signal slot, and transmit a third target video data frame for marking processing of the target indicator light into the first video stream signal; start a video thread, define a second video stream signal in the video thread, bind the second video stream signal with a second custom signal slot, and sequentially transmit data frames of video data of the target interface into the second video stream signal; and call a display method to display the third target video data frame on a test equipment interface, the third target video data frame being a superimposed data frame of an original third target video data frame and the third target video data frame for marking processing of the target indicator light.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 7.

Images