Vehicle-mounted display equipment detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle-mounted display equipment detection method and device, electronic equipment and a storage medium, and relates to the field of vehicle machine screen detection.The method comprises the steps that a video signal is obtained from vehicle-mounted display equipment through a video capture card and converted into a standard image frame sequence, a vehicle machine system serial port log is obtained, the standard image frame sequence is preprocessed, and the vehicle machine system serial port log is obtained; the method comprises the steps of preprocessing a standard image frame sequence, determining visual features based on the preprocessed standard image frame sequence, determining time sequence features of the preprocessed standard image frame sequence based on the visual features, determining environment features based on a serial port log of a vehicle machine system, and determining a detection result of the vehicle-mounted display equipment based on the visual features, the environment features and the time sequence features. By directly collecting the video signal of the vehicle-mounted display device and combining the multi-modal features to perform anomaly detection, the accuracy of vehicle-mounted display device detection is improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle infotainment screen testing, and more specifically, to a method, apparatus, electronic device, and storage medium for testing vehicle display devices. Background Technology

[0002] Existing in-vehicle display testing typically relies on the following two methods: The first method is to directly call the screenshot interface from the system interface to determine whether the display is normal. However, this method cannot detect abnormalities at the display hardware level, such as the screen not turning on or the screen freezing.

[0003] The second method involves capturing images of the screen using a camera for analysis. This method is significantly affected by external conditions such as lighting, angle, and reflections, resulting in poor detection stability.

[0004] As in-vehicle intelligent cockpit systems become increasingly complex, traditional testing methods are struggling to meet the demands for stable and accurate automated testing. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for testing in-vehicle display devices, which can improve the accuracy of testing in-vehicle display devices.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a method for detecting an in-vehicle display device, the method comprising: The video signal is acquired from the vehicle display device via a video capture card and converted into a standard image frame sequence; Obtain the vehicle infotainment system serial port logs; The standard image frame sequence is preprocessed; Visual features are determined based on the preprocessed standard image frame sequence; The temporal characteristics of the preprocessed standard image frame sequence are determined based on the visual features. Environmental characteristics are determined based on the vehicle infotainment system serial port logs; The detection results of the in-vehicle display device are determined based on the visual features, environmental features, and temporal features.

[0007] In an optional implementation, the step of determining visual features based on the preprocessed standard image frame sequence includes: The image features corresponding to the preprocessed standard image frame sequence are determined, wherein the image features include average brightness, brightness variance, contrast, color ratio, texture complexity, and edge density. The preprocessed standard image frame sequence is subjected to high-dimensional spatial feature extraction using a convolutional neural network (CNN). The high-dimensional spatial features include feature vectors that characterize the spatial texture, edge structure, and color distribution of the image. The image features and the high-dimensional spatial features are used as the visual features of the preprocessed standard image frame sequence.

[0008] In an optional implementation, the step of determining the temporal features of the preprocessed standard image frame sequence based on the visual features includes: Visual features from multiple consecutive frames are input into a recurrent neural network (RNN) or a long short-term memory network (LSTM) to model the temporal features of image changes.

[0009] In an optional implementation, the step of determining the detection result of the in-vehicle display device based on the visual features, environmental features, and temporal features includes: The visual features, environmental features, and temporal features are fused to form a multimodal fusion feature; The multimodal fusion features are input into the trained multimodal classification model, and the confidence values ​​of the vehicle display device under multiple predefined categories are output. Obtain the highest confidence value and determine the detection result based on the category corresponding to the highest confidence value.

[0010] In an optional implementation, the step of fusing the visual features, environmental features, and temporal features to form a multimodal fusion feature includes: The visual features, environmental features, and temporal features are concatenated to obtain multimodal fusion features.

[0011] In an optional implementation, the step of fusing the visual features, environmental features, and temporal features to form a multimodal fusion feature includes: Sub-classification models are trained independently based on visual features, environmental features, and temporal features, respectively, and the confidence scores corresponding to visual features, environmental features, and temporal features are output. Based on the confidence level corresponding to the visual feature, determine the first weight corresponding to the visual feature; Based on the confidence level corresponding to the environmental feature, determine the second weight corresponding to the environmental feature; Based on the confidence level corresponding to the time series feature, determine the third weight corresponding to the time series feature; Based on the first weight, the second weight, and the third weight, the visual features, environmental features, and temporal features are weighted and fused to obtain multimodal fusion features.

[0012] In an optional implementation, the method further includes: Compare the highest confidence value with a set threshold. If the highest confidence level is lower than the set threshold, it is marked as suspicious and a prompt message is output for manual review.

[0013] Secondly, embodiments of this application provide a vehicle-mounted display device detection apparatus, the apparatus comprising: The acquisition module is used to acquire video signals from the in-vehicle display device through the video capture card and convert them into a standard image frame sequence; and to acquire the serial port logs of the vehicle system. The processing module is used to preprocess the standard image frame sequence; determine visual features based on the preprocessed standard image frame sequence; determine temporal features based on the visual features; determine environmental features based on the vehicle system serial port logs; and determine the detection result of the vehicle display device based on the visual features, environmental features, and temporal features.

[0014] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the vehicle-mounted display device detection method.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle-mounted display device detection method.

[0016] This application has the following beneficial effects: This application acquires video signals from an in-vehicle display device via a video capture card and converts them into a standard image frame sequence. It also acquires the vehicle system serial port logs, preprocesses the standard image frame sequence, determines visual features based on the preprocessed standard image frame sequence, determines the temporal features of the preprocessed standard image frame sequence based on the visual features, determines environmental features based on the vehicle system serial port logs, and determines the detection result of the in-vehicle display device based on the visual features, environmental features, and temporal features. By directly acquiring the video signal from the in-vehicle display device and combining it with multimodal features for anomaly detection, the accuracy of in-vehicle display device detection is improved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A block diagram of an electronic device provided in an embodiment of the present invention; Figure 2 This is one of the flowcharts illustrating a method for detecting an in-vehicle display device according to an embodiment of the present invention; Figure 3 This is a second schematic flowchart of a method for detecting an in-vehicle display device provided in an embodiment of the present invention; Figure 4 This is a third flowchart illustrating a method for detecting an in-vehicle display device according to an embodiment of the present invention. Figure 5 This is a structural block diagram of a vehicle-mounted display device testing device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0022] In the description of this invention, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0023] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0024] In the description of this application, it should also be noted that, unless otherwise expressly specified and limited, the terms "set up," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0025] The inventors discovered through extensive research that existing in-vehicle display devices are not accurate enough.

[0026] In view of the above-mentioned problems, this embodiment provides a method, apparatus, electronic device, and storage medium for detecting in-vehicle display devices. It can acquire video signals from in-vehicle display devices via a video capture card and convert them into standard image frame sequences. It also acquires the vehicle system serial port logs, preprocesses the standard image frame sequences, determines visual features based on the preprocessed standard image frame sequences, determines temporal features based on the visual features, determines environmental features based on the vehicle system serial port logs, and determines the detection result of the in-vehicle display device based on the visual features, environmental features, and temporal features. By directly acquiring video signals from the in-vehicle display device and combining them with multimodal features for anomaly detection, the accuracy of in-vehicle display device detection is improved. The solution provided in this embodiment is described in detail below.

[0027] This embodiment provides an electronic device capable of training in-vehicle display devices. In one possible implementation, the electronic device can be a user terminal, such as, but not limited to, a server, smartphone, personal computer (PC), tablet computer, personal digital assistant (PDA), mobile internet device (MID), etc.

[0028] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application. The electronic device 100 may further include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0029] The electronic device 100 includes an in-vehicle display device detection device 110, a memory 120, and a processor 130.

[0030] The components of the memory 120 and processor 130 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The vehicle display device detection device 110 includes at least one software function module that can be stored in the memory 120 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 100. The processor 130 is used to execute the executable modules stored in the memory 120, such as the software function modules and computer programs included in the vehicle display device detection device 110.

[0031] The memory 120 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 120 is used to store programs, and the processor 130 executes the programs after receiving execution instructions.

[0032] Please refer to Figure 2 , Figure 2 For application Figure 1 The flowchart below shows a method for testing an in-vehicle display device of an electronic device 100. The method includes a detailed description of each step.

[0033] S201: Acquire video signals from the vehicle display device via a video capture card and convert them into a standard image frame sequence.

[0034] S202: Obtain the vehicle infotainment system serial port log.

[0035] S203: Preprocess the standard image frame sequence.

[0036] S204: Determine visual features based on the preprocessed standard image frame sequence.

[0037] S205: Determine the temporal characteristics of the preprocessed standard image frame sequence based on visual features.

[0038] S206: Determine environmental characteristics based on vehicle infotainment system serial port logs.

[0039] S207: Determine the detection results of the vehicle-mounted display device based on visual features, environmental features, and temporal features.

[0040] A video capture card is used to connect to the output interface of the vehicle display device to capture video signals in real time and convert them into a standard image frame sequence.

[0041] You'll need a DP (Display Device) adapter for the video capture card, and a DP-to-HDMI adapter. One end will plug into the car's infotainment system, and the other end into the computer's HDMI port. The video capture card will then capture the output signal from the in-vehicle display device in real-time at a sampling frequency of 30fps and convert the video signal into a standard image frame sequence.

[0042] The standard image frame sequence is preprocessed, which may include grayscale conversion, noise reduction, equalization, etc., to obtain the preprocessed standard image frame sequence.

[0043] One way to grayscale a standard image frame sequence is to convert each standardized color image frame into a single-channel grayscale image, reducing computational complexity and highlighting brightness information to facilitate the subsequent extraction of features such as edges and textures. Denoising is then performed on each standardized image frame to eliminate random noise introduced during image transmission or acquisition, preventing false detection of edges or texture anomalies.

[0044] Noise reduction methods can include, but are not limited to, the following: Gaussian filtering, median filtering, bilateral filtering, morphological denoising, nonlocal mean denoising, etc.

[0045] Histogram equalization is performed on each standardized image frame to enhance image contrast and bring out details in areas that are too dark or too bright, which helps to identify problems such as color cast and low-contrast gray screen.

[0046] Feature extraction is performed on the preprocessed standard image frame sequence to obtain visual features, which include image features and high-dimensional space features. Image features can include average brightness, brightness variance, contrast ratio, color ratio, texture complexity, and edge density. Average brightness indicates the overall brightness of the vehicle display device, brightness variance indicates whether the screen of the vehicle display device is uniform, contrast ratio indicates whether the vehicle display device is normal and whether there are high and low grayscale changes, color ratio is used to determine the color cast and color abnormality of the vehicle display device, texture complexity is used for screen distortion or noise detection, and edge density is used to determine whether there is image content on the vehicle display device.

[0047] High-dimensional spatial features can include edges, color blocks, shapes, screen icons, UI components, and noise distribution.

[0048] Based on the high-dimensional spatial features of the preprocessed standard image frame sequence, the temporal features of the standard image frame sequence are determined. The temporal features of the preprocessed standard image frame sequence may include the mean of inter-frame differences, the variance of inter-frame differences, the rate of change of brightness in consecutive frames, and the temporal pattern, etc.

[0049] Environmental characteristics can be determined by the vehicle's infotainment system serial port logs. These characteristics may include light intensity, temperature, humidity, etc.

[0050] Finally, the detection results of the vehicle-mounted display device are determined based on visual features, environmental features, and temporal features. Multimodal features are combined for detection to improve detection accuracy.

[0051] There are several ways to determine visual features based on preprocessed standard image frame sequences. In one such method, such as... Figure 3 As shown, it includes the following steps: S301: Determine the image features corresponding to the preprocessed standard image frame sequence.

[0052] The image features include average brightness, brightness variance, contrast, color ratio, texture complexity, and edge density.

[0053] S302: Use a convolutional neural network (CNN) to extract high-dimensional spatial features from the preprocessed standard image frame sequence.

[0054] Among them, high-dimensional spatial features include feature vectors that characterize the spatial texture, edge structure, and color distribution of an image.

[0055] S303: Use the image features and the high-dimensional spatial features as the visual features of the preprocessed standard image frame sequence.

[0056] After processing steps such as grayscale conversion, denoising, and histogram equalization, a set of time-aligned and quality-optimized standard image frame sequences is obtained. This sequence serves as the foundational input for subsequent multimodal analysis, requiring further extraction of discriminative image features in a high-dimensional semantic space to support accurate identification of abnormal states such as bright screens, black screens, distorted screens, color casts, and gray screens. To this end, this scheme employs a deep convolutional neural network (CNN) to perform layer-by-layer abstract representation of each standard image frame, extracting deep feature vectors in terms of texture, edges, brightness distribution, and color structure.

[0057] High-dimensional feature extraction methods based on CNNs can be used to construct lightweight CNN backbone networks by selecting lightweight architectures suitable for embedded deployment, such as MobileNetV2 / V3, ShuffleNet, and ResNet-18. Standard image frames are used as input to the CNN, and high-dimensional feature vectors are output.

[0058] This application embodiment constructs a standardized image frame sequence and utilizes a deep CNN to perform high-dimensional nonlinear mapping on it, thereby achieving a refined representation of the visual state of the vehicle-mounted display screen. The extracted image features not only include low-level information such as brightness, texture, and color, but also integrate high-level semantic features automatically learned by the network, significantly improving the ability to identify various display anomalies.

[0059] There are several ways to determine the temporal features of the preprocessed standard image frame sequence based on visual features. In one approach, the visual features of multiple consecutive frames are input into a recurrent neural network (RNN) or a long short-term memory network (LSTM) to model the temporal features of image changes.

[0060] In vehicle display anomaly detection, analyzing only a single frame image is insufficient to identify dynamic faults. Therefore, this invention further introduces a temporal modeling mechanism. By dynamically analyzing high-dimensional visual feature sequences extracted from multiple consecutive frames over time, it captures the state evolution patterns of the vehicle interface, thereby achieving accurate identification of time-dependent display anomalies.

[0061] Specifically, a high-dimensional spatial feature vector is obtained for each frame in the preprocessed standard image frame sequence using a convolutional neural network (CNN). This feature vector represents the deep visual semantic information of the current frame in terms of texture, edges, brightness distribution, and color structure. Subsequently, the image features of consecutive frames (e.g., T=90S, corresponding to a 3-second video stream at a frame rate of 30fps) are arranged chronologically to form a temporal input sequence, serving as the basic input for temporal modeling. This temporal input sequence is fed into a recurrent neural network or a long short-term memory network (LSTM) for sequence modeling. A unidirectional or multi-layer stacked LSTM structure is preferred, with a hidden unit size of 256, which effectively captures the trend of image state evolution over time and alleviates the gradient vanishing problem. During forward propagation, the LSTM updates its internal hidden state and cell state frame by frame, ultimately outputting a temporal feature vector that aggregates global temporal context information. This vector can be used to indicate whether there are abnormal behaviors such as image stagnation, periodic flickering, or unexpected jumps within the current window.

[0062] Furthermore, a bidirectional LSTM structure or a self-attention mechanism can be introduced on top of the LSTM to enhance the ability to focus on key time nodes. Simultaneously, to adapt to embedded deployment environments, the model can be accelerated with TensorRT and FP16 / INT8 quantization to ensure inference latency is less than 100 milliseconds. The extracted temporal features can not only be used to independently determine the occurrence of dynamic anomalies, but also serve as an important input to the multimodal fusion module, performing feature-level concatenation or decision-level weighted fusion with image features and environmental features, thereby improving the accuracy and robustness of the overall detection system.

[0063] There are several ways to determine the detection results of in-vehicle display devices based on visual features, environmental features, and temporal features. In one such method, such as... Figure 4 As shown, it includes the following steps: S401: Combines visual features, environmental features, and temporal features to form a multimodal fusion feature.

[0064] S402: Input the multimodal fusion features into the trained multimodal classification model and output the confidence values ​​of the vehicle display device under multiple predefined categories.

[0065] S403: Obtain the highest confidence value and determine the detection result based on the category corresponding to the highest confidence value.

[0066] After performing dimensional alignment and standardization on the above three features, a multimodal fusion strategy is used to generate multimodal fusion features.

[0067] In one embodiment, a feature-level concatenation and fusion method is used, which involves directly connecting three feature vectors in a predetermined order to form a joint feature vector: To avoid dominant bias caused by differences in dimensions between modalities, all input features are subjected to L2 normalization or Min-Max standardization before fusion.

[0068] In another embodiment, attention-guided weighted fusion can also be used: a lightweight gating network is constructed, which uses environmental features as contextual input and dynamically generates adjustment weights for the confidence of the visual and temporal branch outputs, thereby achieving environmentally adaptive fusion decision-making; or a decision-level fusion strategy can be used, in which each modality independently outputs preliminary classification results, and then combines a rule engine or Bayesian inference model for final decision-making.

[0069] In another embodiment, sub-classification models are trained independently based on visual features, environmental features, and temporal features, respectively, and output the confidence scores corresponding to the visual features, environmental features, and temporal features. Based on the confidence scores corresponding to the visual features, a first weight corresponding to the visual features is determined; based on the confidence scores corresponding to the environmental features, a second weight corresponding to the environmental features is determined; based on the confidence scores corresponding to the temporal features, a third weight corresponding to the temporal features is determined; and based on the first weight, second weight, and third weight, the visual features, environmental features, and temporal features are weighted and fused to obtain multimodal fusion features.

[0070] The resulting multimodal fusion features are input into a pre-trained multimodal classification model for final state recognition. The multimodal classification model can be selected from structures such as fully connected neural networks (MLP), support vector machines (SVM), random forests, or Transformer classifiers. Its output layer is configured with multiple nodes, each corresponding to multiple predefined in-vehicle display state categories, including but not limited to: normal screen on, black screen, gray screen, distorted screen, frozen screen, color cast, no signal, startup, and low brightness display.

[0071] The multimodal classification model performs a nonlinear mapping on the received multimodal fusion features and outputs a confidence score for each category, representing the probability that the current in-vehicle display device is in that category. For example, the output is in the form of a probability distribution vector.

[0072] Subsequently, the highest confidence value is obtained, and the final result of this detection is determined based on the category corresponding to the highest confidence value.

[0073] In a preferred embodiment, a confidence threshold judgment mechanism is also provided: when the highest confidence value is lower than the set threshold, it is marked as suspicious and a prompt message is output for manual review. The judgment result is only accepted when the highest confidence value is higher than the set threshold.

[0074] Furthermore, the multimodal classification model undergoes supervised training using a large-scale labeled dataset. The training samples cover various vehicle models, screen resolutions, operating system versions, typical fault scenarios, and real-world data under different environmental conditions, ensuring the model possesses good generalization ability and robustness. During training, Focal Loss is introduced to address the class imbalance problem, and data augmentation techniques such as simulated noise injection, brightness perturbation, and frame loss are combined to improve the model's ability to identify edge cases.

[0075] Finally, the test results, along with the confidence scores for each modality, original image snapshots, and environmental parameters, are recorded in the test report database for subsequent quality analysis, fault tracing, and system optimization.

[0076] The multimodal fusion architecture proposed in this application significantly improves the accuracy, stability, and intelligence of in-vehicle display devices.

[0077] Please refer to Figure 5 This application embodiment also provides an application for Figure 1 The vehicle-mounted display device detection device 110 of the electronic device 100 includes: The acquisition module 111 is used to acquire video signals from the vehicle display device through the video capture card and convert them into a standard image frame sequence; and to acquire the vehicle system serial port logs. The processing module 112 is used to preprocess the standard image frame sequence; determine visual features based on the preprocessed standard image frame sequence; determine temporal features based on the visual features; determine environmental features based on the vehicle system serial port log; and determine the detection result of the vehicle display device based on the visual features, environmental features, and temporal features.

[0078] This application also provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer-executable instructions, which, when executed by the processor 130, implement the vehicle-mounted display device detection method.

[0079] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by the processor 130, implements the vehicle-mounted display device detection method.

[0080] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0081] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0082] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0083] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for testing vehicle-mounted display devices, characterized in that, The method includes: The video signal is acquired from the vehicle display device via a video capture card and converted into a standard image frame sequence; Obtain the vehicle infotainment system serial port log; The standard image frame sequence is preprocessed; Visual features are determined based on the preprocessed standard image frame sequence; The temporal characteristics of the preprocessed standard image frame sequence are determined based on the visual features. Environmental characteristics are determined based on the vehicle infotainment system serial port logs; The detection results of the in-vehicle display device are determined based on the visual features, environmental features, and temporal features.

2. The method according to claim 1, characterized in that, The step of determining visual features based on the preprocessed standard image frame sequence includes: The image features corresponding to the preprocessed standard image frame sequence are determined, wherein the image features include average brightness, brightness variance, contrast, color ratio, texture complexity, and edge density. The preprocessed standard image frame sequence is subjected to high-dimensional spatial feature extraction using a convolutional neural network (CNN). The high-dimensional spatial features include feature vectors that characterize the spatial texture, edge structure, and color distribution of the image. The image features and the high-dimensional spatial features are used as the visual features of the preprocessed standard image frame sequence.

3. The method according to claim 1, characterized in that, The step of determining the temporal features of the preprocessed standard image frame sequence based on the visual features includes: Visual features from multiple consecutive frames are input into a recurrent neural network (RNN) or a long short-term memory network (LSTM) to model the temporal features of image changes.

4. The method according to claim 1, characterized in that, The step of determining the detection result of the in-vehicle display device based on the visual features, environmental features, and temporal features includes: The visual features, environmental features, and temporal features are fused to form a multimodal fusion feature; The multimodal fusion features are input into the trained multimodal classification model, and the confidence values ​​of the vehicle display device under multiple predefined categories are output. Obtain the highest confidence value and determine the detection result based on the category corresponding to the highest confidence value.

5. The method according to claim 4, characterized in that, The step of fusing the visual features, environmental features, and temporal features to form a multimodal fusion feature includes: The visual features, environmental features, and temporal features are concatenated to obtain multimodal fusion features.

6. The method according to claim 4, characterized in that, The step of fusing the visual features, environmental features, and temporal features to form a multimodal fusion feature includes: Sub-classification models are trained independently based on visual features, environmental features, and temporal features, respectively, and the confidence scores corresponding to visual features, environmental features, and temporal features are output. Based on the confidence level corresponding to the visual feature, determine the first weight corresponding to the visual feature; Based on the confidence level corresponding to the environmental feature, determine the second weight corresponding to the environmental feature; Based on the confidence level corresponding to the time series feature, determine the third weight corresponding to the time series feature; Based on the first weight, the second weight, and the third weight, the visual features, environmental features, and temporal features are weighted and fused to obtain multimodal fusion features.

7. The method according to claim 4, characterized in that, The method further includes: Compare the highest confidence value with a set threshold. If the highest confidence level is lower than the set threshold, it is marked as suspicious and a prompt message is output for manual review.

8. A vehicle-mounted display device testing device, characterized in that, The device includes: The acquisition module is used to acquire video signals from the in-vehicle display device through the video capture card and convert them into a standard image frame sequence; and to acquire the serial port logs of the vehicle system. The processing module is used to preprocess the standard image frame sequence; determine visual features based on the preprocessed standard image frame sequence; determine temporal features based on the visual features; determine environmental features based on the vehicle system serial port logs; and determine the detection result of the vehicle display device based on the visual features, environmental features, and temporal features.

9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-7.

Images