Information display method and device of vehicle-mounted interaction system, storage medium and electronic equipment

Abstract

The invention relates to an information display method of a vehicle-mounted interaction system, an information display device of the vehicle-mounted interaction system, a storage medium and electronic equipment. The information display method of the vehicle-mounted interaction system is applied to the field of rail transit control, and comprises the following steps: receiving data information sent by a vehicle-mounted ATP system, and obtaining corresponding initial information needing to be updated and displayed; based on the initial information and an information index constructed by an HMI interface display partition index table and / or a storage information index table, determining to-be-displayed information corresponding to the initial information and a display area for positively displaying the to-be-displayed information in the vehicle-mounted HMI system; intercepting first image information of the to-be-displayed information and forward displaying second image information of the to-be-displayed information through the vehicle-mounted HMI system; through the MSSIM model and the dHash model, the first image information and the second image information are subjected to similarity verification, and whether the to-be-displayed information is correctly displayed in the display area of the vehicle-mounted HMI system is determined, so that the display correctness of the image information is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of rail transit control, and in particular, to an information display method for an on-vehicle interaction system, an information display device for an on-vehicle interaction system, a storage medium, and an electronic device. Background Art

[0002] The human-machine interface (HMI) is mainly responsible for displaying various information and parameters during the operation of the train, such as speed, position, fault code, etc. It is like the "eyes" of the train, enabling the train driver to intuitively understand the working state of the train. The HMI usually uses a large display screen to ensure that the driver can clearly view the information under any circumstances. Usually, the on-vehicle controller (VOBC) transmits information to the HMI system periodically through the network, and the HMI system changes the interface image according to the information received periodically to perform on-vehicle system interaction. Among them, the effectiveness of image information display in the HMI system is crucial for the smooth progress of on-vehicle system interaction. Summary of the Invention

[0003] In view of this, embodiments of the present disclosure are expected to provide an information display method for an on-vehicle interaction system, an information display device for an on-vehicle interaction system, a storage medium, and an electronic device.

[0004] The technical solution of the present disclosure is implemented as follows:

[0005] In a first aspect, the present disclosure provides an information display method for an on-vehicle interaction system.

[0006] The information display method for an on-vehicle interaction system provided by the embodiments of the present disclosure includes:

[0007] Establish a storage information index table for the image feature information of the on-vehicle HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route;

[0008] Establish an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, a first index Id_p, a partition type Type_h, a partition Id, and a regional coordinate Z(x, y);

[0009] Receive data information sent by the on-vehicle ATP system, and parse the data information to obtain the corresponding initial information that needs to be updated and displayed;

[0010] Based on the initial information and the information index constructed by the HMI interface display partition index table and / or the storage information index table, determine the information to be displayed corresponding to the initial information and the display area for positively displaying the information to be displayed in the on-vehicle HMI system;

[0011] The to-be-displayed information is displayed in the display area of the in-vehicle HMI system, and a first image information of the to-be-displayed information that has not been positively displayed by the in-vehicle HMI system and a second image information of the to-be-displayed information that has been positively displayed by the in-vehicle HMI system are intercepted;

[0012] The first image information and the second image information are subjected to similarity verification through the MSSIM model and the dHash model to determine whether the to-be-displayed information is correctly displayed in the display area of the in-vehicle HMI system.

[0013] In some embodiments, determining the to-be-displayed information corresponding to the initial information and the display area for positively displaying the to-be-displayed information in the in-vehicle HMI system based on the initial information and the information index constructed from the HMI interface display partition index table and / or the storage information index table includes:

[0014] If the initial information is text information, determining the initial information as the to-be-displayed information, and performing information indexing based on the HMI interface display partition index table to determine the display area for positively displaying the initial information in the in-vehicle HMI system;

[0015] If the initial information is image information, performing information indexing based on the storage information index table to obtain the to-be-displayed information corresponding to the initial information in the information storage area, and performing information indexing based on the HMI interface display partition index table to determine the display area for positively displaying the to-be-displayed information corresponding to the initial information in the in-vehicle HMI system.

[0016] In some embodiments, performing information indexing based on the HMI interface display partition index table to determine the display area for positively displaying the initial information in the in-vehicle HMI system includes:

[0017] Based on the initial information, obtaining the partition type Type_h in the HMI interface display partition index table;

[0018] According to the partition type Type_h, obtaining the area coordinates Z(x, y);

[0019] According to the area coordinates Z(x, y), determining the display area for positively displaying the initial information in the in-vehicle HMI system.

[0020] In some embodiments, performing information indexing based on the storage information index table to obtain the to-be-displayed information corresponding to the initial information in the information storage area includes:

[0021] Based on the initial information, retrieve the first index Id_p and storage path Route according to the name Name in the storage information index table to obtain the information to be displayed corresponding to the initial information;

[0022] The information indexing based on the HMI interface display partition index table to determine the display area for the information to be displayed corresponding to the initial information to be forward displayed in the vehicle-mounted HMI system includes:

[0023] Based on the initial information, obtain the area coordinates Z(x, y) according to the second index Id_p in the HMI interface display partition index table;

[0024] According to the area coordinates Z(x, y), determine the display area for the information to be displayed corresponding to the initial information to be forward displayed in the vehicle-mounted HMI system.

[0025] In some embodiments, the similarity verification of the first image information and the second image information by the MSSIM model and the dHash model to determine whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system includes:

[0026] Perform iterative operation of image similarity by the MSSIM model to determine the similarity value between the first image information and the second image information;

[0027] If the similarity value between the first image information and the second image information is within the judgment threshold range, determine that the first image information and the second image information have a first similarity;

[0028] If the first image information and the second image information have a first similarity, determine the difference value between the pixels in the same area of the first image information and the second image information based on the dHash model;

[0029] If the difference value between the pixels in the same area of the first image information and the second image information is zero, determine that the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system.

[0030] In some embodiments, the similarity verification of the first image information and the second image information by the MSSIM model and the dHash model to determine whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system includes:

[0031] If the similarity verification of the first image information and the second image information by the MSSIM model and the dHash model determines that the information to be displayed is not correctly displayed in the display area of the vehicle-mounted HMI system, output an alarm message to the vehicle-mounted ATP system.

[0032] In some embodiments, establishing a storage information index table for the image feature information of the in-vehicle HMI system includes:

[0033] Performing different directory management on original pictures of different types in the storage area of the in-vehicle HMI system to generate target information;

[0034] Configuring the directory information into the storage information index table;

[0035] Establishing an HMI interface display partition index table includes:

[0036] Partitioning the display area of the HMI interface into an alarm area, a text area, and a picture area;

[0037] Based on the area coordinates corresponding to each partition of the display area of the HMI interface, establishing the HMI interface display partition index table.

[0038] In a second aspect, the present disclosure provides an information display device for an in-vehicle interaction system, including:

[0039] A first information processing module for establishing a storage information index table for the image feature information of the in-vehicle HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route;

[0040] A second information processing module for establishing an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, a first index Id_p, a partition type Type_h, a partition Id, and area coordinates Z(x, y);

[0041] An information parsing module for receiving data information sent by the in-vehicle ATP system and parsing the data information to obtain corresponding initial information that needs to be updated and displayed;

[0042] A to-be-displayed information determination module for determining the to-be-displayed information corresponding to the initial information and the display area for positively displaying the to-be-displayed information in the in-vehicle HMI system based on the initial information and the information index constructed by the HMI interface display partition index table and / or the storage information index table;

[0043] An information display module for displaying the to-be-displayed information in the display area of the in-vehicle HMI system, and intercepting first image information of the to-be-displayed information that has not been positively displayed by the in-vehicle HMI system and second image information of the to-be-displayed information that has been positively displayed by the in-vehicle HMI system;

[0044] A similarity verification module is configured to perform similarity verification on the first image information and the second image information through the MSSIM model and the dHash model, and determine whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system.

[0045] In a third aspect, the present disclosure provides a computer-readable storage medium, on which an information display program of a vehicle-mounted interaction system is stored. When the information display program of the vehicle-mounted interaction system is executed by a processor, the information display method of the vehicle-mounted interaction system described in the first aspect above is implemented.

[0046] In a fourth aspect, the present disclosure provides an electronic device, including a memory, a processor, and an information display program of a vehicle-mounted interaction system stored on the memory and executable on the processor. When the processor executes the information display program of the vehicle-mounted interaction system, the information display method of the vehicle-mounted interaction system described in the first aspect above is implemented.

[0047] The information display method of the vehicle-mounted interaction system provided by the embodiments of the present disclosure includes: establishing a storage information index table for the image feature information of the vehicle-mounted HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route; establishing an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, the first index Id_p, a partition type Type_h, a partition Id, and an area coordinate Z(x, y); receiving data information sent by the vehicle-mounted ATP system, and parsing the data information to obtain corresponding initial information that needs to be updated and displayed; based on the initial information and the information index constructed by the HMI interface display partition index table and / or the storage information index table, determining the information to be displayed corresponding to the initial information and the display area for positively displaying the information to be displayed in the vehicle-mounted HMI system; displaying the information to be displayed in the display area of the vehicle-mounted HMI system, and intercepting the first image information of the information to be displayed that has not been positively displayed by the vehicle-mounted HMI system and the second image information of the information to be displayed that has been positively displayed by the vehicle-mounted HMI system; performing similarity verification on the first image information and the second image information through the MSSIM model and the dHash model, and determining whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system. By establishing a storage information index table for the image feature information of the vehicle-mounted HMI system and an HMI interface display partition index table, the present application realizes the image display of the information received from the vehicle-mounted ATP system, and performs image display correctness verification through the MSSIM model and the dHash model, which is beneficial to improving the correctness of image information display and further beneficial to the smooth interaction of the vehicle-mounted system.

[0048] Additional aspects and advantages of the present disclosure will be given in part in the following description, become apparent in part from the following description, or be learned by practice of the present disclosure. Description of the Drawings

[0049] Figure 1 is a flowchart of an information display method for an in-vehicle interaction system shown according to an exemplary embodiment;

[0050] Figure 2 is a schematic diagram of a display area partition of an in-vehicle interaction system shown according to an exemplary embodiment;

[0051] Figure 3 is a flowchart of an information display of an in-vehicle interaction system shown according to an exemplary embodiment;

[0052] Figure 4 is a schematic structural diagram of an information display device of an in-vehicle interaction system shown according to an exemplary embodiment. Detailed Description of the Embodiments

[0053] Embodiments of the present disclosure will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where like or similar reference numerals denote like or similar elements or elements having like or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure and should not be construed as limiting the present disclosure.

[0054] The human-machine interface (HMI) is mainly responsible for displaying various types of information and parameters during train operation, such as speed, position, fault codes, etc. It is like the "eyes" of the train, allowing the train driver to intuitively understand the working state of the train. The HMI usually uses a large display screen to ensure that the driver can clearly view the information under any circumstances. Usually, the vehicle on-board controller (VOBC) transmits information to the HMI system periodically through the network, and the HMI system changes the interface image according to the information received periodically to conduct in-vehicle system interaction. Among them, the effectiveness of image information display in the HMI system is crucial for the smooth progress of in-vehicle system interaction.

[0055] In view of the above situation, the present disclosure provides an information display method for an in-vehicle interaction system. Figure 1 is a flowchart of an information display method for an in-vehicle interaction system shown according to an exemplary embodiment. As Figure 1 shown, the information display method of the in-vehicle interaction system includes:

[0056] Step 10: Establish a storage information index table for the image feature information of the in-vehicle HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route;

[0057] Step 11, establish an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, a first index Id_p, a partition type Type_h, a partition Id, and regional coordinates Z(x, y).

[0058] Step 12, receive the data information sent by the on-vehicle ATP system, and parse the data information to obtain the initial information that needs to be updated and displayed accordingly.

[0059] Step 13, based on the initial information and the information index constructed from the HMI interface display partition index table and / or the storage information index table, determine the information to be displayed corresponding to the initial information and the display area for positively displaying the information to be displayed in the on-vehicle HMI system.

[0060] Step 14, display the information to be displayed in the display area of the on-vehicle HMI system, and capture the first image information of the information to be displayed that has not been positively displayed by the on-vehicle HMI system and the second image information of the information to be displayed that has been positively displayed by the on-vehicle HMI system.

[0061] Step 15, perform similarity verification on the first image information and the second image information through the MSSIM (Mean Structural SIMilarity) model and the dHash mean hash perception model to determine whether the information to be displayed is correctly displayed in the display area of the on-vehicle HMI system.

[0062] In this exemplary embodiment, when establishing a storage information index table for the image feature information of the on-vehicle HMI system, the interface display information of the HMI can be classified into picture type and text type. Among them, for the classification of the picture type, it includes: normal information, abnormal information, and alarm information. Manage different types of original pictures in different directories in the storage area of the on-vehicle HMI system, and configure the directory information into an index table index_table_p. The index table includes an index Id_p, a name Name, an image type Type_p, and a storage path Route.

[0063] Figure 2 It is a schematic diagram of the display area partition of the on-vehicle interaction system shown according to an exemplary embodiment. As Figure 2As shown, when establishing the display partition index table of the HMI interface, the display information is partitioned into: an alarm area, a text area, and a picture area. The index tables index_table_h for each partition are established. The index table includes an index Id_h, an index Id_p, a partition type Type_h, a partition Id, and regional coordinates Z(x, y); among them, the partition Id is configured as the starting byte position of each field of the communication interface. Among them, the alarm area is used to display alarm information, the text area is used for text display and interactive information, and the picture area is used for picture display and interactive information.

[0064] After the storage information index table of the vehicle-mounted HMI system image feature information and the HMI interface display partition index table are established, an RSSP-I security protocol link between the vehicle-mounted ATP system and the vehicle-mounted HMI system is established. By integrating the RSSP-I security protocol between the two systems, connecting through the UDP network communication method, and configuring the communication cycle between the systems. Through the sequence number, timestamp, timeout protection, source and destination identification, feedback message, and double verification of the RSSP-I protocol, the communication data is processed for duplication, loss, insertion, out-of-order, error, and timeout, ensuring that both communication parties obtain the latest and correct communication data.

[0065] Among them, when the vehicle-mounted HMI system performs forward image display, the vehicle-mounted HMI system can periodically receive data information from the vehicle-mounted ATP. Then, by parsing the data information, the corresponding information that needs to be updated for display is obtained.

[0066] This application realizes the image display of the information received from the vehicle-mounted ATP system by establishing the storage information index table of the vehicle-mounted HMI system image feature information and the HMI interface display partition index table, and performs image display correctness verification through the MSSIM model and the dHash model, which is beneficial to improving the correctness of image information display and further beneficial to the smooth interaction of the vehicle-mounted system.

[0067] In some embodiments, determining the information to be displayed corresponding to the initial information and the display area for forward display of the information to be displayed in the vehicle-mounted HMI system based on the initial information and the information index constructed by the HMI interface display partition index table and / or the storage information index table includes:

[0068] If the initial information is text information, determine that the initial information is the information to be displayed, and perform information indexing based on the HMI interface display partition index table to determine the display area where the initial information is forward displayed in the vehicle-mounted HMI system;

[0069] If the initial information is image information, perform information indexing based on the storage information index table, obtain the information to be displayed corresponding to the initial information in the information storage area, and perform information indexing based on the HMI interface display partition index table to determine the display area for positive display of the information to be displayed corresponding to the initial information in the in-vehicle HMI system.

[0070] In this exemplary embodiment, the information indexing based on the HMI interface display partition index table to determine the display area for positive display of the initial information in the in-vehicle HMI system includes:

[0071] Based on the initial information, obtain the partition type Type_h in the HMI interface display partition index table;

[0072] According to the partition type Type_h, obtain the area coordinates Z(x, y);

[0073] According to the area coordinates Z(x, y), determine the display area for positive display of the initial information in the in-vehicle HMI system.

[0074] In this exemplary embodiment, when the initial information is text information, the display area for text information display can be directly obtained. For example, for text information, according to the interface input information, obtain the partition type Type_h from the index_table_h index table. According to the partition type Type_h, obtain the area coordinates Z(x, y). Perform regional text display on the display interface of the in-vehicle HMI system according to the area coordinates Z(x, y). In this way, the information display of text information on the display interface of the in-vehicle HMI system can be realized.

[0075] In some embodiments, the information indexing based on the storage information index table to obtain the information to be displayed corresponding to the initial information in the information storage area includes:

[0076] Based on the initial information, retrieve the first index Id_p and the storage path Route according to the name Name in the storage information index table to obtain the information to be displayed corresponding to the initial information;

[0077] The information indexing based on the HMI interface display partition index table to determine the display area for positive display of the information to be displayed corresponding to the initial information in the in-vehicle HMI system includes:

[0078] Based on the initial information, obtain the area coordinates Z(x, y) according to the second index Id_p in the HMI interface display partition index table;

[0079] According to the regional coordinates Z(x, y), determine the display area for forward display of the information to be displayed corresponding to the initial information on the in-vehicle HMI system.

[0080] In this exemplary embodiment, when the initial information is a picture, the index Id_p and the storage path Route can be retrieved from index_table_p according to the name Name, and the original picture of the initial information, that is, the information to be displayed corresponding to the initial information, can be obtained through the storage path Route.

[0081] Secondly, obtain the regional coordinates Z(x, y) (coordinate position of the HMI storage area) from the index_table_h index table, and check the partition type Type_h; finally, display the picture in a divided area on the display interface of the in-vehicle HMI system according to the regional coordinates Z(x, y). In this way, the display of picture information on the display interface of the in-vehicle HMI system can be realized.

[0082] For example, according to the parsing of data information, it is determined that the initial information is that the door has been safely closed, then the information to be displayed corresponding to the initial information is a picture of the door having been safely closed. An image of "a picture of the door having been safely closed" is displayed in the picture area on the display interface of the in-vehicle HMI system. Among them, the first image information is the "picture of the door having been safely closed" obtained through indexing and not yet displayed by the in-vehicle HMI system, and the second image information is a screenshot of the "picture of the door having been safely closed" displayed by the in-vehicle HMI system.

[0083] In some embodiments, the method of using the MSSIM model and the dHash model to perform similarity verification on the first image information and the second image information to determine whether the information to be displayed is correctly displayed in the display area of the in-vehicle HMI system includes:

[0084] Perform iterative operations on the image similarity through the MSSIM model to determine the similarity value between the first image information and the second image information;

[0085] If the similarity value between the first image information and the second image information is within the judgment threshold range, it is determined that the first image information and the second image information have a first similarity;

[0086] If the first image information and the second image information have a first similarity, then based on the dHash model, determine the difference value between the pixels in the same area of the first image information and the second image information;

[0087] If the difference value between the pixels in the same area of the first image information and the second image information is zero, it is determined that the information to be displayed is correctly displayed in the display area of the in-vehicle HMI system.

[0088] In this exemplary embodiment, the forward display image information of the in-vehicle HMI system is intercepted, and heterogeneous recognition fusion is performed through the MSSIM and dHash algorithms. For the picture part, first, the forward display image is intercepted according to the data information. By the correspondence between the starting byte position of each field of the communication interface and the partition Id in the index table index_table_h, the regional coordinates Z(x, y) of the partition corresponding to the index table index_table_h are found, and the intercepted image is used as the comparison image P1 (i.e., the second image information). The index Id_p is obtained according to the partition Id in the index table index_table_h, and the storage path Route of the index table index_table_p is found to obtain the original image P0 (i.e., the first image information). It can be understood that the first image information and the second image information having a first similarity in this application means that the first image information and the second image information are extremely similar or very similar.

[0089] Secondly, through the MSSIM algorithm Three iterations are established, where

[0090] In the first and second iterations, the contrast and the structural similarity are calculated respectively. In the third iteration, the brightness is calculated. The judgment threshold is set to [0.9999, 1]. After three iterations of calculation, the similarity value MSSIM(P0, P1) is obtained. When the similarity value MSSIM(P0, P1) is within the threshold range, it is determined that P0 and P1 have a first similarity. Among them, the three iterations of calculation of the MSSIM algorithm in this application are common iterative calculation methods and will not be elaborated in this application.

[0091] Finally, verification is performed through the dHash algorithm. The P0 and P1 pictures are grayscale processed, and the grayscale images are divided into multiple regions;

[0092] According to the difference rule between each pixel and the pixel below it, the difference values of adjacent pixels in each region are calculated;

[0093] A hash code is constructed based on the difference value array, and further calculation of the hash encoding is realized by comparing the size relationship between each difference value and the average difference value, obtaining dHash1 and dHash0;

[0094] The Hamming distances HammD0 and HammD1 of the two hash values dHash1 and dHash0 are calculated; among them, the difference value between the two pictures is determined by SimilarityP = HammD0 - HammD1;

[0095] When the difference value SimilarityP is zero, it is determined that the picture has the first similarity. When SimilarityP is greater than 0, it is determined that the picture does not have the first similarity. When the similarity value MSSIM(P0, P1) between the first image information and the second image information is within [0.9999, 1] and the difference value between the first image information and the second image information is equal to zero, it is determined that the display area of the vehicle-mounted HMI system correctly displays the information to be displayed.

[0096] For the text part, first, perform partition forward display image capture according to the interface information. Through the correspondence between the starting byte position of each field of the communication interface and the partition Id in the index table index_table_h, find the area coordinates Z(x, y) of the partition corresponding to the index table index_table_h, and capture the image as the comparison image P1. Secondly, through the character recognition algorithm, perform image preprocessing, character segmentation, and feature extraction on the text in the image to complete the recognition of the text in the image. Finally, perform interface information comparison according to the extracted text.

[0097] When SimilarityP is equal to zero and MSSIM(P0, P1) is between [0.9999, 1], it is considered that the vehicle-mounted HMI forward display image information is correct; when it does not satisfy that SimilarityP is equal to zero and MSSIM(P0, P1) is between [0.9999, 1], it is determined that the vehicle-mounted HMI forward display image information is inaccurate.

[0098] In this exemplary embodiment, the method of using the MSSIM model and the dHash model to perform similarity verification on the first image information and the second image information to determine whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system includes:

[0099] If it is determined through the similarity verification of the first image information and the second image information by the MSSIM model and the dHash model that the display area of the vehicle-mounted HMI system does not correctly display the information to be displayed, an alarm message is output to the vehicle-mounted ATP system.

[0100] In some embodiments, the establishment of the storage information index table for the image feature information of the vehicle-mounted HMI system includes:

[0101] Perform different directory management on the original pictures of different types in the storage area of the vehicle-mounted HMI system to generate target information;

[0102] Configure the directory information into the storage information index table;

[0103] The establishment of the HMI interface display partition index table includes:

[0104] Partition the HMI interface display area into an alarm area, a text area, and a picture area;

[0105] Based on the regional coordinates corresponding to each partition of the HMI interface display area, establish an HMI interface display partition index table.

[0106] In this exemplary embodiment, an HMI interface display partition index table is established: the display information is divided into an alarm area, a text area, and a picture area, and an index table index_table_h for each partition is established, including index Id_h, index Id_p, partition type Type_h, partition Id (configured as the starting byte position of each field of the communication interface), and regional coordinates Z(x, y), etc. In this way, it is convenient to perform information indexing required for human-machine interaction through the HMI interface display partition index table.

[0107] Figure 3 It is a flowchart of information display of an in-vehicle interaction system shown according to an exemplary embodiment. As Figure 3 shown, the information display process of the in-vehicle interaction system includes:

[0108] Step 20: Establish an in-vehicle HMI system image feature information storage and retrieval table;

[0109] Step 21: Establish an HMI display interface partition retrieval table;

[0110] Step 22: Establish an RSSP-I security protocol link between the in-vehicle ATP system and the in-vehicle HMI system;

[0111] Step 23: The in-vehicle HMI system performs forward image display;

[0112] Step 24: Obtain the forward display image information of the in-vehicle HMI system, and perform heterogeneous recognition and fusion through the MSSIM algorithm and the dHash algorithm;

[0113] Step 25: Output alarm information for the detected abnormal image. In this way, alarm information can be output through the abnormal image to improve the timeliness of abnormal situation alarm, thereby improving the operating safety of the system.

[0114] The present disclosure provides an information display device for an in-vehicle interaction system. Figure 4 It is a schematic structural diagram of an information display device for an in-vehicle interaction system shown according to an exemplary embodiment. As Figure 4 shown, the information display device for the in-vehicle interaction system includes:

[0115] The first information processing module 40 is used to establish a storage information index table for the image feature information of the in-vehicle HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route;

[0116] The second information processing module 41 is used to establish an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, a first index Id_p, a partition type Type_h, a partition Id, and area coordinates Z(x, y);

[0117] The information parsing module 42 is used to receive the data information sent by the in-vehicle ATP system and parse the data information to obtain the corresponding initial information that needs to be updated and displayed;

[0118] The to-be-displayed information determination module 43 is used to determine the to-be-displayed information corresponding to the initial information and the display area for positively displaying the to-be-displayed information in the in-vehicle HMI system based on the initial information and the information index constructed by the HMI interface display partition index table and / or the storage information index table;

[0119] The information display module 44 is used to display the to-be-displayed information in the display area of the in-vehicle HMI system, and intercept the first image information of the to-be-displayed information that has not been positively displayed by the in-vehicle HMI system and the second image information of the to-be-displayed information that has been positively displayed by the in-vehicle HMI system;

[0120] The similarity verification module 45 is used to verify the similarity of the first image information and the second image information through the MSSIM model and the dHash model to determine whether the to-be-displayed information is correctly displayed in the display area of the in-vehicle HMI system.

[0121] In this exemplary embodiment, when establishing the storage information index table for the image feature information of the in-vehicle HMI system, the interface display information of the HMI can be classified into picture type and text type. Among them, for the picture type classification, it includes: normal information, abnormal information, and alarm information. Different types of original pictures are managed in different directories in the storage area of the in-vehicle HMI system, and the directory information is configured into an index table index_table_p. The index table includes an index Id_p, a name Name, an image type Type_p, and a storage path Route.

[0122] When establishing the display partition index table of the HMI interface, the display information is partitioned into: an alarm area, a text area, and a picture area. An index table index_table_h for each partition is established. The index table includes an index Id_h, an index Id_p, a partition type Type_h, a partition Id, and area coordinates Z(x, y); among them, the partition Id is configured as the starting byte position of each field of the communication interface.

[0123] After the storage information index table of the image feature information of the in-vehicle HMI system and the display partition index table of the HMI interface are established, an RSSP-I security protocol link between the in-vehicle ATP system and the in-vehicle HMI system is established. By integrating the RSSP-I security protocol between the two systems, connecting through the UDP network communication method, and configuring the communication cycle between the systems. Through the sequence number, timestamp, timeout protection, source and destination identification, feedback message, and double verification of the RSSP-I protocol, the processing of repetition, loss, insertion, out-of-order, error, and timeout of communication data is realized, ensuring that both communication parties obtain the latest and correct communication data.

[0124] Among them, when the in-vehicle HMI system performs forward image display, the in-vehicle HMI system can periodically receive data information from the in-vehicle ATP. Then, by parsing the data information, the corresponding information that needs to be updated for display is obtained.

[0125] This application realizes the image display of the information received from the in-vehicle ATP system by establishing the storage information index table of the image feature information of the in-vehicle HMI system and the display partition index table of the HMI interface, and performs the correctness verification of the image display through the MSSIM model and the dHash model, which is beneficial to improving the correctness of the image information display and further beneficial to the smooth interaction of the in-vehicle system.

[0126] In some embodiments, the to-be-displayed information determination module is used for

[0127] If the initial information is text information, determine the initial information as the to-be-displayed information, and perform information indexing based on the HMI interface display partition index table to determine the display area where the initial information is displayed in the forward display of the in-vehicle HMI system;

[0128] If the initial information is image information, perform information indexing based on the storage information index table, obtain the to-be-displayed information corresponding to the initial information in the information storage area, and perform information indexing based on the HMI interface display partition index table to determine the display area where the to-be-displayed information corresponding to the initial information is displayed in the forward display of the in-vehicle HMI system.

[0129] In this exemplary embodiment, the to-be-displayed information determination module is used for

[0130] Based on the initial information, obtain the partition type Type_h from the partition index table displayed on the HMI interface;

[0131] According to the partition type Type_h, obtain the area coordinates Z(x, y);

[0132] According to the area coordinates Z(x, y), determine the display area for the positive display of the initial information in the in-vehicle HMI system.

[0133] In this exemplary embodiment, when the initial information is text information, the display area for text information display can be directly obtained. For example, for text information, according to the interface input information, obtain the partition type Type_h from the index_table_h index table. According to the partition type Type_h, obtain the area coordinates Z(x, y). Perform area-based text display on the display interface of the in-vehicle HMI system according to the area coordinates Z(x, y). In this way, the information display of the text information on the display interface of the in-vehicle HMI system can be achieved.

[0134] In some embodiments, the information to be displayed determination module is used for

[0135] Based on the initial information, perform a first search for the first index Id_p and the storage path Route in the stored information index table according to the name Name, and obtain the information to be displayed corresponding to the initial information;

[0136] The method of determining the display area for the positive display of the information to be displayed corresponding to the initial information in the in-vehicle HMI system by performing information indexing based on the HMI interface display partition index table includes:

[0137] Based on the initial information, obtain the area coordinates Z(x, y) from the HMI interface display partition index table according to the second index Id_p;

[0138] According to the area coordinates Z(x, y), determine the display area for the positive display of the information to be displayed corresponding to the initial information in the in-vehicle HMI system.

[0139] In this exemplary embodiment, when the initial information is a picture, the index Id_p and the storage path Route can be retrieved from the index_table_p according to the name Name, and the original picture of the initial information, that is, the information to be displayed corresponding to the initial information, can be obtained through the storage path Route;

[0140] Secondly, obtain the regional coordinates Z(x, y) (coordinate position of the HMI storage area) from the index_table_h index table according to the index Id_p, and check the partition type Type_h; finally, perform regional display of the picture on the display interface of the in-vehicle HMI system according to the regional coordinates Z(x, y). In this way, the display of picture information on the display interface of the in-vehicle HMI system can be realized.

[0141] In some embodiments, the similarity verification module is used for

[0142] Performing iterative operations on the image similarity through the MSSIM model to determine the similarity value between the first image information and the second image information;

[0143] If the similarity value between the first image information and the second image information is within the judgment threshold range, it is determined that the first image information and the second image information have a first similarity;

[0144] If the first image information and the second image information have a first similarity, then based on the dHash model, determine the difference value between the pixels in the same area of the first image information and the second image information;

[0145] If the difference value between the pixels in the same area of the first image information and the second image information is zero, it is determined that the display area of the in-vehicle HMI system correctly displays the information to be displayed.

[0146] In this exemplary embodiment, capture the forward display image information of the in-vehicle HMI system, and perform heterogeneous recognition and fusion through the MSSIM and dHash algorithms. For the picture part, first, perform partition forward display image capture according to the data information. According to the corresponding relationship between the start byte position of each field of the communication interface and the partition Id in the index table index_table_h, find the regional coordinates Z(x, y) of the corresponding partition in the index table index_table_h, and capture the image as the comparison image P1 (i.e., the second image information). Obtain the index Id_p according to the partition Id in the index table index_table_h, and find the storage path Route of the index table index_table_p to obtain the original image P0 (i.e., the first image information).

[0147] Secondly, through the MSSIM algorithm Establish three iterations, where

[0148] In the first and second iterations, calculate the contrast and the structural similarity The third iteration calculates the brightness Set the judgment threshold to [0.9999, 1]. After three iterations of calculation, the similarity value MSSIM(P0, P1) is obtained. When the similarity value MSSIM(P0, P1) is within the threshold range, it is determined that P0 and P1 have the first similarity. Among them, the three iterations of the MSSIM algorithm in this application are common iterative calculation methods and will not be elaborated in this application.

[0149] Finally, perform verification through the dHash algorithm. Process the P0 and P1 images into grayscale, and divide the grayscale image into multiple regions;

[0150] Calculate the difference value of adjacent pixels in each region according to the difference rule between each pixel and the pixel below it;

[0151] Construct a hash code based on the difference value array, and further calculate the hash encoding by comparing the size relationship between each difference value and the average difference value to obtain dHash1 and dHash0;

[0152] Calculate the Hamming distances HammD0 and HammD1 of the two hash values dHash1 and dHash0; among them, the difference value between the two images is determined by SimilarityP = HammD0 - HammD1;

[0153] When the difference value SimilarityP is zero, it is determined that the images have the first similarity. When the difference value SimilarityP is greater than 0, it is determined that the images do not have the first similarity. When the similarity value MSSIM(P0, P1) between the first image information and the second image information is within [0.9999, 1], and the difference value between the first image information and the second image information is equal to zero, it is determined that the display area of the in-vehicle HMI system correctly displays the information to be displayed.

[0154] For the text part, first, perform partition forward display image interception according to the interface information. Based on the correspondence between the starting byte position of each field of the communication interface and the partition Id in the index table index_table_h, find the regional coordinates Z(x, y) of the partition corresponding to the index table index_table_h, and intercept the image as the comparison image P1; secondly, through the character recognition algorithm, perform image preprocessing, character segmentation, and feature extraction on the text in the image to complete the recognition of the text in the image; finally, perform interface information comparison according to the extracted text.

[0155] When SimilarityP is equal to zero and MSSIM(P0, P1) is within the range of [0.9999, 1], it is considered that the image information displayed on the in-vehicle HMI is correct; when the condition that SimilarityP is equal to zero and MSSIM(P0, P1) is within the range of [0.9999, 1] is not met, it is determined that the image information displayed on the in-vehicle HMI is inaccurate.

[0156] In some embodiments, the similarity verification module is configured to

[0157] If the first image information and the second image information are verified for similarity through the MSSIM model and the dHash model, and it is determined that the display area of the in-vehicle HMI system does not correctly display the information to be displayed, an alarm message is output to the in-vehicle ATP system.

[0158] In the present exemplary embodiment, the first information processing module is configured to

[0159] Manage different types of original pictures in different directories in the storage area of the in-vehicle HMI system to generate target information;

[0160] Configure the directory information into the storage information index table;

[0161] The establishment of the HMI interface display partition index table includes:

[0162] Partition the HMI interface display area into an alarm area, a text area, and a picture area;

[0163] Based on the regional coordinates corresponding to each partition of the HMI interface display area, establish the HMI interface display partition index table.

[0164] In the present exemplary embodiment, the HMI interface display partition index table is established: the display information is divided into an alarm area, a text area, and a picture area, and an index table index_table_h for each partition is established, including index Id_h, index Id_p, partition type Type_h, partition Id (configured as the starting byte position of each field of the communication interface), and regional coordinates Z(x, y), etc. In this way, it is convenient to perform information indexing required for human-computer interaction through the HMI interface display partition index table.

[0165] The present disclosure provides a computer-readable storage medium, on which an information display program for an in-vehicle interaction system is stored. When the information display program for the in-vehicle interaction system is executed by a processor, the information display method of the in-vehicle interaction system described in the above embodiments is implemented.

[0166] The present disclosure provides an electronic device, including a memory, a processor, and an information display program of an in-vehicle interaction system stored on the memory and executable on the processor. When the processor executes the information display program of the in-vehicle interaction system, the information display method of the in-vehicle interaction system described in each of the above embodiments is implemented.

[0167] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can determine and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0168] It should be understood that each part of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in the memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0169] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0170] In the description of the present disclosure, it should be understood that the orientation or positional relationships indicated by terms such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present disclosure.

[0171] In addition, the terms "first", "second", etc. used in the embodiments of the present disclosure are only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated in this embodiment. Thus, the features defined with terms such as "first", "second", etc. in the embodiments of the present disclosure can explicitly or implicitly indicate that at least one such feature is included in this embodiment. In the description of the present disclosure, the meaning of the word "plurality" is at least two or more, such as two, three, four, etc., unless otherwise specifically defined in the embodiment.

[0172] In the present disclosure, unless otherwise clearly specified or limited in the embodiments, the terms "installed", "connected", "connected with", and "fixed" etc. appearing in the embodiments shall be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or integrated, and can be understood as a mechanical connection, an electrical connection, etc.; of course, it can also be directly connected, or indirectly connected through an intermediate medium, or can be the communication inside two elements, or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific implementation situations.

[0173] In this disclosure, unless otherwise clearly defined and limited, a first feature being “on” or “under” a second feature may mean that the first and second features are in direct contact, or the first and second features are indirectly in contact via an intermediate medium. Also, a first feature being “above”, “over” and “on top of” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply indicates that the first feature has a higher level of height than the second feature. A first feature being “under”, “beneath” and “underneath” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply indicates that the first feature has a lower level of height than the second feature.

[0174] Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An information display method for an in-vehicle interaction system, characterized in that, Including: Establishing a storage information index table for the image feature information of the in-vehicle HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route; Establishing an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, a first index Id_p, a partition type Type_h, a partition Id, and regional coordinates Z(x, y); Receiving data information sent by the in-vehicle ATP system, and parsing the data information to obtain corresponding initial information that needs to be updated and displayed; Based on the initial information and the information index constructed from the HMI interface display partition index table and / or the storage information index table, determining the information to be displayed corresponding to the initial information and the display area for positively displaying the information to be displayed in the in-vehicle HMI system; Displaying the information to be displayed in the display area of the in-vehicle HMI system, and capturing the first image information of the information to be displayed that has not been positively displayed by the in-vehicle HMI system and the second image information of the information to be displayed that has been positively displayed by the in-vehicle HMI system; Performing similarity verification on the first image information and the second image information through the MSSIM model and the dHash model to determine whether the information to be displayed is correctly displayed in the display area of the in-vehicle HMI system.

2. The information display method of the in-vehicle interaction system according to claim 1, characterized in that, The step of determining the information to be displayed corresponding to the initial information and the display area for positively displaying the information to be displayed in the in-vehicle HMI system based on the initial information and the information index constructed from the HMI interface display partition index table and / or the storage information index table includes: If the initial information is text information, determining the initial information as the information to be displayed, and performing information indexing based on the HMI interface display partition index table to determine the display area for positively displaying the initial information in the in-vehicle HMI system; If the initial information is image information, performing information indexing based on the storage information index table to obtain the information to be displayed corresponding to the initial information in the information storage area, and performing information indexing based on the HMI interface display partition index table to determine the display area for positively displaying the information to be displayed corresponding to the initial information in the in-vehicle HMI system.

3. The information display method of the in-vehicle interaction system according to claim 2, characterized in that, The step of performing information indexing based on the HMI interface display partition index table to determine the display area for positively displaying the initial information in the in-vehicle HMI system includes: Based on the initial information, obtaining the partition type Type_h in the HMI interface display partition index table; According to the partition type Type_h, obtaining the regional coordinates Z(x, y); According to the regional coordinates Z(x, y), determining the display area for positively displaying the initial information in the in-vehicle HMI system.

4. The information display method of the vehicle-mounted interaction system according to claim 2, wherein The step of performing information indexing based on the storage information index table to obtain the information to be displayed corresponding to the initial information in the information storage area includes: Based on the initial information, retrieve the first index Id_p and storage path Route according to the name Name in the storage information index table to obtain the information to be displayed corresponding to the initial information; The information indexing based on the HMI interface display partition index table to determine the display area for the forward display of the information to be displayed corresponding to the initial information in the vehicle-mounted HMI system includes: Based on the initial information, obtain the area coordinates Z(x, y) according to the second index Id_p in the HMI interface display partition index table; According to the area coordinates Z(x, y), determine the display area for the forward display of the information to be displayed corresponding to the initial information in the vehicle-mounted HMI system.

5. The information display method of the in-vehicle interaction system according to claim 1, characterized in that, The similarity verification of the first image information and the second image information through the MSSIM model and the dHash model to determine whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system includes: Perform iterative operation of image similarity through the MSSIM model to determine the similarity value between the first image information and the second image information; If the similarity value between the first image information and the second image information is within the judgment threshold range, determine that the first image information and the second image information have the first similarity; If the first image information and the second image information have the first similarity, determine the difference value between the pixels in the same area of the first image information and the second image information based on the dHash model; If the difference value between the pixels in the same area of the first image information and the second image information is zero, determine that the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system.

6. The information display method of the in-vehicle interaction system according to claim 1, wherein The similarity verification of the first image information and the second image information through the MSSIM model and the dHash model to determine whether the information to be displayed is correctly displayed in the display area of the vehicle-mounted HMI system includes: If it is determined through the similarity verification of the first image information and the second image information by the MSSIM model and the dHash model that the information to be displayed is not correctly displayed in the display area of the vehicle-mounted HMI system, output an alarm message to the vehicle-mounted ATP system.

7. The information display method of the in-vehicle interaction system according to claim 1, wherein The establishment of the storage information index table for the image feature information of the vehicle-mounted HMI system includes: Perform different directory management on the original pictures of different types in the storage area of the vehicle-mounted HMI system to generate target information; Configure the directory information into the storage information index table; The establishment of the HMI interface display partition index table includes: Partition the HMI interface display area into an alarm area, a text area, and a picture area; Based on the area coordinates corresponding to each partition of the HMI interface display area, establish the HMI interface display partition index table.

8. An information display device for a vehicle-mounted interaction system, characterized in that, Includes: A first information processing module for establishing a storage information index table for the image feature information of the vehicle-mounted HMI system; wherein, the storage information index table includes a first index Id_p, a name Name, an image type Type_p, and a storage path Route; The second information processing module is used to establish an HMI interface display partition index table; wherein, the HMI interface display partition index table includes a second index Id_h, a first index Id_p, a partition type Type_h, a partition Id, and area coordinates Z(x, y); The information parsing module is used to receive the data information sent by the vehicle-mounted ATP system and parse the data information to obtain the corresponding initial information that needs to be updated and displayed; The to-be-displayed information determination module is used to determine the to-be-displayed information corresponding to the initial information and the display area for positively displaying the to-be-displayed information in the vehicle-mounted HMI system based on the initial information and the information index constructed by the HMI interface display partition index table and / or the storage information index table; The information display module is used to display the to-be-displayed information in the display area of the vehicle-mounted HMI system, and intercept the first image information of the to-be-displayed information that has not been positively displayed by the vehicle-mounted HMI system and the second image information of the to-be-displayed information that has been positively displayed by the vehicle-mounted HMI system; The similarity verification module is used to verify the similarity of the first image information and the second image information through the MSSIM model and the dHash model to determine whether the to-be-displayed information is correctly displayed in the display area of the vehicle-mounted HMI system.

9. A computer-readable storage medium, characterized in that, Stored thereon is an information display program of a vehicle-mounted interaction system. When the information display program of the vehicle-mounted interaction system is executed by a processor, the information display method of the vehicle-mounted interaction system according to any one of claims 1-7 is implemented.

10. An electronic device, characterized in that, It includes a memory, a processor, and an information display program of a vehicle-mounted interaction system stored on the memory and executable on the processor. When the processor executes the information display program of the vehicle-mounted interaction system, the information display method of the vehicle-mounted interaction system according to any one of claims 1-7 is implemented.

Images