Display equipment detection method and device

Abstract

The invention discloses a detection method and device for display equipment. The method comprises the steps of collecting a display image displayed by target display equipment under a control instruction, wherein the display image at least comprises an information indication area, a text display area and an icon display area; comparing the color data of the information indication area with the target color data to obtain an information prompt detection result of performing information prompt detection on the target display equipment; performing text and confidence comparison on the display text recognized in the text display area to obtain a text detection result of performing text display detection on the target display equipment; performing template matching on the icon display area to obtain an icon detection result of performing icon detection on the target display equipment; and generating a target detection result for detecting the target display equipment according to the information prompt detection result, the text detection result and the icon detection result. Therefore, according to the scheme provided by the embodiment of the invention, the detection accuracy of the target display equipment can be improved.

Description

Technical Field

[0001] The present application belongs to the field of automotive electronics technology, and in particular relates to a detection method and apparatus for a display device. Background Art

[0002] A car's instrument panel displays the operating status of various vehicle systems. For example, it can display information such as speed, mileage, engine speed, oil pressure, water temperature, battery level, and fuel level. Accurate dashboard displays allow drivers to quickly understand vehicle status, ensuring personal and property safety. To ensure proper function, dashboard testing is essential before a vehicle is sold.

[0003] In related technologies, the instrument panel and host computer are typically connected via a CAN (Controller Area Network) cable, and the instrument panel's operation is checked by signal detection or visual observation. However, this method has low detection accuracy and carries the risk of false detection or missed detection of the instrument panel. Summary of the Invention

[0004] Embodiments of the present application provide a method and apparatus for detecting a display device, which can improve the detection accuracy of a target display device.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting a display device, the method comprising: acquiring a display image displayed by a target display device under a control instruction, wherein the display image comprises at least: an information indication area, a text display area, and an icon display area; comparing color data of the information indication area with target color data to obtain an information prompt detection result for information prompt detection of the target display device; performing text and confidence comparison on the display text identified in the text display area to obtain a text detection result for text display detection of the target display device; performing template matching on the icon display area to obtain an icon detection result for icon detection of the target display device; generating a target detection result for detecting the target display device based on the information prompt detection result, the text detection result, and the icon detection result.

[0006] In a second aspect, an embodiment of the present application provides a detection device for a display device, the device comprising: an image acquisition module for acquiring a display image displayed by a target display device under a control instruction, wherein the display image comprises at least: an information indication area, a text display area, and an icon display area; a color comparison module for comparing the color data of the information indication area with the target color data to obtain an information prompt detection result for information prompt detection of the target display device; a text comparison module for performing text and confidence comparison on the display text identified in the text display area to obtain a text detection result for text display detection of the target display device; a template matching module for performing template matching on the icon display area to obtain an icon detection result for icon detection of the target display device; a detection module for generating a target detection result for detecting the target display device based on the information prompt detection result, the text detection result, and the icon detection result.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the detection method of the display device as described in the first aspect is implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the detection method for the display device as described in the first aspect is implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the detection method of the display device as described in the first aspect.

[0010] As can be seen from the above content, in the embodiment of the present application, by performing color comparison on the information indication area of the target display device, it is determined whether the prompt information displayed in the information indication area is accurate; by performing text and confidence comparison on the text in the text display area of the target display device, it is determined whether the text in the text display area is correctly displayed; and by performing template matching on the icon display area of the target display device, it is determined whether the icon in the target display area is correctly displayed, thereby detecting the target display device from three aspects: color recognition, text recognition, and icon recognition, so as to improve the detection accuracy of the target display device. In addition, when recognizing text, not only text comparison but also confidence comparison is performed to improve the accuracy of text recognition, thereby improving the detection accuracy of the target display device.

[0011] It can be seen that the solution provided by the embodiment of the present application can improve the detection accuracy of the target display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a schematic diagram of the structure of a detection platform provided by one embodiment of the present application;

[0014] Figure 2 This is a flow chart of a method for detecting a display device provided by one embodiment of the present application;

[0015] Figure 3 is a schematic diagram of a display image provided by an embodiment of the present application;

[0016] Figure 4 is a schematic diagram of a text display area provided by an embodiment of the present application;

[0017] Figure 5 This is an overall flow chart of a detection method for a display device provided by an embodiment of the present application;

[0018] Figure 6 is a schematic diagram of text detection results provided by an embodiment of the present application;

[0019] Figure 7 is a structural schematic diagram of a detection device for a display device provided in another embodiment of the present application;

[0020] Figure 8 This is a structural diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0021] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0022] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0023] For ease of understanding, before explaining the solution provided in this application, the background of the solution provided in this application is first explained.

[0024] In related technologies, when testing a vehicle dashboard, the dashboard is usually connected to a host computer via a CAN line during actual vehicle testing. By acquiring signals, it is determined whether the input and output signals of the dashboard controller are normal, or by directly observing with the naked eye whether the dashboard display is normal.

[0025] In the aforementioned instrument panel testing solution, during the on-vehicle testing phase, testers are required to manually operate buttons on the vehicle's display screen or generate CAN / LIN (Local Interconnect Network) bus signals to interact with the vehicle's computer. This triggers indicator lights on or off, pops up prompts, switches pages, triggers navigation functions, and so on. Afterward, testers are required to manually observe each test result and record whether the test results meet expectations.

[0026] The above-mentioned testing method relies on manpower. Without the subjective observation and record measurement of the tester, it is impossible to conduct a complete and accurate instrument test.

[0027] Secondly, the instrument panel of the actual vehicle can only be tested after the experimental prototype is built, which results in a waste of resources and time to a certain extent.

[0028] Thirdly, in the existing dashboard testing process, the test results cannot be recorded in writing, making it impossible to trace back and review.

[0029] Finally, existing signal-based instrument cluster testing methods determine whether the instrument cluster is functioning properly by checking the CAN / LIN signal transmission and reception status between the vehicle computer and other controllers. In this method, if there is an anomaly in the signal transmission process, the instrument cluster display will be abnormal. However, because the signal sent by the vehicle computer is normal, the instrument cluster will be misjudged as abnormal.

[0030] In order to solve the problems in the prior art, the embodiments of the present application provide a method and apparatus for detecting a display device. The following first introduces the method for detecting a display device provided by the embodiments of the present application.

[0031] The detection method of the display device provided in the embodiment of the present application can be applied to the detection platform. As an example, the detection platform can be as follows Figure 1 As shown, the testing platform comprises at least: an experimental platform 10, a test bench 11, an image acquisition device 13 and a host computer 14. The image acquisition device 13 may be a high-definition camera, and the host computer 14 may be a computing device with data processing and data analysis functions.

[0032] like Figure 1 As shown, a test bench 11 is fixed above an experimental platform 10, on which a display device 12 to be tested is placed. The display device is connected to an integrated controller for simulating status signals via a CAN bus. In an embodiment of the present application, the display device to be tested may be a vehicle dashboard.

[0033] like Figure 1 As shown, an image acquisition device 13 is mounted and fixed on an extension frame above the test bench 11, at a distance L directly above the experimental platform 10. Image acquisition device 13 is connected to a host computer 14 and transmits captured real-time images to the host computer 14. The host computer 14 can capture multiple images containing the instrument panel within a preset time period and store them in its memory for subsequent image processing to achieve instrument panel inspection.

[0034] Figure 2 FIG. 1 is a flow chart showing a method for detecting a display device according to an embodiment of the present application. Figure 2 As shown, the method includes the following steps S201 to S205:

[0035] Step S201 : collecting a display image displayed by a target display device under a control instruction.

[0036] In step S201 , the target display device is a dashboard of a vehicle, and the image acquisition device may acquire an image displayed by the target display device, where the display image only includes the dashboard and does not include other parts.

[0037] In the embodiment of the present application, the control command may be, but is not limited to, an acceleration command, an alarm command, a caller ID command, etc. Under different control commands, the target display device presents different display interfaces. For example, under a caller ID command, the target display device displays the caller number; under a seat belt reminder command, the target display device displays the seat belt indicator light; under a fuel warning command, the target display device displays the fuel indicator light.

[0038] In addition, in step S201, the display image displayed by the target display device includes at least: an information indication area, a text display area, and an icon display area. Figure 3 In the displayed image shown, there are multiple text display areas Q1 and multiple icon display areas Q2, but only one information prompt area Q3.

[0039] Step S202 : comparing the color data of the information indication area with the target color data to obtain an information prompt detection result of performing information prompt detection on the target display device.

[0040] In step S202, the target color data is the color data that the information indication area of the target display device is expected to display under the control instruction. For example, when the control instruction is a vehicle right turn instruction, the information indication area of the target display device should display a green arrow pointing to the right, and the target color data at least includes the color information corresponding to green.

[0041] In step S202, by comparing the color displayed in the information indication area with the color that the information indication area should display under the control instruction, it is determined whether the color display of the information indication area is normal. That is, in this embodiment of the application, the information prompt detection result includes normal color display and abnormal color display.

[0042] Step S203 : performing text and confidence comparison on the displayed text recognized in the text display area to obtain a text detection result of performing text display detection on the target display device.

[0043] In step S203, the displayed text within the text display area is detected, primarily through text recognition and confidence level detection, to improve the accuracy of the text detection results. Similar to the detection of the information indication area, the text detection results include two results: abnormal text display and normal text display. Abnormal text display indicates that the displayed text does not match the target text and / or the confidence level of the displayed text is lower than the confidence threshold.

[0044] Step S204 : performing template matching on the icon display area to obtain an icon detection result for performing icon detection on the target display device.

[0045] In step S204, the icon display area is detected by using template matching, that is, by comparing the icons displayed in the icon display area with the icons that should be displayed in the target display area under the control instruction to obtain icon detection results. The icon detection results include normal icon display and abnormal icon display.

[0046] Step S205 : generating a target detection result for detecting the target display device according to the information prompt detection result, the text detection result, and the icon detection result.

[0047] In step S205 , after the target display device is detected from three aspects, namely, color, text, and icon, and three detection results are obtained, it is determined whether the target display device is normal according to the three detection results.

[0048] Specifically, when the information prompt detection result indicates that the color display in the information indication area is normal, and the text detection result indicates that the text display in the text display area is normal, and the icon detection result indicates that the icon display in the icon display area is normal, it is determined that the target display device is in a normal state; when the information prompt detection result indicates that the color display in the information indication area is abnormal, and / or the text detection result indicates that the text display in the text display area is abnormal, and / or the icon detection result indicates that the icon display in the icon display area is abnormal, it is determined that the target display device is in an abnormal state.

[0049] That is, in the embodiment of the present application, only when the color, text, and icon are displayed normally, is it determined that the target display device is in a normal state, so as to improve the detection accuracy of the target display device.

[0050] It should be noted that the detection of color, text and icon can be performed in sequence, in parallel, or in combination, and there is no specific limitation here.

[0051] Based on the scheme defined in the above steps S201 to S205, it can be known that in the embodiment of the present application, by performing color comparison on the information indication area of the target display device, it is determined whether the prompt information displayed in the information indication area is accurate; by performing text and confidence comparison on the text in the text display area of the target display device, it is determined whether the text in the text display area is correctly displayed; and by performing template matching on the icon display area of the target display device, it is determined whether the icon in the target display area is correctly displayed, thereby detecting the target display device from three aspects: color recognition, text recognition, and icon recognition, so as to improve the detection accuracy of the target display device. In addition, when recognizing text, not only text comparison but also confidence comparison is performed to improve the accuracy of text recognition, thereby improving the detection accuracy of the target display device.

[0052] It can be seen that the solution provided by the embodiment of the present application can improve the detection accuracy of the target display device.

[0053] The following is an introduction to the implementation process of the method provided in the embodiment of the present application.

[0054] The detection of the information indication area mainly includes the following steps: initialization, color recognition, color comparison, and adding the detection results to the detection report. These steps are introduced below.

[0055] During the initialization phase, before inspecting the target display device's information indicator area, the area must first undergo color conversion. Specifically, the inspection platform extracts the information indicator area from the displayed image based on the preset coordinates. The color data for each pixel in the information indicator area is then converted from red-green-blue (RGB) data to hue-saturation-value (HSV) data.

[0056] In the above embodiment, the coordinates of the information indication area can be determined by the coordinates of multiple points in the area outline of the information indication area. For example, when the information indication area is a rectangular area, the coordinates of the information indication area can be the coordinates of two diagonal corners in the rectangular area.

[0057] It should be noted that the detection platform pre-stores the information indication area coordinates corresponding to the information indication areas in target display devices of different types or models. After obtaining the display image captured by the image acquisition device, the information indication area determined by the information indication area coordinates can be extracted from the display image.

[0058] Furthermore, after extracting the information indication area from the displayed image, the pixels in the information indication area are in RGB format. To facilitate subsequent data and improve the accuracy of information prompt area detection, the color data in RGB format is converted to color data in HSV format, where H represents hue, S represents saturation, and V represents lightness. As an example, the conversion between RGB data and HSV data can be achieved by formula (1):

[0059]

[0060] In formula (1), max is the maximum channel value among the three color channels of red r, green g, and blue b; min is the minimum channel value among the three color channels of red r, green g, and blue b; h, s, and v represent the hue value, saturation, and lightness, respectively.

[0061] Furthermore, after completing the color conversion, the detection platform performs color recognition and color comparison on the information indication area. Specifically, based on the correspondence between color types and HSV data ranges, the detection platform collects data statistics on the HSV data of all pixels in the information indication area to obtain the number of pixels corresponding to multiple color types. Then, the detection platform obtains the target color type with the largest number of pixels from the multiple color types. If the color data corresponding to the target color type matches the target color data, the detection platform determines that the color display in the information indication area is normal.

[0062] In the above embodiment, each color type is the type of color displayed by the target display device under different instructions. For example, the HSV data range corresponding to brown is: H∈[6,23], S∈[33,255], V∈[25,168]. If the HSV data of a pixel point is within the above data range, it can be determined that the color corresponding to the pixel point is brown. In addition, the target color data is the color data that the information indication area of the target display device is expected to display under the control instruction. For example, the color of the right turn indicator light is green. When the control instruction is a right turn instruction for the vehicle, the target color data is the HSV data of green.

[0063] In one example, after completing the color conversion, it is necessary to obtain the HSV data range corresponding to each color type in the information indication area, wherein the color type in the information indication area is pre-set, and its corresponding color type includes but is not limited to background color, icon color, and other colors. That is, in the embodiment of the present application, it is necessary to pre-set the HSV data range corresponding to the background color, the HSV data range corresponding to the icon color, and the HSV data range corresponding to the other colors. For example, if the background color is brown, then after the conversion of formula (1), it can be determined that the HSV data range corresponding to brown is: H∈[6,23], S∈[33,255], V∈[25,168]; if the icon color is green, after the conversion of formula 1, then after the conversion of formula (1), it can be determined that the HSV data range corresponding to green is: H∈[32,75], S∈[33,255], V∈[25,255].

[0064] It should be noted that in actual applications, the HSV data range may be changed according to the color matching style of the target display device and the color of the cut image.

[0065] After the color conversion between RGB and HSV, the color corresponding to each pixel can be determined based on the HSV data range and the HSV data corresponding to each pixel. For example, the HSV values of pixel A are H=35, S=200, and V=200. The HSV value meets the pre-set green value range, and the color corresponding to the pixel is determined to be green. The number of green pixels is increased by 1. The same applies to other colors.

[0066] After traversing all pixels in the information indication area, the number of pixels corresponding to each color type is obtained. The color types are then sorted in descending order of pixel count. The color type with the largest number of pixels is determined as the color type displayed in the information indication area, and a visual text result is output, for example, the indicator light color is green. It is then determined whether the identified color type is consistent with the color type that the information indication area should display under the control instruction. If so, the color display of the information indication area is normal; otherwise, the color display is abnormal.

[0067] The above method can be used to detect whether the indicator light on the instrument panel is on, whether the pop-up message is displayed normally, etc. For example, for the indicator light, the color is green when the indicator light is on, and the color is gray when the indicator light is not on. If the control command indicates that the indicator light is on, but the color in the information indication area is detected to be gray, it can be determined that the indicator light display is abnormal.

[0068] It should be noted that in the above method, the color type with the largest number of pixels is used as the color type of the indicator light or pop-up message in the information indication area, but the information indication area usually has a background, and the background area is usually larger than the indicator area or pop-up area. Therefore, in the process of counting the number of pixels corresponding to the color type, the number of pixels corresponding to the background color needs to be removed.

[0069] Specifically, first obtain the background color type corresponding to the background area corresponding to the target display device, and the HSV data range corresponding to each color type; then, remove the HSV data corresponding to the background color type from the HSV data of all pixels in the information indication area to obtain the HSV data corresponding to the remaining pixels; then, based on the HSV data range corresponding to each color type, perform data statistics on the HSV data corresponding to the remaining pixels to obtain the number of pixels corresponding to each color type.

[0070] For example, if pixel B has an HSV value of H=10, S=40, and V=153, and its corresponding HSV data falls within the HSV data range for the background color brown, then when counting the number of pixels corresponding to that color type, pixel B is not counted in either color type, and the color determination continues with the next pixel. This eliminates the influence of the background color on the recognition results during color recognition, improving color recognition accuracy.

[0071] Detection of text display areas can be achieved through text comparison and confidence comparison. Specifically, the detection platform first performs optical character recognition on the text in the text display area to obtain the displayed text and the text confidence corresponding to the displayed text. Then, the displayed text is compared with the target text and the text confidence is compared with the target confidence threshold corresponding to the displayed text to obtain the text detection result for the text displayed on the target display device.

[0072] In the above embodiment, the target text is the text that is expected to be displayed in the text display area under the control instruction. For example, in the case where the control instruction is an incoming call instruction, the text display area can be as follows: Figure 4 As shown, in this scenario, the target text includes at least a phone number, "answer", "hang up" and other texts.

[0073] In addition, during text recognition, OCR (Optical Character Recognition) technology may be used to extract text in the text display area. While extracting the text, the confidence level corresponding to the text is automatically generated.

[0074] To improve the accuracy of text recognition, the detection platform preprocesses the text display area before performing optical character recognition on the text in the text display area.

[0075] Specifically, the detection platform extracts the text display area from the display image according to the preset text display area coordinates, and performs binarization and denoising on the text display area to obtain a pre-processed text display area.

[0076] To extract the text display area, the detection platform uses an optical character recognition method to determine the shape of the text by detecting dark and light patterns, and then uses the optical character recognition method to translate the shape into computer text. The text display area can be obtained by intercepting the rectangular area where the text is located in the display image captured by the image acquisition device. Since the pop-up window and page card display on the instrument panel are fixed in position, the position of the above-mentioned rectangular area in the display image is also fixed. In the embodiment of the present application, the text display area can also be determined by the two-dimensional coordinates of the four corner points of the rectangular area.

[0077] After obtaining the text display area, the detection platform performs image enhancement preprocessing on the text display area, mainly grayscale binarization and image denoising. The binarization threshold can be determined according to the display characteristics of different types of display devices. The color image can be converted into a black and white image through binarization, and then the outlier black noise in the black image is filtered out by clustering to avoid interference from other colors on the text detection results. Then, through OCR text recognition technology, the Chinese characters, English letters and numbers in the text display area are extracted, and the text detection results are generated through assertion judgment.

[0078] After preprocessing the text display area, the displayed text is compared with the target text, and the text confidence is compared with the target confidence threshold corresponding to the displayed text to obtain the text detection result. Specifically, the detection platform performs a similarity comparison between the displayed text and the target text to obtain text similarity. If the text similarity is greater than the similarity threshold, the target confidence threshold corresponding to the text type of the displayed text is determined based on the association between the text type and the confidence threshold. If the text confidence is greater than the target confidence threshold, the text display on the target display device is determined to be normal.

[0079] As an example, Figure 4 For example, the text recognized by OCR technology is compared with the target text to obtain text similarity. When the text similarity is higher than the similarity threshold, it can be determined that the text recognition is correct. Then, the confidence level of the recognized text is determined. When the confidence level of the text is also higher than the corresponding confidence threshold, it can be determined that the text is displayed accurately.

[0080] It should be noted that different confidence thresholds can be set according to the characteristics of the display device and the structural characteristics of different types of text. For example, in an embodiment of the present application, the confidence threshold can be set within the range of 0.85-0.95, and different text types correspond to different confidence thresholds. For example, for Chinese characters, the corresponding confidence threshold may be 0.95; for numbers, the corresponding confidence threshold is 0.9; and for English, the corresponding confidence threshold is 0.85.

[0081] When it is determined that the text similarity is greater than the similarity threshold and the confidence is also greater than the corresponding confidence threshold, it can be determined that the text displayed in the text display area is the target text, and a visual text detection result is output to prompt the tester whether the test passes, thereby realizing the detection of text pop-ups or interface jumps in the target display device.

[0082] For icon display areas, the detection platform uses template matching to detect them. Specifically, the detection platform first obtains a first grayscale image of the template icon corresponding to the target icon, and a second grayscale image corresponding to the icon display area. It then searches the second grayscale image for the target grayscale area that has the greatest similarity to the first grayscale image. It then determines the icon similarity between the target grayscale area and the target icon. If the icon similarity exceeds a similarity threshold, the platform determines that the icon display within the icon display area is normal.

[0083] In the above embodiment, the template icon can be a pre-set icon displayed in the icon display area under different control instructions. To improve detection accuracy, before performing template matching on the icon display area, the icon display area needs to be converted into a grayscale image (i.e., a second grayscale image).

[0084] In one example, the detection platform may search and find the template icon from the second grayscale image by grayscale value comparison, wherein the grayscale value comparison may be implemented by template similarity calculation, as shown in formula (2):

[0085]

[0086] In formula (2), R(x, y) represents the icon similarity, T represents the target icon, I represents the template icon, (x, y) represents the pixel coordinates of the target icon, and (x′, y′) represents the pixel coordinates of the template icon.

[0087] The similarity calculation method of formula (2) is used to search for areas in the icon display area that are similar in grayscale to the template icon, and the icon similarity R(x, y) is obtained. Then, the icon similarity is compared with a preset icon similarity threshold to screen out the area with the highest similarity. Then, color detection can be combined with color detection to determine whether the icon display is normal. That is, in this embodiment of the application, the detection of the icon display area can include two aspects, namely, detecting whether the icon is displayed and whether the icon display color is correct.

[0088] In the above example, the icon similarity threshold may be determined according to the type of display device, or may be set to a fixed value, for example, 0.85.

[0089] This concludes the introduction to the detection of the icon display area.

[0090] In one embodiment, Figure 5 The overall flow chart of the detection method of the display device provided by the embodiment of the present application is shown. Figure 5 It can be seen that in the embodiment of the present application, the display device is mainly tested from three aspects: color, text and icon. The specific process is as follows:

[0091] Step S50: The real-time image displayed by the target display device is captured by an image capture device and saved as a display image.

[0092] In this step, after the image acquisition device is connected to the host computer, the image acquisition device is called through the algorithm in the host computer, and the display images captured by the image acquisition device are obtained and stored in real time. The number of images captured per second can also be set.

[0093] Step S51, extracting the information indication area from the displayed image, and detecting the information indication area to obtain an information prompt detection result;

[0094] Step S52, extracting the text display area from the displayed image, and detecting the text display area to obtain a text detection result;

[0095] Step S53: extract the icon display area from the display image, and detect the icon display area to obtain an icon detection result.

[0096] It should be noted that the above steps S51 to S53 can be executed in a preset order or in parallel. In this application, the execution order of the above steps S51 to S53 is not limited.

[0097] The execution process of the above steps S51 to S53 is introduced below respectively.

[0098] like Figure 5 As shown, step S51 includes steps S510 to S515:

[0099] Step S510, extracting an information indication area from the displayed image;

[0100] Step S511, converting the color format of the information indication area from RGB format to HSV format;

[0101] Step S512, filtering out the background color of the information indication area after format conversion, and determining the HSV values of the remaining pixels;

[0102] Step S513, comparing the HSV value of each pixel with the HSV data range, determining the color type corresponding to each pixel, and counting the number of pixels corresponding to each color type;

[0103] Step S514, determining the color type with the largest number of pixels as the color type corresponding to the information indication area;

[0104] Step S515 , comparing the recognized color type with the target color type to determine whether the color display of the information indication area is normal.

[0105] like Figure 5 As shown, step S52 includes steps S520 to S525:

[0106] Step S520, extracting a text display area from the display image;

[0107] Step S521, performing image enhancement preprocessing on the text display area;

[0108] Step S522, performing text recognition on the text display area to obtain a text recognition result;

[0109] Step S523, determining whether there is text in the text display area based on the text recognition result, wherein if it is determined that there is text, executing step S524; otherwise, determining that the text display in the text display area is abnormal;

[0110] Step S524, detecting whether the text confidence is greater than a confidence threshold. If the text confidence is less than the confidence threshold, it can be determined that the text display in the text display area is abnormal. Otherwise, step S525 is executed;

[0111] Step S525: Output the visual text detection result. Figure 4 , and its corresponding text detection results can be as follows Figure 6 shown.

[0112] like Figure 5 As shown, step S53 includes steps S530 to S533:

[0113] Step S530, extracting the icon display area from the display image and obtaining the template icon;

[0114] Step S531, obtaining a target grayscale area having the greatest similarity to the template icon from the icon display area;

[0115] Step S532 , determining whether the icon similarity between the target grayscale area and the template icon is greater than a preset similarity threshold.

[0116] Step S533 : when the icon similarity is greater than a preset similarity threshold, generating a visualized icon detection result; when the icon similarity is less than or equal to the preset similarity threshold, determining that the icon detection is abnormal.

[0117] After obtaining the information prompt detection result, text detection result and icon detection result respectively through the above steps S51, S52 and S53, when the above three detection results are all normal, it can be determined that the display device is normal; otherwise, it can be determined that the detection device is abnormal.

[0118] This completes the introduction of the method provided in the embodiments of this application.

[0119] The solution provided by the embodiment of the present application can realize the testing of intelligent automobile instruments with low cost, simple operation and strong reusability. Compared with the traditional automobile instrument testing solution based on real vehicle testing, the solution provided by the embodiment of the present application reduces the occupation of manpower and equipment resources and reduces costs. At the same time, it can realize the execution of automated test sequences based on the adjustment of initial parameters, thereby improving the efficiency of the test implementation process. In addition, in the embodiment of the present application, all input images are obtained through integrated algorithms. It is only necessary to build a test bench and install a fixed high-definition camera. This method greatly reduces the cost of hardware system calibration and maintenance. The algorithm can modify the operating parameters according to actual needs. The accuracy of the results obtained will not be reduced, will not be affected by the external weather conditions of rain, snow or cloudy, and will not cause misjudgment due to subjective reasons of the tester. Finally, from the perspective of test implementation and result recording and evaluation, the host computer used in the solution provided by the embodiment of the present application can also automatically execute the sequence and generate a report. The function of automatically acquiring the input image and the interface for modifying the parameters are integrated in the instrument recognition algorithm. The visual image results and text results finally generated can be automatically added to the test report, which is convenient for the tester to trace and review the problem.

[0120] In summary, the solution provided in the embodiment of the present application implements the testing of intelligent automobile instruments through text recognition and image processing, which can effectively reduce the waste of manpower and equipment resources and improve the execution efficiency of automated testing.

[0121] The present application also provides a detection device for a display device, such as Figure 7 As shown, the detection device 700 of the display device includes: an image acquisition module 701 , a color comparison module 702 , a text comparison module 703 , a template matching module 704 and a detection module 705 .

[0122] The image acquisition module 701 is used to acquire the display image displayed by the target display device under the control instruction, wherein the display image at least includes: an information indication area, a text display area, and an icon display area;

[0123] A color comparison module 702 is used to compare the color data of the information indication area with the target color data to obtain an information prompt detection result of the target display device;

[0124] The text comparison module 703 is used to perform text and confidence comparison on the displayed text recognized in the text display area to obtain a text detection result for performing text display detection on the target display device;

[0125] The template matching module 704 is used to perform template matching on the icon display area to obtain an icon detection result for performing icon detection on the target display device;

[0126] The detection module 705 is configured to generate a target detection result for detecting a target display device according to the information prompt detection result, the text detection result, and the icon detection result.

[0127] In one embodiment, the detection device of the display device also includes: a color conversion module, which is used to extract the information indication area from the display image according to preset information indication area coordinates before comparing the color data of the information indication area with the target color data to obtain the information prompt detection result of the target display device; and convert the color data of each pixel in the information indication area from red-green-blue RGB data to hue-saturation-value HSV data.

[0128] In one embodiment, the color comparison module includes: a data statistics module, a color type acquisition module, and a color detection module. The data statistics module is used to perform data statistics on the HSV data of all pixels in the information indication area based on the correspondence between the color type and the HSV data range to obtain the number of pixels corresponding to multiple color types, wherein each color type is the type of color displayed by the target display device under different instructions; the color type acquisition module is used to obtain the target color type with the largest number of pixels from the multiple color types; and the color detection module is used to determine that the color display in the information indication area is normal when the color data corresponding to the target color type matches the target color data, wherein the target color data is the color data expected to be displayed in the information indication area of the target display device under the control instruction.

[0129] In one embodiment, the data statistics module is specifically used to obtain the background color type corresponding to the background area corresponding to the target display device, and the HSV data range corresponding to each color type; remove the HSV data corresponding to the background color type from the HSV data of all pixels in the information indication area to obtain the HSV data corresponding to the remaining pixels; and perform data statistics on the HSV data corresponding to the remaining pixels according to the HSV data range corresponding to each color type to obtain the number of pixels corresponding to each color type.

[0130] In one embodiment, the text comparison module includes: a text recognition module and a text detection module. The text recognition module is used to perform optical character recognition on the text in the text display area to obtain the displayed text and the text confidence corresponding to the displayed text; the text detection module is used to compare the displayed text with the target text and compare the text confidence with the target confidence threshold corresponding to the displayed text to obtain a text detection result for displaying the text on the target display device, wherein the target text is the text that is expected to be displayed in the text display area under the control instruction.

[0131] In one embodiment, the detection device of the display device also includes: a preprocessing module, which is used to extract the text display area from the display image according to preset text display area coordinates before performing optical character recognition on the text in the text display area to obtain the displayed text and the text confidence corresponding to the displayed text; and perform binarization and denoising on the text display area to obtain the preprocessed text display area.

[0132] In one embodiment, the text detection module is specifically used to perform a similarity comparison between the displayed text and the target text to obtain text similarity; when the text similarity is greater than the similarity threshold, the target confidence threshold corresponding to the text type of the displayed text is determined based on the association between the text type and the confidence threshold; when the text confidence is greater than the target confidence threshold, it is determined that the text display of the target display device is normal.

[0133] In one embodiment, the template matching module is specifically used to obtain a first grayscale image of a template icon corresponding to the target icon, and a second grayscale image corresponding to the icon display area; search for a target grayscale area with the greatest similarity to the first grayscale image from the second grayscale image; obtain the icon similarity between the target grayscale area and the target icon; and when the icon similarity is greater than a similarity threshold, determine that the icon display in the icon display area is normal.

[0134] In one embodiment, the detection module is specifically used to determine that the target display device is in a normal state when the information prompt detection result indicates that the color display in the information indication area is normal, and the text detection result indicates that the text display in the text display area is normal, and the icon detection result indicates that the icon display in the icon display area is normal; and to determine that the target display device is in an abnormal state when the information prompt detection result indicates that the color display in the information indication area is abnormal, and / or the text detection result indicates that the text display in the text display area is abnormal, and / or the icon detection result indicates that the icon display in the icon display area is abnormal.

[0135] The detection device for a display device provided in an embodiment of the present application can implement each process implemented in the aforementioned method embodiment, and to avoid repetition, they will not be described here.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0137] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0138] The electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0139] Specifically, the processor 801 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0140] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 802 may include removable or non-removable (or fixed) media. Where appropriate, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory.

[0141] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0142] The processor 801 reads and executes computer program instructions stored in the memory 802 to implement any one of the display device detection methods in the above embodiments.

[0143] In one example, the electronic device may further include a communication interface 803 and a bus 810. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 810 and communicate with each other.

[0144] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0145] Bus 810 comprises hardware, software or both, couples the parts of electronic equipment to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 810 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0146] In addition, in conjunction with the display device detection method in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the display device detection methods in the above embodiments is implemented.

[0147] In addition, in combination with the display device detection method in the above embodiments, the present application embodiment can provide a computer program product for implementation. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device executes and implements the display device detection method in any of the above embodiments.

[0148] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0149] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in unit, a function card or the like. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0150] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0151] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the detection methods and devices of the display devices according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of the boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0152] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A method for detecting a display device, characterized in that: include: Acquiring a display image displayed by a target display device under a control instruction, wherein the display image at least includes: an information indication area, a text display area, and an icon display area; Comparing the color data of the information indication area with the target color data to obtain an information prompt detection result of performing information prompt detection on the target display device; Performing text and confidence comparison on the displayed text recognized in the text display area to obtain a text detection result for performing text display detection on the target display device; Performing template matching on the icon display area to obtain an icon detection result for performing icon detection on the target display device; A target detection result for detecting the target display device is generated according to the information prompt detection result, the text detection result, and the icon detection result.

2. The method according to claim 1, characterized in that Before comparing the color data of the information indication area with the target color data to obtain an information prompt detection result of performing an information prompt detection on the target display device, the method further includes: extracting the information indication area from the displayed image according to preset information indication area coordinates; The color data of each pixel in the information indication area is converted from red-green-blue RGB data to hue-saturation-value HSV data.

3. The method according to claim 2, characterized in that Comparing the color data of the information indication area with the target color data to obtain an information prompt detection result of performing information prompt detection on the target display device, including: According to the correspondence between color types and HSV data ranges, data statistics are performed on the HSV data of all pixels in the information indication area to obtain the number of pixels corresponding to multiple color types, wherein each color type is a type of color displayed by the target display device under different instructions; Obtaining the target color type with the largest number of pixels from the multiple color types; When the color data corresponding to the target color type matches the target color data, it is determined that the color display in the information indication area is normal, wherein the target color data is the color data expected to be displayed in the information indication area of the target display device under the control instruction.

4. The method according to claim 3, characterized in that According to the correspondence between color types and HSV data ranges, data statistics are performed on the HSV data of all pixels in the information indication area to obtain the number of pixels corresponding to multiple color types, including: Obtain the background color type corresponding to the background area corresponding to the target display device, and the HSV data range corresponding to each color type; Remove the HSV data corresponding to the background color type from the HSV data of all pixels in the information indication area to obtain HSV data corresponding to the remaining pixels; According to the HSV data range corresponding to each color type, data statistics are performed on the HSV data corresponding to the remaining pixels to obtain the number of pixels corresponding to each color type.

5. The method according to claim 1, wherein Performing text and confidence comparison on the displayed text recognized in the text display area to obtain a text detection result of performing text display detection on the target display device, including: Performing optical character recognition on the text in the text display area to obtain displayed text and a text confidence level corresponding to the displayed text; The displayed text is compared with the target text, and the text confidence is compared with the target confidence threshold corresponding to the displayed text to obtain a text detection result for text display on the target display device, wherein the target text is the text expected to be displayed in the text display area under the control instruction.

6. The method according to claim 5, characterized in that Before performing optical character recognition on the text in the text display area to obtain the displayed text and the text confidence corresponding to the displayed text, the method further includes: Extracting the text display area from the display image according to preset text display area coordinates; Binarization and denoising are performed on the text display area to obtain a pre-processed text display area.

7. The method according to claim 5 or 6, characterized in that Comparing the displayed text with the target text, and comparing the text confidence with a target confidence threshold corresponding to the displayed text, to obtain a text detection result for displaying text on the target display device, including: Performing a similarity comparison between the displayed text and the target text to obtain text similarity; In a case where the text similarity is greater than the similarity threshold, determining a target confidence threshold corresponding to the text type of the displayed text according to an association relationship between the text type and the confidence threshold; In a case where the text confidence is greater than the target confidence threshold, it is determined that the text display of the target display device is normal.

8. The method according to claim 1, characterized in that Performing template matching on the icon display area to obtain an icon detection result for performing icon detection on the target display device includes: Acquire a first grayscale image of a template icon corresponding to a target icon, and a second grayscale image corresponding to the icon display area; searching, from the second grayscale image, a target grayscale region having the greatest similarity to the first grayscale image; Obtaining icon similarity between the target grayscale area and the template icon; When the icon similarity is greater than a similarity threshold, it is determined that the icons in the icon display area are displayed normally.

9. The method according to claim 1, characterized in that Generating a target detection result for detecting the target display device according to the information prompt detection result, the text detection result, and the icon detection result, including: When the information prompt detection result indicates that the color display in the information indication area is normal, the text detection result indicates that the text display in the text display area is normal, and the icon detection result indicates that the icon display in the icon display area is normal, determining that the target display device is in a normal state; When the information prompt detection result indicates that the color display in the information indication area is abnormal, and / or the text detection result indicates that the text display in the text display area is abnormal, and / or the icon detection result indicates that the icon display in the icon display area is abnormal, it is determined that the target display device is in an abnormal state.

10. A detection device for a display device, characterized in that: include: An image acquisition module, configured to acquire a display image displayed by a target display device under a control instruction, wherein the display image includes at least: an information indication area, a text display area, and an icon display area; A color comparison module is used to compare the color data of the information indication area with the target color data to obtain an information prompt detection result of the information prompt detection on the target display device; A text comparison module, configured to perform text and confidence comparison on the displayed text identified in the text display area, and obtain a text detection result for performing text display detection on the target display device; a template matching module, configured to perform template matching on the icon display area to obtain an icon detection result for performing icon detection on the target display device; A detection module is used to generate a target detection result for detecting the target display device according to the information prompt detection result, the text detection result and the icon detection result.

Images