Control panel detection method, device, system and computer-readable storage medium

Abstract

The present application relates to the field of control panel detection technology, and discloses a control panel detection method, device, system, and computer-readable storage medium. The method includes: obtaining an image stream to be detected; wherein the image stream to be detected includes image frames corresponding to the control panel; dividing the image frames in the image stream to be detected into events to obtain a number of events, and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; performing anomaly detection on the events of the second event type to obtain a detection result; and storing the target image frame and / or the timestamp of the target image frame in the events of the first event type. In this way, it is possible to detect whether there is a display anomaly on the control panel during the image frame change process.

Description

Technical Field

[0001] The present application relates to the field of control panel detection technology, and in particular to a control panel detection method, device, system, and computer-readable storage medium. Background Art

[0002] As living standards continue to improve, more and more devices have corresponding control panels. Users can control the operation of the device by operating the control panel. The control panel can also display corresponding content to inform the status of the device.

[0003] The shortcoming is that there is currently no effective means to identify display anomalies on the control panel. Summary of the Invention

[0004] The control panel detection method, device, system, and computer-readable storage medium provided in the present application can detect whether there is a display anomaly on the control panel during image frame changes.

[0005] In a first aspect, the present application provides a control panel detection method, the method comprising: obtaining an image stream to be detected; wherein the image stream to be detected includes image frames corresponding to the control panel; performing event division on the image frames in the image stream to be detected to obtain a plurality of events, and an event type corresponding to each event; wherein the event type comprises a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; performing anomaly detection on the events of the second event type to obtain a detection result; storing the target image frame and / or the timestamp of the target image frame in the events of the first event type; wherein the target image frame is one of the plurality of image frames in the events of the first event type.

[0006] Among them, the control panel includes at least one display area; the image frames in the image stream to be detected are divided into events to obtain a number of events, including: performing event division on each display area in the image frame to obtain a number of events corresponding to each display area; performing anomaly detection on events of the second event type to obtain detection results, including: performing anomaly detection on events of the second event type corresponding to each display area to obtain detection results.

[0007] Among them, the display area includes: at least one of: an icon display area, a character display area and an indicator light display area; performing anomaly detection on events of the second event type corresponding to each display area to obtain detection results, including: respectively obtaining anomaly detection algorithms corresponding to the icon display area, the character display area and / or the indicator light display area; using the anomaly detection algorithm to perform anomaly detection on events of the second event type corresponding to the respective corresponding icon display areas, the character display areas and / or the indicator light display areas to obtain detection results.

[0008] Among them, the anomaly detection algorithm includes an icon recognition algorithm and a flicker recognition algorithm; the anomaly detection algorithm is used to perform anomaly detection on events of the second event type corresponding to the icon display area to obtain a detection result, including: using the icon recognition algorithm to perform icon type recognition and integrity detection on image frames in events of the second event type corresponding to the icon display area; after the integrity detection passes, a preset number of events before and after the current event are selected, and the flicker recognition algorithm is used to determine the flicker frequency of the image frames in the current event and the preset number of events before and after, thereby obtaining a detection result.

[0009] Among them, the anomaly detection algorithm includes a flicker recognition algorithm, a character recognition algorithm, an alarm recognition algorithm and a time anomaly recognition algorithm; the anomaly detection algorithm is used to perform anomaly detection on the event of the second event type corresponding to the character display area to obtain a detection result, including: using the character recognition algorithm to perform character recognition on the image frame in the event of the second event type corresponding to the character display area; when the character recognition result is a time character, the time anomaly recognition algorithm is used to perform time anomaly recognition on the image frame in the event of the second event type; when the character recognition result is an alarm character, the alarm recognition algorithm is used to perform alarm code recognition on the image frame in the event of the second event type; and, using the flicker recognition algorithm to perform flicker anomaly detection on the image frame in the event of the second event type to obtain a detection result.

[0010] Among them, the time anomaly recognition algorithm is used to perform time anomaly recognition on the image frames in the events of the second event type, including: using the time anomaly recognition algorithm to perform digital display integrity detection on the image frames in the events of the second event type to obtain a first detection result; and using the time anomaly recognition algorithm to compare the time characters corresponding to the current event and the time characters corresponding to the previous event in the events of the second event type to obtain a second detection result; using the time anomaly recognition algorithm to identify the time length corresponding to the current event to obtain a third detection result; and using the first detection result, the second detection result, and the third detection result as the final detection result.

[0011] Among them, the anomaly detection algorithm includes a flicker recognition algorithm; using the anomaly detection algorithm to perform anomaly detection on events of the second event type corresponding to the indicator light display area to obtain a detection result, including: using the flicker recognition algorithm to perform flicker anomaly detection on events of the second event type corresponding to the indicator light display area to obtain a detection result.

[0012] Among them, after obtaining the detection result, the method also includes: marking the event of the second event type whose detection result indicates abnormality, and sending the marked event of the second event type to the manual terminal; receiving the confirmation result fed back by the manual terminal; when the confirmation result indicates that the marked event of the second event type is normal, using the image frame corresponding to the marked event of the second event type as a sample.

[0013] Before obtaining the image stream to be detected, the method also includes: providing a detection configuration interface, and displaying the template image frame corresponding to the image stream to be detected on the detection configuration interface; wherein, the detection configuration interface provides several recognition algorithms to be selected; receiving the user's selection of the target display area on the template image frame, and the selection of the target recognition algorithm; binding the target display area and the target recognition algorithm.

[0014] In a second aspect, the present application provides a detection device for a control panel, which includes: a memory and a processor, the processor being connected to the memory, the memory being used to store a computer program, and the computer program, when executed by the processor, being used to implement the method provided in the first aspect.

[0015] In a third aspect, the present application provides a detection system for a control panel, the detection system comprising: an image acquisition device and a detection device, the image acquisition device being used to acquire image frames corresponding to when the control panel is working, and the detection device being such as the detection device provided in the second aspect.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the method provided in the first aspect.

[0017] The beneficial effects of the present application are as follows: Different from the prior art, the control panel detection method, device, system and computer-readable storage medium provided by the present application obtain an image stream to be detected corresponding to the control panel; then, according to the event type, the image frames in the image stream to be detected are divided into events to obtain several events and the event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; then, anomaly detection is performed on the events of the second event type to obtain a detection result, thereby detecting whether there is a display anomaly of the control panel during the image frame change process; and storing the target image frame and / or the timestamp of the target image frame in the event of the first event type; wherein the target image frame is one of the several image frames in the event of the first event type, and there is no need to store all the image frames in the event of the first event type, which not only ensures the integrity of the events in the image stream, but also reduces the storage space occupied. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0019] Figure 1 This is a flow chart of the first embodiment of the control panel detection method provided by the present application;

[0020] Figure 2 This is a flow chart of a second embodiment of the control panel detection method provided by the present application;

[0021] Figure 3 This is a flow chart of a third embodiment of the control panel detection method provided by the present application;

[0022] Figure 4 This is a flow chart of an embodiment of abnormality detection for a character display area provided by the present application;

[0023] Figure 5 yes Figure 4 A flow chart of an embodiment of step 42;

[0024] Figure 6 This is a flow chart of a fourth embodiment of the control panel detection method provided by the present application;

[0025] Figure 7This is a flow chart of a fifth embodiment of the control panel detection method provided by the present application;

[0026] Figure 8 This is a flow chart of a sixth embodiment of the control panel detection method provided by the present application;

[0027] Figure 9 This is a structural diagram of an embodiment of a detection device for a control panel provided by the present application;

[0028] Figure 10 This is a structural diagram of an embodiment of a control panel detection system provided by the present application;

[0029] Figure 11 It is a structural diagram of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some, rather than all, structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0031] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] As living standards continue to improve, more and more devices have corresponding control panels. Users can control the operation of the device by operating the control panel. The control panel can also display corresponding content to inform the status of the device.

[0033] The shortcoming is that there is currently no effective means to identify display anomalies on the control panel.

[0034] Based on this, the present application proposes to obtain the image stream to be detected corresponding to the control panel; then, according to the event type, the image frames in the image stream to be detected are divided into events to obtain several events, and the event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; then, anomaly detection is performed on the events of the second event type to obtain a detection result, thereby detecting whether there is a display anomaly on the control panel during the image frame change process; and storing the target image frame and / or the timestamp of the target image frame in the event of the first event type; wherein the target image frame is one of the several image frames in the event of the first event type, and there is no need to store all the image frames in the event of the first event type, which can not only ensure the integrity of the events in the image stream, but also reduce the storage space occupied. For details, please refer to the technical solution of any of the following embodiments.

[0035] See Figure 1 , Figure 1 1 is a flow chart of a first embodiment of a control panel detection method provided by this application. The method includes:

[0036] Step 11: Acquire the image stream to be detected; wherein the image stream to be detected includes image frames corresponding to the control panel.

[0037] In some embodiments, an image capture device can be used to capture images corresponding to the control panel while it is operating, thereby obtaining an image stream of the control panel during operation. In other words, this image stream can be used as the image stream to be detected. For example, the image capture device can be aimed at the control panel to capture an image stream corresponding to the control panel during operation.

[0038] In some embodiments, the control panel can be provided on a home appliance, such as a control panel on a washing machine, a control panel on a range hood, a control panel on a microwave oven, a control panel on an induction cooker, a control panel on a refrigerator, a control panel on a smart home, and a series of other home appliances.

[0039] During operation, the control panel can provide corresponding displays to help users understand the status of the corresponding home appliance. For example, the control panel can display corresponding function icons, alarm codes, operation countdowns, indicator lights, etc. Based on this, this application can be used to detect whether the control panel's display-related functions are abnormal, thereby facilitating subsequent maintenance, improvement, and upgrades of the control panel.

[0040] Step 12: Perform event division on the image frames in the image stream to be detected to obtain a number of events and the event type corresponding to each event.

[0041] In some embodiments, the event type includes a first event type and a second event type. No image change occurs between image frames corresponding to events corresponding to the first event type. Image change occurs between image frames corresponding to events corresponding to the second event type.

[0042] In some embodiments, if the event type corresponding to the event is the first event type, it means that the control panel is normal at this time, and step 14 is executed.

[0043] In some embodiments, if the event type corresponding to the event is the second event type, it indicates that the control panel may be abnormal at this time, and step 13 is executed. It is understood that image changes between image frames can be normal changes or abnormal changes. Therefore, they need to be detected.

[0044] Step 13: Perform anomaly detection on the events of the second event type to obtain a detection result.

[0045] In some embodiments, because image changes between image frames can be normal or abnormal, anomaly detection is performed on the second event type to obtain a detection result. If the detection result is normal, no further action is required. If the detection result is abnormal, the abnormal event needs to be marked or recorded. The record may include the corresponding timestamp of the abnormal event, the corresponding sequence number of the abnormal event, etc., to facilitate subsequent backtracking as a basis for backtracking.

[0046] In some embodiments, a corresponding anomaly detection algorithm can be used to detect anomalies in image frames of events of the second event type, such as character integrity detection, warning code detection, countdown detection, indicator signal detection, etc. The corresponding anomaly detection algorithm can be configured based on the specific functions of the control panel.

[0047] Step 14: Store the target image frame and / or the time stamp of the target image frame in the event of the first event type.

[0048] The target image frame is one of several image frames in an event of the first event type.

[0049] Because no image changes occur between image frames in an event of the first event type, it can be considered normal. Therefore, only one image frame in the event of the first event type is stored to reduce the amount of stored data. For example, the first image frame in the event of the first event type can be stored, along with the timestamp corresponding to the image frame.

[0050] In some embodiments, any image frame in an event of the first event type may be stored.

[0051] In some embodiments, only the timestamp of the target image frame in the event of the first event type may be stored to reduce the amount of stored data. The corresponding image frame may be subsequently obtained from the image stream based on the timestamp.

[0052] In some embodiments, the target image frame and the time stamp of the target image frame in the event of the first event type may be stored.

[0053] In this embodiment, an image stream to be detected corresponding to a control panel is obtained; and then the image frames in the image stream to be detected are divided into events according to event types to obtain a number of events and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; then an abnormality detection is performed on the events of the second event type to obtain a detection result, thereby detecting whether there is a display abnormality of the control panel during the image frame change process; and storing the target image frame and / or the timestamp of the target image frame in the event of the first event type; wherein the target image frame is one of the several image frames in the event of the first event type, and there is no need to store all the image frames in the event of the first event type, which not only ensures the integrity of the events in the image stream, but also reduces the storage space occupied.

[0054] See Figure 2 , Figure 2 1 is a flow chart of a second embodiment of a control panel detection method provided by this application. The method includes:

[0055] Step 21: Acquire the image stream to be detected; wherein the image stream to be detected includes image frames corresponding to the control panel.

[0056] In some embodiments, the control panel includes at least one display area, for example, the control panel includes at least one of an icon display area, a character display area, and an indicator light display area.

[0057] The icon display area is used to display corresponding function icons. For example, the icon display area on a refrigerator control panel may display refrigerator operating mode icons. For example, the icon display area on a range hood control panel may display range hood operating mode icons. For example, the icon display area on a washing machine control panel may display laundry mode icons.

[0058] The character display area is used to display corresponding characters. For example, the characters can be alarm codes, countdowns, numbers, Chinese characters, and other characters that can express meanings that the user can understand. Of course, when the display is abnormal, the content displayed in the character display area is not understandable to the user. For example, the character display area on the control panel of a refrigerator can display the refrigerator's operating hours, alarm codes, etc. For example, the character display area on the control panel of a range hood can display the countdown of the range hood's current operating mode, alarm codes, etc. For example, the character display area on the control panel of a washing machine can display the countdown of the washing machine's current operating mode, alarm codes, etc.

[0059] The indicator light display area can be composed of physical or virtual indicator lights. The indicator light display area is used to indicate the status of the control panel and its corresponding device through the status of the indicator lights. For example, the lights may flash, turn on and off, or change color.

[0060] Step 22: Divide each display area in the image frame into events to obtain a number of events corresponding to each display area.

[0061] The event types include a first event type and a second event type; no image changes occur between image frames corresponding to events corresponding to the first event type, and image changes occur between image frames corresponding to events corresponding to the second event type.

[0062] In this embodiment, since there is at least one display area on the control panel, each display area can be divided into events to obtain a number of events corresponding to the display area of ​​each display type.

[0063] Step 23: Perform an abnormality detection on the events of the second event type corresponding to each display area to obtain a detection result.

[0064] After each display area is divided into events, an abnormality detection can be performed on the events of the second event type corresponding to each display area according to the display area to obtain a detection result.

[0065] In some embodiments, a corresponding anomaly detection algorithm can be used based on the display characteristics of each display area to perform anomaly detection on image frames in the corresponding second event type. For example, character integrity detection, warning code detection, countdown detection, indicator signal detection, etc. The corresponding anomaly detection algorithm can be configured based on the specific functions of the control panel.

[0066] Step 24: Store the target image frame and / or the time stamp of the target image frame in the event of the first event type.

[0067] The target image frame is one of the image frames in an event of the first event type.

[0068] In some embodiments, step 24 may have the same or similar technical solutions as other embodiments of the present application, and will not be described in detail here.

[0069] In this embodiment, an image stream to be detected corresponding to the control panel is obtained; and then, each display area in the image frame is divided into events according to the event type to obtain a number of events and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; then, an abnormality detection is performed on the events of the second event type in each display area to obtain a detection result, thereby detecting whether there is a display abnormality in each display area of ​​the control panel during the image frame change process; and storing the target image frame and / or the timestamp of the target image frame in the event of the first event type; wherein the target image frame is one of the several image frames in the event of the first event type, and there is no need to store all the image frames in the event of the first event type, which not only ensures the integrity of the events in the image stream, but also reduces the storage space occupied.

[0070] See Figure 3 , Figure 3 1 is a flow chart of a third embodiment of a control panel detection method provided by this application. The method includes:

[0071] Step 31: Acquire an image stream to be detected; wherein the image stream to be detected includes image frames corresponding to the control panel.

[0072] In some embodiments, the control panel includes at least one display area, for example, the control panel includes at least one of an icon display area, a character display area, and an indicator light display area.

[0073] In some embodiments, the control panel includes an icon display area.

[0074] In some embodiments, the control panel includes a character display area.

[0075] In some embodiments, the control panel includes an indicator light display area.

[0076] In some embodiments, the control panel includes an icon display area and a character display area.

[0077] In some embodiments, the control panel includes an icon display area and an indicator light display area.

[0078] In some embodiments, the control panel includes a character display area and an indicator light display area.

[0079] In some embodiments, the control panel includes an icon display area, a character display area, and an indicator light display area.

[0080] Step 32: Divide each display area in the image frame into events to obtain a number of events corresponding to each display area.

[0081] The event types include a first event type and a second event type; no image changes occur between image frames corresponding to events corresponding to the first event type, and image changes occur between image frames corresponding to events corresponding to the second event type.

[0082] Step 33: Obtain the abnormality detection algorithms corresponding to the icon display area, the character display area and / or the indicator light display area respectively.

[0083] In some embodiments, corresponding anomaly detection algorithms may be configured in advance for different display areas, and when executing step 33 , the corresponding anomaly detection algorithms may be directly acquired according to the corresponding configurations.

[0084] In some embodiments, the control panel includes an icon display area, and the abnormality detection algorithm corresponding to the icon display area is obtained.

[0085] In some embodiments, the control panel includes a character display area, and an abnormality detection algorithm corresponding to the character display area is obtained.

[0086] In some embodiments, the control panel includes an indicator light display area, and an abnormality detection algorithm corresponding to the indicator light display area is obtained.

[0087] In some embodiments, the control panel includes an icon display area and a character display area, and the abnormality detection algorithms corresponding to the icon display area and the character display area are obtained respectively.

[0088] In some embodiments, the control panel includes an icon display area and an indicator light display area, and the abnormality detection algorithms corresponding to the icon display area and the indicator light display area are obtained respectively.

[0089] In some embodiments, the control panel includes a character display area and an indicator light display area, and the abnormality detection algorithms corresponding to the character display area and the indicator light display area are obtained respectively.

[0090] In some embodiments, the control panel includes an icon display area, a character display area, and an indicator light display area, and the abnormality detection algorithms corresponding to the icon display area, the character display area, and the indicator light display area are obtained respectively.

[0091] Step 34: Utilize an anomaly detection algorithm to perform an anomaly detection on events of the second event type corresponding to the respective icon display areas, character display areas, and / or indicator light display areas to obtain a detection result.

[0092] In some embodiments, the abnormality detection algorithm for the icon display area includes an icon recognition algorithm and a flicker recognition algorithm. Based on this, the icon recognition algorithm can be used to perform icon type recognition and integrity detection on image frames in events of the second event type corresponding to the icon display area. After the integrity detection passes, a preset number of events before and after the current event are selected, and the flicker recognition algorithm is used to determine the flicker frequency of the image frames in the current event and the preset number of events before and after the current event, thereby obtaining a detection result.

[0093] In some embodiments, an icon recognition algorithm is used to perform icon type recognition and integrity detection on the image frames in the events of the second event type corresponding to the icon display area. If the corresponding icon is not recognized, the possible problem is that the icon is not displayed, or the icon display is incomplete and cannot be matched, then the detection result may be that the icon display in the display area is abnormal. After the integrity test is passed, a preset number of events before and after the current event are selected, and the flicker recognition algorithm is used to determine the flicker frequency of the image frames in the current event and the preset number of events before and after. If it flickers and cannot be matched to the normal flicker frequency, the detection result may be that the icon display is abnormal. If it flickers and matches the normal flicker frequency, the detection result may be that the icon display is normal. When the detection result indicates an abnormality, the event is marked.

[0094] In some embodiments, the abnormality detection algorithm for the character display area includes a flicker recognition algorithm, a character recognition algorithm, an alarm recognition algorithm, and a time abnormality recognition algorithm. Figure 4 ,Anomaly detection of the character display area includes the following processes:

[0095] Step 41: Perform character recognition on the image frames of the events of the second event type corresponding to the character display area using a character recognition algorithm.

[0096] Because the character display area can display a variety of characters, such as time characters, alarm codes, etc. Based on this, when performing abnormality detection on the character display area, step 41 is first performed to identify the type of characters displayed in the character display area. For example, the character type can be a time character, an alarm code, etc.

[0097] Step 42: When the character recognition result is a time character, a time anomaly recognition algorithm is used to perform time anomaly recognition on image frames in the event of the second event type.

[0098] In some embodiments, time characters are usually used for timing, such as timing of corresponding functions. For example, the character display area on the control panel of an electric rice cooker can display the timing of cooking time, the character display area on the control panel of a washing machine can display the timing of washing time, and the character display area on the control panel of a range hood can display the timing of smoke extraction time, etc. Such as countdown and forward timing. During the timing process, the time characters will change with time. For example, they will change by seconds or minutes. Therefore, during the display process, the time characters may be incompletely displayed or the switching of the time characters may be abnormal, such as the current display is 6, the next display is 8, and 7 is not displayed. There may also be abnormal time character switching. For example, if it changes by seconds, the current display is 6, the next second display is 6, and according to logic, the next second should display 7. Based on this, the time anomaly recognition algorithm can be used to perform time anomaly recognition on the image frames in the events of the second event type to identify these anomalies.

[0099] In some embodiments, the time anomaly identification algorithm also needs to perform a flicker detection operation on the time character to identify whether the character display area displays an abnormality.

[0100] In some embodiments, see Figure 5 , step 42 may be the following process:

[0101] Step 421: Perform digital display integrity detection on image frames in events of the second event type using a temporal anomaly recognition algorithm to obtain a first detection result.

[0102] In some embodiments, when a time character is displayed in the character display area, a time anomaly recognition algorithm is used to perform a digital display integrity check on image frames in events of the second event type. If the displayed time character is detected to be complete, the first detection result is that the character display is complete; if the displayed time character is detected to be incomplete, the first detection result is that the character display is incomplete.

[0103] Step 422: using a time anomaly recognition algorithm, compare the time character corresponding to the current event with the time character corresponding to the previous event in the events of the second event type to obtain a second detection result.

[0104] In some embodiments, even if the time characters are displayed completely, a time character switching anomaly may occur. This is called a time character switching anomaly. For example, if the time character corresponding to the current event is 6 and the time character corresponding to the previous event is 5 during a forward counting process, the second detection result indicates that the time character switching is normal. For example, if the time character corresponding to the current event is 6 and the time character corresponding to the previous event is 4 during a forward counting process, the second detection result indicates that the time character switching anomaly occurs.

[0105] For example, during the countdown, if the time character corresponding to the current event is 6 and the time character corresponding to the previous event is 7, the second detection result indicates that the time character switching is normal. For example, during the countdown, if the time character corresponding to the current event is 6 and the time character corresponding to the previous event is 8, the second detection result indicates that the time character switching is abnormal.

[0106] Step 423: Use a time anomaly recognition algorithm to identify the time length corresponding to the current event to obtain a third detection result.

[0107] In some embodiments, even if the time characters are displayed completely and the time character switching is normal, it is possible that the switching duration is abnormal. For example, during the countdown, the time character corresponding to the current event is 6, and the time character corresponding to the previous event is 7, but the time length corresponding to the current event is 2 seconds. In this case, the third detection result indicates that the time character switching duration is abnormal.

[0108] For example, during the countdown process, the time character corresponding to the current event is 6, and the time character corresponding to the previous event is 7, but the time length corresponding to the current event is 1 second, then the third detection result is that the time character switching duration is normal.

[0109] Step 424: The first detection result, the second detection result, and the third detection result are taken as the final detection result.

[0110] In some embodiments, the first detection result, the second detection result, and the third detection result may be used as the final detection result, so as to know the specific situation of the character display area when displaying the time character.

[0111] Step 43: When the character recognition result is an alarm character, an alarm recognition algorithm is used to perform alarm code recognition on the image frame in the event of the second event type.

[0112] In some embodiments, the warning character is used to provide an alarm prompt. For example, a corresponding alarm code may be used to indicate an abnormality in the control panel or the home appliance corresponding to the control panel. Furthermore, when the character recognition result is a warning character, an alarm recognition algorithm may be used to identify the alarm code in the image frame of the second event type, and the identified alarm code may be pushed to the user's terminal device.

[0113] In some embodiments, the warning recognition algorithm needs to perform operations such as integrity detection and flicker detection on the warning characters to identify whether the character display area displays abnormalities.

[0114] Step 44: Use a flicker recognition algorithm to perform flicker anomaly detection on image frames in events of the second event type to obtain a detection result.

[0115] In some embodiments, the character display area is typically displayed using a screen such as an LED (Light Emitting Diode) screen or an LCD (Liquid Crystal Display), which has a corresponding flicker frequency. If the screen is abnormal and / or the data transmission is incorrect, the flicker frequency will be abnormal. Based on this, a flicker recognition algorithm is used to detect flicker anomalies in image frames in events of the second event type to identify whether the character display area displays abnormalities and obtain corresponding detection results.

[0116] Through the above process of detecting abnormalities in the character display area, it is possible to know whether corresponding abnormalities occur in the character display area during operation. When the final detection result indicates abnormality, the event is marked.

[0117] In some embodiments, the abnormality detection algorithm for the indicator light display area includes a flicker recognition algorithm. Based on this, the flicker recognition algorithm can be used to perform flicker abnormality detection on the events of the second event type corresponding to the indicator light display area to obtain a detection result.

[0118] Step 35: Store the target image frame and / or the time stamp of the target image frame in the event of the first event type.

[0119] The target image frame is one of the image frames in an event of the first event type.

[0120] In some embodiments, step 35 may have the same or similar technical solutions as other embodiments of the present application, and will not be described in detail here.

[0121] In this embodiment, an image stream to be detected corresponding to the control panel is obtained; and then, each display area in the image frame is divided into events according to the event type to obtain a plurality of events and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; then, an abnormality detection is performed on the events of the second event type in the icon display area, the character display area and / or the indicator light display area to obtain a detection result, thereby detecting whether there is a display abnormality in the icon display area, the character display area and / or the indicator light display area of ​​the control panel during the image frame change process; and storing the target image frame and / or the timestamp of the target image frame in the event of the first event type; wherein the target image frame is one of the plurality of image frames in the event of the first event type, and there is no need to store all the image frames in the event of the first event type, which not only ensures the integrity of the events in the image stream, but also reduces the storage space occupied.

[0122] In some embodiments, see Figure 6 After getting the test results, the following process can be followed:

[0123] Step 61: Mark the event of the second event type whose detection result indicates abnormality, and send the marked event of the second event type to the manual terminal.

[0124] In some embodiments, because the algorithm may produce erroneous detection results due to accuracy issues, manual re-inspection can be combined to confirm the events of the second event type whose detection results indicate abnormalities. Based on this, the events of the second event type whose detection results indicate abnormalities can be marked and the marked events of the second event type can be sent to the manual terminal.

[0125] In some embodiments, a timestamp corresponding to an event of the second event type may be sent to a manual terminal. After receiving the timestamp, the manual terminal may backtrack the image stream according to the event stamp and perform manual confirmation.

[0126] In some embodiments, because events can have contextual relationships, the timestamp corresponding to the second event type and the timestamps of a preset number of events before and after the event can be sent to a human terminal. This allows the human terminal to determine the cause and effect of the event based on the timestamps, understand the logic before and after the anomaly, and reduce the effort of reproducing the anomaly. After receiving the timestamps, the human terminal can use the event stamps to backtrack the image stream and perform manual confirmation.

[0127] Step 62: Receive the confirmation result fed back by the manual terminal.

[0128] The confirmation result may indicate that the marked event of the second event type is normal, or may indicate that the marked event of the second event type is abnormal.

[0129] Step 63: When the confirmation result indicates that the marked event of the second event type is normal, the image frame corresponding to the marked event of the second event type is used as a sample.

[0130] If the confirmation result indicates that the marked second event type is normal, it can indicate that the algorithm has made a recognition error. Therefore, the image frame corresponding to the marked second event type is used as a sample. This sample can be used to retrain the model corresponding to the corresponding algorithm to improve the algorithm accuracy.

[0131] When the confirmation result indicates that the marked event of the second event type is abnormal, it is stored as abnormal data.

[0132] In some embodiments, as the accuracy of the model corresponding to the algorithm improves, it may not be necessary to send all of the above-mentioned marked events of the second event type to the manual terminal. Instead, they can be selectively sent based on the confidence level of the marked events of the second event type. For example, marked events of the second event type with a confidence level below a threshold may be sent to the manual terminal for manual review, thereby reducing labor costs.

[0133] In some embodiments, after obtaining the detection result, the event of the second event type for which the detection result indicates an abnormality may be marked, and the marked event of the second event type may be displayed on a display interface to facilitate manual review by the user. A confirmation result of the manual review feedback is received. When the confirmation result indicates that the marked event of the second event type is normal, the image frame corresponding to the marked event of the second event type is used as a sample.

[0134] As the accuracy of the model corresponding to the algorithm improves, it is no longer necessary to send all of the marked second event type events to a human terminal. Instead, they can be selectively sent based on the confidence level of the marked second event type events. For example, marked second event type events with a confidence level below a threshold will be sent to a human terminal for manual review, thereby reducing labor costs.

[0135] In some embodiments, see Figure 7 Before obtaining the image stream to be detected, the following process can also be performed:

[0136] Step 71: providing a detection configuration interface, and displaying a template image frame corresponding to the image stream to be detected on the detection configuration interface; wherein the detection configuration interface provides several recognition algorithms to be selected.

[0137] In some embodiments, taking into account the diversity of types and structures of control panels, this is equivalent to a non-standard detection scenario. If each type of control panel is to be tested, the abnormal identification of non-standard scenarios requires a large amount of algorithm resources, which is costly. Based on this, the present application proposes to provide users with a detection configuration interface so that users can specify the display area to be identified and specify the recognition algorithm through manual configuration, thereby reducing the resource consumption of algorithm recognition and improving the accuracy of recognition. In this way, the variable recognition in non-standard scenarios is reduced, which greatly reduces the consumption of computing power and the performance requirements of the camera for collecting image streams, and can reduce the detection cost.

[0138] Step 72: Receive the user's selection of the target display area on the template image frame and the selection of the target recognition algorithm.

[0139] In some embodiments, when a user selects an icon display area on a template image frame, the abnormality detection algorithms that may be selected for the area include an icon recognition algorithm and a flicker recognition algorithm.

[0140] In some embodiments, when a user selects a character display area on a template image frame, the anomaly detection algorithms that may be selected for the character display area include a flicker recognition algorithm, a character recognition algorithm, an alarm recognition algorithm, and a time anomaly recognition algorithm.

[0141] In some embodiments, when a user selects an indicator light display area on a template image frame, the abnormality detection algorithm that can be selected for the indicator light display area includes a flicker recognition algorithm.

[0142] In some embodiments, the user may select corresponding recognition algorithms for the display areas according to the actual display areas of the control panel.

[0143] Step 73: Bind the target display area and the target recognition algorithm.

[0144] After the target display area and the target recognition algorithm are bound together, when the image stream to be detected is subsequently detected, the corresponding display area can be detected for abnormalities using the corresponding target recognition algorithm based on the binding relationship.

[0145] In some embodiments, any of the above-mentioned object recognition algorithms can be constructed based on an AI (Artificial Intelligence) model. The AI ​​model can be based on various types of AI models, such as a deep learning model, a convolutional neural network, or a large language model.

[0146] In one application scenario, combined with Figure 8 To explain:

[0147] Step 801: Configure the identification area.

[0148] In some embodiments, a detection configuration interface may be provided. The user configures the recognition area in the detection configuration interface to help the system determine the recognition area. The recognition area corresponds to the display area on the control panel.

[0149] Step 802: Configure a recognition algorithm for the recognition area.

[0150] In some embodiments, a recognition algorithm is configured for the recognition area, and after configuration, an abnormality recognition algorithm model of each recognition area is bound and an abnormality library is loaded.

[0151] For example, in the detection configuration interface, generate a target recognition frame and adjust it to coincide with the recognition area. At this point, the recognition area corresponding to the target recognition frame can be identified as the subsequent recognition area. After the adjustment is completed, select the algorithm model in the drop-down box and then configure the algorithm.

[0152] In some embodiments, corresponding recognition algorithms can be configured for different display areas.

[0153] For example, you can manually limit the recognition range by selecting a specific area and specify an anomaly recognition algorithm based on the actual conditions (display content) of each recognition area. For example, for the countdown area, you need to configure algorithms for countdown anomaly recognition, flicker frequency recognition, and text defect recognition; for the icon display area, you need to configure algorithms for icon recognition and flicker frequency recognition; and for the LED display area (indicator light display area), you need to configure a flicker recognition algorithm.

[0154] Step 803: Load the sample gallery.

[0155] Step 804: Acquire the image stream of the control panel.

[0156] After executing steps 801 to 803 , step 804 may be executed to obtain an image stream when the control panel is working.

[0157] Step 805: Decompose the image stream according to the recognition area to obtain the image stream corresponding to each recognition area.

[0158] Step 806: performing event division on the image stream corresponding to each recognition area to obtain events of the first event type and events of the second event type.

[0159] In some embodiments, the image stream can be broken down into images frame by frame, and image recognition technology can be used to identify the display screen of each area. Each change is stored as an event in the database, and event backtracking capabilities are provided.

[0160] Step 807: Input the event of the second event type into the corresponding recognition algorithm for abnormality determination.

[0161] In some embodiments, events of the second event type are imported into a corresponding recognition algorithm model to perform abnormality determination.

[0162] For example, for the icon display area, the AI ​​algorithm is first used to identify the image frame after frame-by-frame slicing to determine its basic graphics, and the icon recognition algorithm is called to determine the recognition type and completeness of the icon. If the corresponding icon is not matched, it will prompt an incomplete display exception. If the icon is complete, the four events before and after are selected, and the flickering abnormality recognition algorithm is used to determine its flickering frequency. If it flickers and cannot be matched to the normal flickering frequency, it will prompt an icon display abnormality. If no abnormality is identified, step 808 is executed until the detection is completed.

[0163] If an anomaly is identified, the event is marked and step 809 is executed.

[0164] Step 808: Continue to identify the next event of the second event type.

[0165] Step 809: Manual judgment.

[0166] The event type is manually determined through event backtracking data to determine whether it is correct. If the manual system makes an error in the judgment, the image frame corresponding to the event is stored in the sample library, and the sample library is completed as in step 810, and the next event is entered at the same time. If the system judgment is correct, no operation is performed, and the system waits for manual determination whether to end the task. If so, the detection ends, otherwise it continues to enter the loop.

[0167] Step 810: Update the sample gallery.

[0168] The main reason why manual configuration of the recognition area is required in this application scenario is that the detection of non-standard scenes has too many uncertain variables. The scenes can be infinitely enumerated and rely entirely on the system, which consumes a lot of computing power and is extremely costly. This is one of the reasons why AI cannot be applied to non-standard scenes. This application scenario limits the recognition range through manual selection and specifies the corresponding anomaly recognition algorithm based on the actual situation of each recognition area. These two methods reduce variables and can effectively reduce the system's dependence on computing power and camera performance, greatly reducing system R&D costs.

[0169] Data stored here includes images and videos. To efficiently utilize storage resources, only one image frame (one image) is saved for each event within a standard timeframe. For example, if the recognition area (display area) only displays the number 8 for one minute, only the first image frame of the number 8 is retained for that minute. The new image is not saved to the database until the next event occurs. Regarding image streams, only one copy is stored throughout the entire process, and no slices are stored.

[0170] The above-mentioned anomalies may include: discontinuous time countdown, abnormal flashing, alarm code recognition, text defect recognition, etc.; for time anomaly recognition, this application uses basic text recognition, event changes and event time difference to make judgments. Taking the time countdown as an example: first, identify whether the time characters (numbers) are displayed normally, and then compare the current event with the previous event to determine whether the difference is a standard time (such as: 1 minute or 1 second). In addition, the time difference between events should be combined to determine whether the entire event cycle is a standard time.

[0171] The above-mentioned replay refers to the playback of image frames and image streams before and after an abnormal event. Image replay not only records the abnormal event itself but also displays normal events before and after it. Image stream replay supports video playback from 5 seconds before to 5 seconds after the abnormal event. These two replay methods allow users to understand the logic before and after the abnormal event, reducing the work of reproducing the abnormality.

[0172] See Figure 9, Figure 9 1 is a schematic diagram of the structure of an embodiment of a control panel detection device provided by the present application. The detection device 90 includes: a memory 91 and a processor 92. The processor 92 is connected to the memory 91. The memory 91 is used to store a computer program. When the computer program is executed by the processor 92, it is used to implement the following method:

[0173] Acquire an image stream to be detected; wherein the image stream to be detected includes image frames corresponding to a control panel; perform event division on the image frames in the image stream to be detected to obtain a plurality of events and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between image frames corresponding to events corresponding to the first event type, and image change occurs between image frames corresponding to events corresponding to the second event type; perform anomaly detection on events of the second event type to obtain a detection result; and store the first image frame in the event of the first event type.

[0174] In some embodiments, the control panel includes at least one display area; when the computer program is executed by the processor 92, it is also used to implement the following method: performing event division on each display area in the image frame to obtain a number of events corresponding to each display area; performing anomaly detection on events of the second event type to obtain detection results, including: performing anomaly detection on events of the second event type corresponding to each display area to obtain detection results.

[0175] In some embodiments, the display area includes: at least one of an icon display area, a character display area, and an indicator light display area; when the computer program is executed by the processor 92, it is also used to implement the following method: respectively obtain the abnormality detection algorithms corresponding to the icon display area, the character display area, and / or the indicator light display area; use the abnormality detection algorithm to perform abnormality detection on events of the second event type corresponding to the respective corresponding icon display areas, character display areas, and / or indicator light display areas to obtain detection results.

[0176] In some embodiments, the anomaly detection algorithm includes an icon recognition algorithm and a flicker recognition algorithm; when the computer program is executed by the processor 92, it is also used to implement the following method: using the icon recognition algorithm to perform icon type recognition and integrity detection on the image frames in the events of the second event type corresponding to the icon display area; after the integrity detection passes, a preset number of events before and after the current event are selected, and the flicker recognition algorithm is used to determine the flicker frequency of the image frames in the current event and the preset number of events before and after, and then obtain the detection result.

[0177] In some embodiments, the anomaly detection algorithm includes a flicker recognition algorithm, a character recognition algorithm, an alarm recognition algorithm and a time anomaly recognition algorithm; when the computer program is executed by the processor 92, it is also used to implement the following method: using the character recognition algorithm to perform character recognition on the image frames in the event of the second event type corresponding to the character display area; when the character recognition result is a time character, using the time anomaly recognition algorithm to perform time anomaly recognition on the image frames in the event of the second event type; when the character recognition result is an alarm character, using the alarm recognition algorithm to perform alarm code recognition on the image frames in the event of the second event type; and using the flicker recognition algorithm to perform flicker anomaly detection on the image frames in the event of the second event type to obtain a detection result.

[0178] In some embodiments, when the computer program is executed by the processor 92, it is also used to implement the following method: using a time anomaly recognition algorithm to perform digital display integrity detection on image frames in events of the second event type to obtain a first detection result; and using a time anomaly recognition algorithm to compare the time characters corresponding to the current event in events of the second event type with the time characters corresponding to the previous event to obtain a second detection result; using the time anomaly recognition algorithm to identify the time length corresponding to the current event to obtain a third detection result; and using the first detection result, the second detection result, and the third detection result as the final detection result.

[0179] In some embodiments, the anomaly detection algorithm includes a flicker recognition algorithm; when the computer program is executed by the processor 92, it is also used to implement the following method: using the flicker recognition algorithm to perform flicker anomaly detection on events of the second event type corresponding to the indicator light display area to obtain a detection result.

[0180] In some embodiments, after obtaining the detection result, the computer program, when executed by the processor 92, is also used to implement the following method: marking the event of the second event type for which the detection result indicates an abnormality, and sending the marked event of the second event type to the manual terminal; receiving the confirmation result fed back by the manual terminal; when the confirmation result indicates that the marked event of the second event type is normal, using the image frame corresponding to the marked event of the second event type as a sample.

[0181] In some embodiments, before acquiring the image stream to be detected, the computer program, when executed by the processor 92, is also used to implement the following method: providing a detection configuration interface, and displaying the template image frame corresponding to the image stream to be detected on the detection configuration interface; wherein, the detection configuration interface provides several recognition algorithms to be selected; receiving the user's selection of the target display area on the template image frame, and the selection of the target recognition algorithm; binding the target display area and the target recognition algorithm.

[0182] In some embodiments, when the computer program is executed by the processor 92, it is also used to implement the method of any embodiment of the present application.

[0183] See Figure 10 , Figure 10 1 is a schematic diagram of an embodiment of a control panel detection system provided herein. The detection system 100 includes an image acquisition device 101 and a detection device 90. The image acquisition device 101 is used to capture image frames corresponding to the operation of the control panel. The detection device 90 is such as the detection device 90 provided herein.

[0184] See Figure 11 , Figure 11 1 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 110 is used to store a computer program 111. When the computer program 111 is executed by a processor, it is used to implement the following method:

[0185] Acquire an image stream to be detected; wherein the image stream to be detected includes image frames corresponding to a control panel; perform event division on the image frames in the image stream to be detected to obtain a plurality of events and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between image frames corresponding to events corresponding to the first event type, and image change occurs between image frames corresponding to events corresponding to the second event type; perform anomaly detection on events of the second event type to obtain a detection result; and store the first image frame in the event of the first event type.

[0186] In some embodiments, the control panel includes at least one display area; when the computer program 111 is executed by the processor, it is also used to implement the following method: performing event division on each display area in the image frame to obtain a number of events corresponding to each display area; performing anomaly detection on events of the second event type to obtain detection results, including: performing anomaly detection on events of the second event type corresponding to each display area to obtain detection results.

[0187] In some embodiments, the display area includes: at least one of an icon display area, a character display area, and an indicator light display area; when the computer program 111 is executed by the processor, it is also used to implement the following method: respectively obtain the anomaly detection algorithms corresponding to the icon display area, the character display area, and / or the indicator light display area; use the anomaly detection algorithm to perform anomaly detection on events of the second event type corresponding to the respective corresponding icon display areas, character display areas, and / or indicator light display areas to obtain detection results.

[0188] In some embodiments, the anomaly detection algorithm includes an icon recognition algorithm and a flicker recognition algorithm; when the computer program 111 is executed by the processor, it is also used to implement the following method: using the icon recognition algorithm to perform icon type recognition and integrity detection on the image frames in the events of the second event type corresponding to the icon display area; after the integrity detection passes, a preset number of events before and after the current event are selected, and the flicker recognition algorithm is used to determine the flicker frequency of the image frames in the current event and the preset number of events before and after, and then obtain the detection result.

[0189] In some embodiments, the anomaly detection algorithm includes a flicker recognition algorithm, a character recognition algorithm, an alarm recognition algorithm and a time anomaly recognition algorithm; when the computer program 111 is executed by the processor, it is also used to implement the following method: using the character recognition algorithm to perform character recognition on the image frames in the event of the second event type corresponding to the character display area; when the character recognition result is a time character, using the time anomaly recognition algorithm to perform time anomaly recognition on the image frames in the event of the second event type; when the character recognition result is an alarm character, using the alarm recognition algorithm to perform alarm code recognition on the image frames in the event of the second event type; and using the flicker recognition algorithm to perform flicker anomaly detection on the image frames in the event of the second event type to obtain a detection result.

[0190] In some embodiments, when the computer program 111 is executed by the processor, it is also used to implement the following method: using a time anomaly recognition algorithm to perform digital display integrity detection on image frames in events of the second event type to obtain a first detection result; and using a time anomaly recognition algorithm to compare the time characters corresponding to the current event in events of the second event type with the time characters corresponding to the previous event to obtain a second detection result; using a time anomaly recognition algorithm to identify the time length corresponding to the current event to obtain a third detection result; and using the first detection result, the second detection result, and the third detection result as the final detection result.

[0191] In some embodiments, the anomaly detection algorithm includes a flicker recognition algorithm; when the computer program 111 is executed by the processor, it is also used to implement the following method: using the flicker recognition algorithm to perform flicker anomaly detection on events of the second event type corresponding to the indicator light display area to obtain a detection result.

[0192] In some embodiments, after obtaining the detection result, the computer program 111 is also used to implement the following method when executed by the processor: marking the event of the second event type for which the detection result indicates abnormality, and sending the marked event of the second event type to the manual terminal; receiving the confirmation result fed back by the manual terminal; when the confirmation result indicates that the marked event of the second event type is normal, using the image frame corresponding to the marked event of the second event type as a sample.

[0193] In some embodiments, before obtaining the image stream to be detected, the computer program 111, when executed by the processor, is also used to implement the following method: providing a detection configuration interface, and displaying the template image frame corresponding to the image stream to be detected on the detection configuration interface; wherein, the detection configuration interface provides several recognition algorithms to be selected; receiving the user's selection of the target display area on the template image frame, and the selection of the target recognition algorithm; binding the target display area and the target recognition algorithm.

[0194] In some embodiments, when the computer program 111 is executed by a processor, it is also used to implement the method of any embodiment of the present application.

[0195] In summary, the control panel detection method, device, system and computer-readable storage medium provided in the present application obtain the image stream to be detected corresponding to the control panel; then divide the image frames in the image stream to be detected into events according to the event type, and obtain several events, and the event type corresponding to each event; wherein the event type includes a first event type and a second event type; no image change occurs between the image frames corresponding to the events corresponding to the first event type, and image change occurs between the image frames corresponding to the events corresponding to the second event type; then anomaly detection is performed on the events of the second event type to obtain a detection result, thereby detecting whether there is a display anomaly of the control panel during the image frame change process; and storing the target image frame and / or the timestamp of the target image frame in the event of the first event type; wherein the target image frame is one of the several image frames in the event of the first event type, and there is no need to store all the image frames in the event of the first event type, which not only ensures the integrity of the events in the image stream, but also reduces the storage space occupied.

[0196] Furthermore, the present application can be applied to the detection of display anomalies on the control panels of household appliances. This allows for the acquisition of images of the control panel during operation or during testing of the control panel, obtaining an image stream, and then testing the image stream to detect whether the control panel has display anomalies. In this way, occasional display anomalies and other anomalies on the control panel can be detected.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0198] If the integrated units in the other embodiments described above are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processing circuit component (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0199] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A control panel detection method, characterized in that: The method comprises: Acquire an image stream to be detected; wherein the image stream to be detected includes an image frame corresponding to the control panel; the control panel includes at least one display area; Performing event division on each display area in the image frame to obtain a plurality of events corresponding to each display area and an event type corresponding to each event; wherein the event type includes a first event type and a second event type; an event corresponding to the first event type corresponds to an event in which no image change occurs between the image frames, and an event corresponding to the second event type corresponds to an event in which an image change occurs between the image frames; Performing anomaly detection on events of the second event type corresponding to each of the display areas to obtain a detection result; Storing a target image frame in an event of the first event type and / or a timestamp of the target image frame; wherein the target image frame is one of a plurality of image frames in the event of the first event type; Before acquiring the image stream to be detected, the method further includes: Providing a detection configuration interface, and displaying the template image frame corresponding to the image stream to be detected on the detection configuration interface; wherein the detection configuration interface provides several recognition algorithms to be selected; receiving a user's selection of a target display area on the template image frame and a selection of a target recognition algorithm; The target display area is bound to the target recognition algorithm.

2. The detection method according to claim 1, wherein The display area includes at least one of an icon display area, a character display area, and an indicator light display area; The performing abnormality detection on the event of the second event type corresponding to each of the display areas to obtain a detection result includes: Respectively obtaining anomaly detection algorithms corresponding to the icon display area, the character display area, and / or the indicator light display area; The abnormality detection algorithm is used to perform abnormality detection on events of the second event type corresponding to the respective corresponding icon display areas, the character display areas and / or the indicator light display areas to obtain a detection result.

3. The detection method according to claim 2, characterized in that The anomaly detection algorithm includes an icon recognition algorithm and a flicker recognition algorithm; The performing anomaly detection on the event of the second event type corresponding to the icon display area using the anomaly detection algorithm to obtain a detection result includes: Performing icon type recognition and integrity detection on the image frames in the event of the second event type corresponding to the icon display area using the icon recognition algorithm; After the integrity test is passed, a preset number of events before and after the current event are selected, and the flicker recognition algorithm is used to determine the flicker frequency of the image frames in the current event and the preset number of events before and after, thereby obtaining a test result.

4. The detection method according to claim 2, characterized in that The anomaly detection algorithm includes a flicker recognition algorithm, a character recognition algorithm, an alarm recognition algorithm and a time anomaly recognition algorithm; The performing anomaly detection on the event of the second event type corresponding to the character display area using the anomaly detection algorithm to obtain a detection result includes: Performing character recognition on the image frame in the event of the second event type corresponding to the character display area using the character recognition algorithm; When the character recognition result is a time character, performing time anomaly recognition on the image frames in the event of the second event type using the time anomaly recognition algorithm; When the character recognition result is a warning character, using the warning recognition algorithm to perform warning code recognition on the image frame in the event of the second event type; Furthermore, the flicker recognition algorithm is used to perform flicker anomaly detection on the image frames in the event of the second event type to obtain a detection result.

5. The detection method according to claim 4, characterized in that The using the temporal anomaly recognition algorithm to perform temporal anomaly recognition on the image frames in the event of the second event type includes: performing a digital display integrity test on the image frames in the event of the second event type using the temporal anomaly recognition algorithm to obtain a first test result; and using the time anomaly recognition algorithm to compare the time character corresponding to the current event with the time character corresponding to the previous event in the events of the second event type to obtain a second detection result; Identifying the time length corresponding to the current event using the time anomaly identification algorithm to obtain a third detection result; The first detection result, the second detection result, and the third detection result are taken as the final detection result.

6. The detection method according to claim 2, characterized in that The anomaly detection algorithm includes a flicker recognition algorithm; The performing anomaly detection on the event of the second event type corresponding to the indicator light display area using the anomaly detection algorithm to obtain a detection result includes: The flicker recognition algorithm is used to perform flicker anomaly detection on the event of the second event type corresponding to the indicator light display area to obtain a detection result.

7. The detection method according to any one of claims 1 to 6, characterized in that After obtaining the detection result, the method further includes: marking an event of the second event type for which the detection result indicates an abnormality, and sending the marked event of the second event type to a manual terminal; Receiving a confirmation result fed back by the manual terminal; When the confirmation result indicates that the marked event of the second event type is normal, the image frame corresponding to the marked event of the second event type is used as a sample.

8. A detection device for a control panel, characterized in that: The detection device includes: a memory and a processor, the processor is connected to the memory, the memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the method according to any one of claims 1 to 7.

9. A control panel detection system, characterized in that: The detection system includes: an image acquisition device and a detection device, the image acquisition device is used to acquire image frames corresponding to the operation of the control panel, and the detection device is the detection device according to claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 7.

Images