Indicator light detection method, program product, electronic device and storage medium

Abstract

The invention discloses an indicator light detection method, a program product, electronic equipment and a storage medium, and relates to the technical field of indicator light detection.The indicator light detection method comprises the steps that more comprehensive multi-modal data of a target indicator light is obtained by obtaining one or more frames of image data, electrical data and coded data of the target indicator light; by calculating the information entropy of one or more frames of images carrying the information of the environment where the target indicator lamp is located, the confidence of the image data is determined, so that the obtained image data can adapt to the environment change, and by calculating the deviation degree of the electrical data, whether the target indicator lamp has an abnormal condition is determined. By determining the conflict information corresponding to the coded data, determining whether the target indicator lamp has the problem of inconsistent running states, and detecting the target indicator lamp through various data, the technical problems that the environment change cannot be adapted and the misjudgment rate is relatively high are solved, and the technical effect of improving the accuracy of the detection result is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of indicator light detection, and in particular to a detection method, a program product, an electronic device, and a storage medium for an indicator light. Background Art

[0002] The detection of indicator lights has long relied on manual visual judgment, resulting in low efficiency and a high misjudgment rate. In related technologies, the indicator lights are mainly detected by single-modal detection and static fusion. The above methods monitor the operating state of the indicator lights through safety decision-making, brightness uniformity analysis, and stability analysis, and combine with user terminals for fault feedback. However, the above methods cannot adapt to environmental changes and have a high misjudgment rate. Summary of the Invention

[0003] The present application provides a detection method, a program product, an electronic device, and a storage medium for an indicator light, so as to at least solve the problem that the detection method in related technologies cannot adapt to environmental changes and has a high misjudgment rate.

[0004] The present application provides a detection method for an indicator light, including: Obtaining at least one frame of image data, electrical data, and coding data corresponding to a target indicator light; Based on the at least one frame of image data, the electrical data, and the coding data, respectively calculating the information entropy corresponding to the at least one frame of image data, the deviation degree corresponding to the electrical data, and the conflict information corresponding to the coding data; Based on the information entropy, the deviation degree, and the conflict information, determining a detection score corresponding to the target indicator light; Based on the detection score, determining a detection result corresponding to the target indicator light.

[0005] The present application further provides a computer program product, including: A first processing module, configured to obtain at least one frame of image data, electrical data, and coding data corresponding to a target indicator light; A second processing module, configured to respectively calculate the information entropy corresponding to the at least one frame of image data, the deviation degree corresponding to the electrical data, and the conflict information corresponding to the coding data based on the at least one frame of image data, the electrical data, and the coding data; A third processing module, configured to determine a detection score corresponding to the target indicator light based on the information entropy, the deviation degree, and the conflict information; A fourth processing module, configured to determine a detection result corresponding to the target indicator light based on the detection score.

[0006] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above-mentioned indicator detection methods when executing the computer program.

[0007] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of any of the above-mentioned indicator detection methods when executed by a processor.

[0008] The present application also provides a computer program product including a computer program, and the computer program implements the steps of any of the above-mentioned indicator detection methods when executed by a processor.

[0009] Through the present application, by obtaining one or more frames of image data, electrical data, and coding data of the target indicator, more comprehensive multi-modal data of the target indicator is obtained, enabling a better understanding of the operating conditions of the target indicator. By calculating the information entropy of one or more frames of images carrying the environmental information of the target indicator, the confidence level of the image data is determined, enabling the acquired image data to adapt to environmental changes. By calculating the deviation degree of the electrical data, it is determined whether there is an abnormal situation with the target indicator. By determining the conflict information corresponding to the coding data, it is determined whether there is a problem of inconsistent operating states with the target indicator. Through multiple types of data, the target indicator is jointly detected, and a detection score is calculated. Therefore, the technical problem that the detection method cannot adapt to environmental changes and has a high false positive rate can be solved, achieving the technical effect of improving the accuracy of the detection result. Description of the Drawings

[0010] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 One of the flow diagrams of a method for detecting an indicator provided by an embodiment of the present application; Figure 2 One of the principle diagrams of a method for detecting an indicator provided by an embodiment of the present application; Figure 3 Another principle diagram of a method for detecting an indicator provided by an embodiment of the present application; Figure 4 Another flow diagram of a method for detecting an indicator provided by an embodiment of the present application; Figure 5 The structural diagram of a computer program product provided by an embodiment of the present application. Detailed implementation manners

[0012] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the protection scope of the present application.

[0013] It should be noted that in the description of the present application, the terms "including", "comprising" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0015] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the detection method of the indicator light depends, the specific application environment architecture or specific hardware architecture will be described herein.

[0016] An embodiment of the present application provides a method for detecting an indicator light. The method will be described in detail in combination with the execution process of the detection method of the indicator light.

[0017] Specifically, Figure 1 is a flowchart of a method for detecting an indicator light according to an embodiment of the present application.

[0018] As Figure 1 shown, the method for detecting the indicator light includes: step 110, step 120, step 130, and step 140.

[0019] Step 110: Obtain at least one frame of image data, electrical data, and coding data corresponding to the target indicator light; In this step, the target indicator light is the LED light to be detected.

[0020] The target indicator light can be different types of indicator lights such as server LEDs, indicator lights in industrial equipment status monitoring scenarios, indicator lights in smart home scenarios, and indicator lights in in-vehicle electronics scenarios.

[0021] For example, in an industrial equipment status monitoring scenario, the motor indicator light is detected by fusing vibration sensor data.

[0022] Optimizing the diagnosis of lamp status by combining user habit data in the smart home scenario.

[0023] Increasing the temperature mode in the in-vehicle electronics scenario to enhance the detection of dashboard LEDs.

[0024] The image data is the image of the target indicator light collected.

[0025] The electrical data is the relevant data such as the voltage, current, and power of the target indicator light collected.

[0026] The encoded data is the preset code obtained from the register, representing the working state of the target indicator light.

[0027] For example, the preset code is 0b101, indicating that the target indicator light flashes red.

[0028] During the actual execution process, one or more frames of image data can be collected through an image sensor. For example, image data can be collected through a Basler ace2 2440-120uc industrial camera.

[0029] It should be noted that the Basler ace2 2440-120uc industrial camera (120fps, global shutter) can take high-speed pictures without smear and eliminate the interference of ambient light color temperature.

[0030] Of course, during the actual execution process, other different types of image sensors can also be used.

[0031] The electrical data can be collected through measuring devices such as a voltage meter, an ammeter, a multimeter, and an electrical sensor.

[0032] The encoded data can be read from the status register.

[0033] Such as Figure 2 As shown, during the actual execution process, image data, electrical data, and encoded data can be obtained through the data acquisition layer. That is, through an industrial camera, an RGB image stream is collected to obtain one or more frames of video streams; electrical data such as voltage and current is collected through an electrical sensor, and the preset code (i.e., the encoded data) is obtained through the status register.

[0034] Step 120: Based on at least one frame of image data, electrical data, and encoded data, calculate the information entropy corresponding to at least one frame of image data, the deviation degree corresponding to the electrical data, and the conflict information corresponding to the encoded data respectively; In this step, the information entropy is a value quantifying the confidence of the image data.

[0035] The information entropy can be expressed as .

[0036] It should be noted that information entropy is the core indicator for measuring uncertainty in information theory. The larger the value of information entropy, the higher the uncertainty (degree of chaos) of the system and the lower the confidence.

[0037] In the actual implementation process, one or more frames of images may be processed first, pixel information in the image data may be counted, and the information entropy of the image data may be calculated based on the pixel information obtained by the statistics and the calculation formula of the information entropy.

[0038] It should be noted that information entropy has the advantages of strong probability sensitivity, excellent mathematical properties and wide applicability.

[0039] Specifically, entropy is highly sensitive to the uniformity of probability distribution and can intuitively reflect the "hesitation level" of the model.

[0040] The entropy function performs well in terms of convexity and differentiability.

[0041] Entropy is a standard method for uncertainty measurement in multi-classification problems (such as decision trees, Bayesian models).

[0042] The deviation is a measure of the difference between the electrical data of the target indicator light at the current acquisition moment and the normal operating state.

[0043] Deviation is information that can quantify the degree of abnormality of electrical data.

[0044] The deviation can be expressed as .

[0045] It should be noted that, when the electrical data includes a variety of different types of data, such as current and voltage data, the deviation is an indicator that integrates the voltage and current abnormalities.

[0046] In the actual implementation process, different electrical data of the target indicator light can be compared with the electrical data under normal operating conditions to obtain the deviation.

[0047] For example, the collected current of the target indicator light is compared with the voltage of the target indicator light under a preset normal operating state to obtain the current deviation.

[0048] For another example, the collected voltage of the target indicator light is compared with the voltage of the target indicator light under a preset normal operating state to obtain the voltage deviation.

[0049] The conflict information is whether the actual working status of the target indicator light is consistent with the coded data.

[0050] The conflict information includes: conflict or no conflict.

[0051] The conflict information can be obtained by comparing the actual working state of the target indicator light with the encoded data read from the status register.

[0052] For example, when the actual working state of the target indicator light is that the red light is flashing and the encoded data indicates that the green light is flashing, the conflict information is conflict.

[0053] During the actual execution process, the electrical signal of the target indicator light working can be captured by an oscilloscope to determine the actual working state of the target indicator light.

[0054] Step 130: Determine the detection score corresponding to the target indicator light based on information entropy, deviation, and conflict information; In this step, the detection score is a credibility score obtained by fusing the multimodal data of the target indicator light.

[0055] The detection score of the target indicator light can be calculated by summing up the information entropy, deviation, and conflict information.

[0056] During the actual execution process, the information entropy, deviation, and conflict information can also be normalized to obtain the detection score.

[0057] Step 140: Determine the detection result corresponding to the target indicator light based on the detection score.

[0058] In this step, during the actual execution process, the detection score can be compared with the first threshold to determine the detection result corresponding to the target indicator light.

[0059] When the detection score is less than the first threshold, it is determined that the detection result is detection anomaly and an alarm is triggered; When the detection score is greater than or equal to the first threshold, the detection result is passing the detection.

[0060] The first threshold is a preset value for determining whether early warning is needed.

[0061] The specific value of the first threshold can be user-defined or determined based on the actual situation. For example, the first threshold can be 0.8 or 0.7; this application does not make a limitation.

[0062] During the actual execution process, taking the detection score of 0.62 as an example, an alarm can be triggered to timely remind the maintenance personnel to pay attention to the abnormal situation.

[0063] The inventor found during the R & D process that in the related art, the indicator light is mainly detected by means of unimodal detection and static fusion. The above method monitors the operation state of the indicator light through safety decision-making, brightness uniformity analysis, and stability analysis, and combines with the user terminal for fault feedback. However, the above method cannot adapt to environmental changes and has a high misjudgment rate.

[0064] In this application, by obtaining one or more frames of image data, electrical data, and coding data of the target indicator light, more comprehensive multi-modal data of the target indicator light can be obtained, enabling a better understanding of the operating conditions of the target indicator light. By calculating the information entropy of one or more frames of images carrying the environmental information of the target indicator light, the confidence level of the image data is determined, enabling the obtained image data to adapt to environmental changes. By calculating the deviation degree of the electrical data, it is determined whether there is an abnormal situation with the target indicator light. By determining the conflict information corresponding to the coding data, it is determined whether there is a problem with inconsistent operating states of the target indicator light. Through multiple types of data, the target indicator light is jointly detected, and a detection score is calculated to improve the accuracy of the detection result.

[0065] According to the indicator light detection method provided by the embodiments of this application, by obtaining one or more frames of image data, electrical data, and coding data of the target indicator light, and respectively processing the image data, electrical data, and coding data to obtain the information entropy, deviation degree, and conflict information, through multiple types of data, the target indicator light is jointly detected, and a detection score is calculated to improve the accuracy of the detection result.

[0066] In some embodiments, step 120 may further include: Based on at least one frame of image data, determine the color probability distribution and lighting state corresponding to the target indicator light; Based on the color probability distribution, calculate the information entropy corresponding to at least one frame of image data; Based on the electrical data and preset standard electrical data, calculate the deviation degree corresponding to the electrical data; Based on at least one frame of image data and the coding data, determine the conflict information corresponding to the coding data.

[0067] In this embodiment, the color probability distribution is the RGB color distribution of the image data.

[0068] The lighting state is the lighting condition of the target indicator light.

[0069] The lighting state includes: lighting color, lighting frequency, etc.

[0070] The lighting color includes: red, green, and blue light colors.

[0071] The lighting frequency includes: constant lighting, flashing, etc.

[0072] For example, the lighting state can be: green light constantly on and red light flashing, etc.

[0073] During the actual execution process, the pixel colors of each pixel point can be statistically analyzed, and based on the total number of pixel points and the number of pixel points of different colors, the color probability distribution is calculated, and the lighting state is determined based on the change situation of multiple frames of image data.

[0074] After obtaining the color probability distribution, calculate the information entropy corresponding to the image data through the information entropy calculation formula.

[0075] In the actual execution process, the information entropy can be calculated based on the following formula:

[0076] where is the information entropy; is the probability distribution corresponding to color i.

[0077] can be (red probability), (green probability), and (blue probability).

[0078] .

[0079] During the detection of the target indicator light, when the color judgment is very certain, the red probability = 0.95, the green probability = 0.03, the blue probability = 0.02, then (low uncertainty).

[0080] In the case where the color cannot be distinguished, such as the red probability = 0.34, the green probability = 0.33, the blue probability = 0.33, then (high uncertainty).

[0081] The preset standard electrical data is the standard working electrical data of the target indicator light.

[0082] In the actual execution process, after obtaining the electrical data, the deviation degree can be calculated based on the following formula:

[0083] where is the deviation degree; is the voltage value collected in real time (unit: volt, V); is the nominal voltage (the standard working voltage of the target indicator light); is the standard deviation of the current (unit: ampere, A); is the average value of the current (unit: ampere, A).

[0084] Among them, the deviation degree of the voltage is , and the current fluctuation coefficient is 。

[0085] Taking = 4.0V, = 3.3V, = 3 mA, = 20 mA as an example, the calculation is as follows:

[0086] Deviation > 0.3, it can be determined as an electrical anomaly; ≤ 0.1, it can be determined that the electrical parameters are normal; when ∈ (0.1, 0.3) indicates that there are controllable deviations in the electrical parameters.

[0087] In the actual execution process, the conflict information can be determined by comparing the actual working state corresponding to multiple image data with the working state represented by the encoded data.

[0088] The actual working state can be obtained by analyzing the changes between multiple images.

[0089] According to the detection method of the indicator light provided in the embodiments of the present application, by processing one or more frames of image data, the color probability distribution and the lighting state corresponding to the target indicator light are obtained, effectively identifying the color and blinking situation of the target indicator light through the image data. Based on the color probability distribution, the information entropy of the image data is calculated, improving the accuracy of the calculated information entropy. By comparing the electrical data with the preset standard electrical data, the deviation degree of the electrical data is calculated, providing a clear comparison standard, making the calculated deviation degree more in line with the actual usage scenario. Through one or more frames of image data and the encoded data, the conflict information can be directly compared, improving the efficiency of determining the conflict information.

[0090] In some embodiments, based on at least one frame of image data, determining the color probability distribution and the lighting state corresponding to the target indicator light may further include: Inputting at least one frame of image data into the target recognition model to obtain the color probability distribution output by the target recognition model; Determining the lighting state based on the brightness change of consecutive frames of images of the target quantity in at least one frame of image data.

[0091] In this embodiment, the target recognition model is a model for image recognition of one or more frames of image data.

[0092] The target recognition model can be a neural network model, a machine learning model, etc.

[0093] For example, the target recognition model can be a YOLO-Lite model, NanoDet, etc.

[0094] Of course, in the actual execution process, the target recognition model can also be any other model that can recognize image data, and the present application does not make any limitations.

[0095] The number of targets is a preset value of the number of image data required to determine the lighting state of the target indicator light.

[0096] The specific value of the number of targets can be determined based on user-defined or based on the actual situation. For example, the number of targets can be 5 or 7; the present application does not make any limitations.

[0097] In the actual execution process, after obtaining one or more frames of image data, the image data can be input into the YOLO-Lite model. The YOLO-Lite model recognizes the image data to obtain the LED color, that is, the RGB probability distribution of the image, and outputs the color probability distribution: red probability ( ), green probability ( ), blue probability ( ).

[0098] After obtaining multiple frames of images, select consecutive frames of images with the number of targets. For example, select consecutive 5 frames of image data from multiple frames of images, compare the magnitude relationship between the brightness change of the consecutive 5 frames of image data and the brightness change threshold, and determine the lighting state based on the magnitude relationship.

[0099] The brightness change threshold is a preset value for determining whether the target indicator light blinks or is constantly on.

[0100] The specific value of the brightness change threshold can be determined based on the actual situation. For example, the brightness change threshold can be 30% or 40%; the present application does not make any limitations.

[0101] Taking the brightness change threshold of 30% as an example, when the brightness change of the consecutive 5 frames of image data ≥ 30%, the lighting state can be determined as blinking.

[0102] When the brightness change of the consecutive 5 frames of image data < 30%, the lighting state can be determined as constantly on.

[0103] According to the detection method of the indicator light provided by the embodiments of the present application, by processing one or more frames of image data through the target recognition model, the color probability distribution of the image data is obtained, improving the efficiency of obtaining the color probability distribution. By the brightness change of the consecutive frames of images with the number of targets, the brightness change situation of the target indicator light during the image acquisition period is determined, so as to determine the lighting situation of the target indicator light, improving the accuracy of the determined lighting state.

[0104] In some embodiments, based on at least one frame of image data and the encoded data, determining the conflict information corresponding to the encoded data may further include: Perform visual recognition on at least one frame of image to obtain a recognition result; the recognition result is the lighting result corresponding to the target indicator light; When the recognition result is consistent with the lighting state of the indicator light corresponding to the encoded data, determine that the conflict information is non-conflicting; When the recognition result is inconsistent with the lighting state of the indicator light corresponding to the encoded data, determine that the conflict information is conflicting.

[0105] In this embodiment, after obtaining one or more frames of image data, the image data can be subjected to image recognition through an image recognition model to obtain a recognition result output by the image recognition model. For example, the recognition result is that the red light is always on, or the green light is flashing, etc.

[0106] After obtaining the recognition result, compare the recognition result with the lighting state corresponding to the encoded data.

[0107] For example, the recognition result is "the green light is always on", and the encoded data is 0b101, indicating that the red light is flashing. The conflict information is conflicting, that is = 1; Another example is that the recognition result is "the red light is flashing", and the encoded data is 0b101, indicating that the red light flashing code is consistent with the recognition result. The conflict information is non-conflicting, that is = 0.

[0108] According to the detection method of the indicator light provided by the embodiment of the present application, by performing visual recognition on the image data to obtain a recognition result, effectively determine the actual lighting situation of the target indicator light, compare the recognition result with the encoded data to obtain conflict information, and the judgment logic is simple and easy to operate.

[0109] In some embodiments, based on information entropy, deviation, and conflict information, determining the detection score corresponding to the target indicator light may further include: Calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the encoded data based on information entropy, deviation, and conflict information respectively; Determine the detection score corresponding to the target indicator light by weighting the information entropy with the first weight, weighting the deviation with the second weight, and weighting the conflict information with the third weight.

[0110] In this embodiment, after obtaining the information entropy, deviation, and conflict information, the first weight, the second weight, and the third weight can be calculated based on the actual values of the information entropy, deviation, and conflict information.

[0111] In the actual execution process, the first weight, the second weight, and the third weight can be calculated respectively based on the following formulas:

[0112] Among them, is the weight; and are the basic weights; and are the deviation degrees of the data; is the dynamic decay factor; .

[0113] The basic weights can be optimized through grid search (range: 0.1 - 0.8) to determine the value with the highest comprehensive accuracy.

[0114] can be (visual basic weight), (electrical basic weight), and (flag bit basic weight, that is, the weight corresponding to the encoded data).

[0115] The specific values of the basic weights can be determined based on user-defined or actual situations, and this application does not make any limitations.

[0116] For example, the visual basic weight is 0.6; the electrical basic weight is 0.3; the flag bit basic weight is 0.1.

[0117] Another example, the visual basic weight is 0.4; the electrical basic weight is 0.4; the flag bit basic weight is 0.2.

[0118] The dynamic decay factor can control the influence intensity of uncertainty on the weight.

[0119] The specific value of the dynamic decay factor can be determined based on actual situations, and this application does not make any limitations. Experiments show that when the value is 2.0, it can effectively balance the weight sensitivity and stability.

[0120] includes (information entropy), (deviation degree), and (conflicting information).

[0121] is the confidence fluctuation of the visual recognition result (such as color misjudgment caused by light changes and occlusion).

[0122] is the electrical parameter abnormality (such as voltage deviation from the nominal value and current fluctuation, etc.).

[0123] is the conflict flag between the hardware preset state and the real-time detection result (such as the register encoding is inconsistent with the actual LED state), that is .

[0124] Taking = 0.4, = 0.2, Taking = 0 as an example, the calculated first weight, second weight, and third weight can be respectively:

[0125]

[0126]

[0127] Among them, is the first weight; is the second weight; is the third weight.

[0128] is the core control parameter of multimodal fusion, which can reflect the reliability of each modal data in real time. Through the exponential decay function exp(-β ), the uncertainty is mapped to the weight adjustment coefficient.

[0129] The smaller, the higher the visual confidence, and the larger the overall weight ; The larger, the less reliable the visual result, and the impact on decision-making is reduced.

[0130] Deviation > 0.3, it is determined as electrical abnormality, and the electrical weight is reduced; ≤ 0.1, the electrical parameters are normal, and the high weight is maintained. When ∈ (0.1, 0.3), it means that there are controllable deviations in the electrical parameters, but it has not reached the emergency fault level. The second weight is adaptively adjusted through the weight calculation formula.

[0131] When the conflict information is in conflict, that is, = 1, the flag weight is reduced from 0.1 to 0.05 to reduce its impact on decision-making; when the conflict information is not in conflict, that is, = 0, the flag is credible, and the basic weight can be maintained.

[0132] It should be noted that through the above formula, the visual basic weight, electrical basic weight, and flag basic weight can be dynamically adjusted to obtain the first weight, second weight, and third weight, which can automatically reduce the decision weight of unreliable data in a complex environment, improve the robustness of the detection process, and achieve dynamic adaptability.

[0133] During the process of dynamically adjusting the weight, if the vision is interfered (such as causing to become larger), it will cause The weight decreases and relies on electrical and flag bit compensation, that is, increasing one or more of the second weight and the third weight.

[0134] Electrical anomalies (such as causing to increase) will cause the weight to decrease and the visual weight to increase.

[0135] Flag bit conflicts (such as = 1) will cause the weight to decrease and reduce the impact of the conflicting mode.

[0136] As shown in Table 1, it describes the relevant data in the process of dynamic weight adjustment.

[0137] Table 1

[0138] As Figure 3 shown, it describes the change process of the weight value in the process of dynamic weight adjustment.

[0139] Visual weight (the top curve): t = 0~10s: The initial weight is 0.6, and the visual data confidence is high.

[0140] t = 10s: The voltage exceeds the limit, and the visual weight rises to 0.7 (the electrical weight decreases and the visual compensation occurs).

[0141] t = 30s: Visual occlusion causes the confidence to decrease, and the weight drops to 0.4.

[0142] Electrical weight (the middle curve): t = 10s: The voltage exceeding the limit triggers a deviation Ue = 0.262, and the weight drops suddenly from 0.3 to 0.15.

[0143] t = 30s: When there is visual occlusion, the electrical parameters are stable (Ue = 0.05), and the weight is compensated to 0.4.

[0144] Flag bit weight (the bottom curve): t = 10s: Due to voltage anomaly and flag bit conflict, the weight rises from 0.1 to 0.15.

[0145] t = 30s: When there is no flag bit conflict during visual occlusion, the weight returns to 0.2.

[0146] Key event annotation: Voltage exceeding the limit (t = 10s): The electrical weight decreases and the visual weight is compensated.

[0147] Visual occlusion (t = 30s): The visual weight decreases and the electrical weight is compensated.

[0148] After obtaining the first weight, the second weight, and the third weight, the detection score corresponding to the target indicator light is determined by weighting the information entropy with the first weight, weighting the deviation with the second weight, and weighting the conflict information with the third weight.

[0149] During the actual execution process, the detection score can be calculated based on the following formula:

[0150] where is the detection score; is the first weight; is the information entropy; is the second weight; is the deviation; is the third weight; is the conflict information.

[0151] When the calculated detection score is less than the first threshold, an alarm is triggered.

[0152] For example, the nominal voltage is 3.3V and the measured voltage is 4V. =(|4.0 - 3.3|) / 3.3 + 0.05 = 0.262. Weight adjustment: The electrical weight (the second weight) is reduced from 0.3 to 0.15, and the visual weight (the first weight) is increased to 0.7.

[0153] Result: The detection score is 0.62, and an alarm is triggered.

[0154] Another example, electrical parameters: The current is stable ( = 1mA), = 0.05.

[0155] Weight adjustment: The electrical weight (the second weight) is compensated to 0.4, and the comprehensive score is 0.82, passing the detection.

[0156] According to the detection method of the indicator light provided by the embodiments of the present application, by calculating the first weight, the second weight, and the third weight respectively through the information entropy, the deviation, and the conflict information, it is possible to automatically reduce the decision weight of unreliable data in a complex environment, improve the robustness of the detection process, achieve dynamic adaptability, and thus calculate the detection score based on the weights adapted to the dynamic environment, improving the accuracy of the calculated detection score.

[0157] As Figure 4 shown, in some embodiments, calculating the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the encoded data respectively based on the information entropy, the deviation, and the conflict information includes: When the degrees of change of at least one frame of image data, electrical data, and coding data within the target period are all less than the first change threshold, calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on information entropy, deviation degree, and conflict information.

[0158] In this embodiment, the target period is a preset value for determining whether there is a mutation in the acquired data.

[0159] The specific value of the target period can be determined based on the actual situation. For example, the target period can be 3 seconds or 5 seconds, etc., and this application does not make a limitation.

[0160] The degree of change is the degree of fluctuation of the data.

[0161] The first change threshold is a preset value for determining the fluctuation situation of the acquired data.

[0162] The first change threshold can include a color change threshold, an electrical data change threshold, and a coding data change threshold.

[0163] During the actual execution process, multiple frames of image data, electrical data, and coding data within the target period can be acquired, the degrees of change of various types of data within the target period can be calculated, and the degrees of change of various types of data are respectively compared with their corresponding thresholds. When the degrees of change are all less than the first change threshold, it is determined that the multiple frames of image data, electrical data, and coding data within the acquired target period are data without mutation. At this time, based on information entropy, deviation degree, and conflict information, calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data.

[0164] According to the detection method of the indicator light provided by the embodiments of the present application, by comparing the magnitude relationship between the degrees of change of at least one frame of image data, electrical data, and coding data within the target period and the first change threshold, to determine whether there is a mutation in the data within the target period, effectively judge the accuracy of the acquired image data, electrical data, and coding data, and thus calculate the first weight, the second weight, and the third weight based on the information entropy, deviation degree, and conflict information calculated from the accurate data, improving the accuracy and usability of the calculated weights.

[0165] Continue to refer to Figure 4 In some embodiments, when the degrees of change of at least one frame of image data, electrical data, and coding data within the target period are all less than the first change threshold, calculating the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on information entropy, deviation degree, and conflict information may further include: When at least one of the information entropy being greater than the information entropy threshold, the deviation being greater than the deviation threshold, and the conflict information being in conflict exists, and the change degrees of at least one frame of image data, electrical data, and coding data within the target time period are all less than the first change threshold, calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on the information entropy, the deviation, and the conflict information.

[0166] In this embodiment, the information entropy threshold is a preset value for judging the confidence level of the image data.

[0167] The deviation threshold is a preset value for judging whether the electrical data is abnormal.

[0168] In the actual execution process, the specific values of the information entropy threshold and the deviation threshold can be determined based on the actual situation, and this application does not make any limitations.

[0169] In the actual execution process, when the information entropy is greater than the information entropy, it indicates that the confidence level of the image data is low, and it can be determined that a multimodal conflict is detected.

[0170] When the deviation is greater than the deviation threshold, it indicates that the electrical data is abnormal, and it can be determined that a multimodal conflict is detected.

[0171] When the conflict information is in conflict, it can also be determined that a multimodal conflict is detected.

[0172] As Figure 4 shown, when a multimodal conflict is detected, primary arbitration can be performed. For example, check the consistency of the data collected within the previous 3 seconds.

[0173] When the data collected within 3 seconds is consistent, calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data based on the information entropy, the deviation, and the conflict information.

[0174] According to the indicator light detection method provided by the embodiments of the present application, by comparing the magnitude relationship between the change degree of the collected data and the first change threshold when a multimodal conflict is detected to calculate the corresponding weights, and performing primary arbitration when a conflict is found, the occurrence of invalid arbitration is reduced.

[0175] In some embodiments, calculating the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on the information entropy, the deviation, and the conflict information includes: When there is at least one degree of change among the degrees of change of at least one frame of image data, electrical data, and coding data within the target time period that is not less than the first change threshold, match the at least one frame of image data, electrical data, and coding data with the preset standard mode information; Calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on information entropy, deviation degree, and conflict information; When the at least one frame of image data, electrical data, and coding data match the preset standard mode information, replace the first weight, the second weight, and the third weight with the preset weights.

[0176] In this embodiment, the preset standard mode information is the state combination of the indicator lights collected in advance.

[0177] The preset standard mode information is used for mode matching during conflict arbitration.

[0178] The preset standard mode information is the recorded historical fault cases and the information under normal operating conditions, such as the LED state sequences corresponding to voltage overlimit and communication interruption, etc.

[0179] Each state sequence in the preset standard mode information also corresponds to weighted information, that is, the preset weight corresponding to each state sequence.

[0180] The preset weight is the weight corresponding to the state sequence in the preset standard mode information that matches the actual data of the target indicator light.

[0181] It can include electrical anomalies: voltage overlimit (such as actual 4.0V, nominal 3.3V), current fluctuation (such as >0.1); visual anomalies: color deviation, abnormal flicker frequency; state flag bit conflict: the register coding is inconsistent with the actual LED state (such as coding 0b101 corresponding to "red light flashing", but actually the green light is always on).

[0182] Under normal working conditions, the LED states include standard states such as LED startup self-check, running, and sleep. Collect the color RGB distribution (such as red: >0.9), stable current value, and flag bit consistency under normal conditions.

[0183] During the actual execution process, time-domain features can be extracted for each state sequence: color probability distribution, flicker determination (i.e., lighting state), electrical deviation degree, state flag bit feature.

[0184] After obtaining the historical fault cases and the information under normal operating conditions, mark the data in a manual annotation manner, such as "fault A", "normal B", to obtain the preset standard mode information.

[0185] In the case that there is one or more degrees of change in the degrees of change of the image data, electrical data, and coding data within the determined target period that are not less than the first change threshold, the collected image data, electrical data, and coding data can be matched with the preset standard mode information to obtain a matching result.

[0186] In the actual execution process, the weighted Euclidean distance can be used to measure the matching degree between the real-time data and the pattern library, that is, the image data, electrical data, and coding data are matched with the preset standard mode information through the following formula:

[0187] where is the matching degree, , and are the visual weight, electrical weight, and flag bit weight respectively, , and are the clustering centers of various types of data respectively; is the information entropy, is the deviation degree, is the conflict information.

[0188] In the case that the calculated matching degree is less than the matching degree threshold, it can be determined that the image data, electrical data, and coding data match the preset standard mode information.

[0189] The specific value of the matching degree threshold can be based on user definition or determined based on the actual situation. For example, the matching degree threshold can be 0.1 or 0.2; this application does not make a limitation.

[0190] Taking the matching degree threshold of 0.1 as an example, in the case that the calculated D < 0.1, it can be determined that the matching is successful, and the preset weight override strategy is triggered, that is, the preset weight overrides the first weight, the second weight, and the third weight.

[0191] As Figure 4 shown, in the case of inconsistent primary arbitration, further intermediate arbitration is performed, and the collected data is matched with the pre-stored LED pattern library (i.e., the preset standard mode information). In the case of a match, the calculated weight is overridden.

[0192] According to the detection method of the indicator light provided by the embodiments of the present application, when it is determined that there is one or more degrees of change in the collected data that are not less than the first change threshold, the collected image data, electrical data, and coding data are further matched with the preset standard mode information. When the corresponding data is matched from the preset standard mode information, the first weight, the second weight, and the third weight calculated based on the information entropy, the deviation degree, and the conflict information are overwritten by the preset weight, so as to improve the usability of the replaced first weight, second weight, and third weight.

[0193] Continuing to refer to Figure 4 , in some embodiments, calculating the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on the information entropy, the deviation degree, and the conflict information may further include: When there is at least one degree of change in at least one frame of image data, electrical data, and coding data within the target time period that is not less than the first change threshold, matching the at least one frame of image data, electrical data, and coding data with the preset standard mode information; When the target data in the at least one frame of image data, electrical data, and coding data does not match the preset standard mode information, obtaining the abnormal information confirmed by the user; Based on the abnormal information, adjusting the first weight, the second weight, and the third weight to obtain a new first weight, a new second weight, and a new third weight for the indicator light.

[0194] In this embodiment, the abnormal information is the abnormal type confirmed manually.

[0195] The abnormal information may include: information such as hardware failure and environmental interference.

[0196] The new first weight is the new first weight obtained after adjusting the first weight based on the abnormal information.

[0197] The new second weight is the new second weight obtained after adjusting the second weight based on the abnormal information.

[0198] The new third weight is the new third weight obtained after adjusting the third weight based on the abnormal information.

[0199] As Figure 4 shown, when both the primary arbitration and the intermediate arbitration fail, the ultimate arbitration is further triggered. By means of manual intervention, the abnormal information is confirmed, and based on the abnormal information, the first weight, the second weight, and the third weight are adjusted.

[0200] Taking the color mixing and communication interruption scenarios as an example, the detection process of the target indicator light is described.

[0201] The LED flashes alternately between red and green (hardware preset to "red light on constantly"), and at the same time the flag bit fails ( = 1).

[0202] The processing flow is as follows: Primary arbitration: When it is detected that the conflict between vision and the flag bit lasts for 3 seconds, intermediate arbitration is triggered.

[0203] Intermediate arbitration: Match the pattern library. If the match is successful, overwrite the current weight (the vision weight is increased to 0.7); if the match fails, enter the ultimate arbitration.

[0204] Ultimate arbitration: Manually confirm the communication failure, update the pattern library and correct the flag bit weight formula (αf is adjusted from 0.1 to 0.05).

[0205] According to the detection method of the indicator light provided by the embodiment of the present application, when it is determined that the change degree of the collected data is not less than the first change threshold and the collected data does not match the preset standard pattern information, through the way of manual intervention, obtain the abnormal information confirmed by the user, obtain the abnormal situation of the target indicator light, and thus, based on the abnormal information, adjust the first weight, the second weight and the third weight to improve the accuracy of the obtained new first weight, new second weight and new third weight.

[0206] In some embodiments, adjusting the first weight, the second weight and the third weight based on the abnormal information to obtain the new first weight, new second weight and new third weight may further include: Adjust the dynamic decay factor, the visual base weight corresponding to the first weight, the electrical base weight corresponding to the second weight and the flag bit base weight corresponding to the third weight respectively; Based on the adjusted dynamic decay factor, adjusted visual base weight, adjusted electrical base weight and adjusted flag bit base weight, determine the new first weight, new second weight and new third weight.

[0207] In this embodiment, during the actual execution process, after obtaining the abnormal information, the dynamic decay factor, the visual base weight corresponding to the first weight, the electrical base weight corresponding to the second weight and the flag bit base weight corresponding to the third weight can be adjusted based on the actual situation of the abnormal information, and thus, based on the adjusted dynamic decay factor, adjusted visual base weight, adjusted electrical base weight, adjusted flag bit base weight, information entropy, deviation degree and conflict information, calculate the new first weight, new second weight and third weight.

[0208] In some embodiments, different image data, electrical data, and coding data can be collected in advance and the features of the collected different data can be learned through a machine learning model, so as to learn the adjustment ranges of the dynamic attenuation factor, visual base weight, electrical base weight, and flag bit base weight.

[0209] In other embodiments, adjustment data input by a user can also be received to adjust the dynamic attenuation factor, the visual base weight corresponding to the first weight, the electrical base weight corresponding to the second weight, and the flag bit base weight corresponding to the third weight.

[0210] According to the detection method of the indicator light provided by the embodiments of the present application, by adjusting the dynamic attenuation factor, visual base weight, electrical base weight, and flag bit base weight, the calculation methods of the new first weight, new second weight, and new third weight are effectively adjusted, the fitness of the calculated new weights to the actual situation is improved, it is more in line with the actual situation, and the accuracy of the new weights is improved.

[0211] Continue to refer to Figure 4 , in some embodiments, after obtaining the abnormal information confirmed by the user, the method may further include: Storing the abnormal information into a circular buffer; Training a target detection model based on the data in the circular buffer, where the target detection model is used to detect the target indicator light.

[0212] In this embodiment, the circular buffer is an area for storing sample data.

[0213] After obtaining the abnormal information confirmed by the user, the abnormal information and its related abnormal data can be added to the circular buffer for training the user model.

[0214] In the actual execution process, the abnormal data can be used as sample data, and the abnormal information can be used as a sample label to train the target detection model for online learning and continuously optimize the target detection model.

[0215] Taking the scenarios of LED surface contamination and current signal fluctuation as examples, the detection process of the target indicator light is described.

[0216] Dust covers the LED lamp shade (light transmittance decreases), and the current signal fluctuates.

[0217] Vision: Recognized as weak red light (Pr = 0.7), but the flicker detection fails.

[0218] Electricity: The current signal contains high-frequency noise.

[0219] Final arbitration: Manually recheck and confirm "red light always on", update the mode library (store the manual recheck result in the circular buffer) and reset the flag bit weight.

[0220] According to the detection method of the indicator light provided by the embodiment of the present application, the continuously obtained abnormal information is stored in the circular buffer, and the target detection model is optimized based on the data in the circular buffer, so as to optimize the detection accuracy and detection efficiency of the target detection model.

[0221] In some embodiments, training the target detection model based on the data in the circular buffer may further include: When the data volume in the circular buffer is greater than the target data volume, an old task data set and a new task data set are obtained based on the acquisition times of the data in the circular buffer; Train the target detection model based on the old task data set and the new task data set.

[0222] In this embodiment, the target data volume is to judge whether the data volume of the circular buffer reaches the preset value for training the target detection model.

[0223] The specific value of the target data volume can be user-defined or determined based on the actual situation. For example, the target data volume can be 1000 or 1500, etc.; the present application does not make a limitation.

[0224] The old task data set is a data set composed of the data with relatively early acquisition times in the circular buffer.

[0225] The new task data set is a data set composed of the data with relatively late acquisition times in the circular buffer.

[0226] In the actual execution process, according to the acquisition times of the data in the circular buffer, the data in the circular buffer is divided into an old task data set and a new task data set. First, the target price detection model is trained through the old task data set, and based on the training results of the old task data set and the new task data set, the target detection model is trained, so that the target detection model can protect the key information in the old task data during the training process based on the new task data set.

[0227] According to the detection method of the indicator light provided by the embodiment of the present application, by dividing the data in the circular buffer into an old task data set and a new task data set, and training the target detection model through the old and new task data sets, the target detection model can save the key information in the old task data, protect historical knowledge, improve the detection accuracy of the target detection model and expand the usage scenarios.

[0228] In some embodiments, training the target detection model based on the old task data set and the new task data set may further include: Train the target detection model based on the old task data set to obtain old data model parameters; Calculate the information amount of the old data model parameters on the old task data set; Train an object detection model based on the old data model parameters, the amount of information, and the new task dataset.

[0229] In this embodiment, the old data model parameters are the model parameters obtained by training an object detection model with the old task dataset.

[0230] The amount of information is the information reflecting the sensitivity of the old data model parameters to the prediction results of the old task.

[0231] The amount of information can be obtained based on the following steps: Data sampling: Randomly sample a batch of data from the old task dataset.

[0232] Gradient calculation: Calculate the gradient of the loss function with respect to the parameters for each sample of.

[0233] Square and average: Take the square of the gradient and average it over all samples.

[0234] The calculation formula is as follows:

[0235] Among them, is the amount of information; is the amount of data; is the feature vector of the nth old data; is the label of the nth old data; is the old data model parameter; is the loss function.

[0236] In the actual execution process, after obtaining the old data model parameters and the amount of information, the object detection model can be further trained based on the old data model parameters, the amount of information, and the new task dataset.

[0237] In the actual execution process, the object detection model can be trained by means of the continual learning algorithm of Elastic Weight Consolidation (EWC).

[0238] The continual learning algorithm of Elastic Weight Consolidation can prevent the neural network from forgetting the knowledge of the old task when learning a new task.

[0239] The core idea of the continual learning algorithm of Elastic Weight Consolidation is to protect the key information of historical tasks by restricting the update of important parameters.

[0240] The total EWC loss function can be determined based on the following formula:

[0241] Among them, is the loss function for the new task (such as cross-entropy and mean squared error, etc.); is the regularization strength coefficient, controlling the strength of the old task constraint (can take values from 100 to 1000); is the information quantity, measuring the importance to the old task; are the old task model parameters, are the parameter values (historical optimal values) after the old task is trained.

[0242] It should be noted that for the parameter the gradient fluctuates greatly in the old task, indicating that it is sensitive to the task, and the value is relatively high. If the gradient is close to zero, it means that the parameter has little impact on the task, and the value is relatively low.

[0243] The following is an explanation of the training steps of the object detection model.

[0244] 1. Train the old task Normally train the model until convergence, and save the parameters .

[0245] Output: Model parameters , and the accuracy of Task A.

[0246] 2. Calculate the Fisher information matrix Calculate the for each parameter on the dataset of Task A.

[0247] Store: Save and (usually only store the diagonal elements).

[0248] 3. Train the new task Define the total loss:

[0249] Backpropagation: Optimize , and update the parameters .

[0250] 4. Effect verification Check whether the accuracy of the old task remains stable (the detection accuracy of the old task remains above 95%), and whether the accuracy of the new task reaches the expectation (the detection accuracy of the new task is improved to 80%). Through EWC, the multi-modal LED detection system can continuously learn new tasks (such as green LED detection) without forgetting historical tasks (such as red LED detection), achieving true lifelong learning (Lifelong Learning).

[0251] According to the detection method of the indicator light provided by the embodiments of the present application, preliminary training is performed through an old task dataset to obtain old task model parameters, and the information amount corresponding to the old task model parameters is calculated. Based on the old task model parameters, the information amount, and the new task dataset, catastrophic forgetting is alleviated, and the object detection model is dynamically optimized through an online learning mechanism, so that the trained object detection model can adapt to various environments.

[0252] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner.

[0253] For the detection method of the indicator light provided by the embodiments of the present application, the execution subject can be a computer program product. In the embodiments of the present application, taking the computer program product executing the detection method of the indicator light as an example, the computer program product provided by the embodiments of the present application is described.

[0254] As Figure 5 shown, the computer program product includes: a first processing module 510, a second processing module 520, a third processing module 530, and a fourth processing module 540.

[0255] The first processing module 510 is configured to obtain at least one frame of image data, electrical data, and coding data corresponding to the target indicator light; The second processing module 520 is configured to calculate the information entropy corresponding to at least one frame of image data, the deviation degree corresponding to the electrical data, and the conflict information corresponding to the coding data respectively based on the at least one frame of image data, the electrical data, and the coding data; The third processing module 530 is configured to determine a detection score corresponding to the target indicator light based on the information entropy, the deviation degree, and the conflict information; The fourth processing module 540 is configured to determine a detection result corresponding to the target indicator light based on the detection score.

[0256] According to the computer program product provided by the embodiments of the present application, by obtaining one or more frames of image data, electrical data, and coding data of the target indicator light, and respectively processing the image data, the electrical data, and the coding data to obtain the information entropy, the deviation degree, and the conflict information, and jointly detecting the target indicator light through multiple types of data, a detection score is calculated, thereby improving the accuracy of the detection result.

[0257] In some embodiments, the second processing module 520 may further be configured to: Determine the color probability distribution and the lighting state corresponding to the target indicator light based on at least one frame of image data; Calculate the information entropy corresponding to at least one frame of image data based on the color probability distribution; Calculate the deviation corresponding to the electrical data based on the electrical data and the preset standard electrical data; Determine the conflict information corresponding to the coding data based on at least one frame of image data and the coding data.

[0258] In some embodiments, the second processing module 520 can also be used for: Input at least one frame of image data into the target recognition model to obtain the color probability distribution output by the target recognition model; Determine the lighting state based on the brightness change of consecutive frames of images of the target quantity in at least one frame of image data.

[0259] In some embodiments, the second processing module 520 can also be used for: Perform visual recognition on at least one frame of image to obtain a recognition result; the recognition result is the lighting result corresponding to the target indicator light; When the recognition result is consistent with the indicator light state corresponding to the coding data, determine that the conflict information is non-conflicting; When the recognition result is inconsistent with the indicator light state corresponding to the coding data, determine that the conflict information is conflicting.

[0260] In some embodiments, the third processing module 530 can also be used for: Calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on the information entropy, the deviation, and the conflict information; Determine the detection score corresponding to the target indicator light by weighting the information entropy with the first weight, the deviation with the second weight, and the conflict information with the third weight.

[0261] In some embodiments, the third processing module 530 can also be used for: When the change degrees of at least one frame of image data, electrical data, and coding data within the target time period are all less than the first change threshold, calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on the information entropy, the deviation, and the conflict information.

[0262] In some embodiments, the third processing module 530 can also be used for: When at least one of the change degrees of at least one frame of image data, electrical data, and coding data within the target time period is not less than the first change threshold, match at least one frame of image data, electrical data, and coding data with the preset standard mode information; Calculate the first weight corresponding to the visual data, the second weight corresponding to the electrical data, and the third weight corresponding to the coding data respectively based on the information entropy, the deviation, and the conflict information; When at least one frame of image data, electrical data, and coding data match the preset standard mode information, replace the first weight, the second weight, and the third weight with preset weights.

[0263] In some embodiments, the third processing module 530 may further be configured to: When at least one of the degrees of change of at least one frame of image data, electrical data, and coding data within the target time period is not less than a first change threshold, match the at least one frame of image data, electrical data, and coding data with the preset standard mode information; When the target data among at least one frame of image data, electrical data, and coding data does not match the preset standard mode information, obtain the abnormal information confirmed by the user; Based on the abnormal information, adjust the first weight, the second weight, and the third weight to obtain a new first weight, a new second weight, and a new third weight.

[0264] In some embodiments, the third processing module 530 may further be configured to: Adjust the dynamic decay factor, the visual base weight corresponding to the first weight, the electrical base weight corresponding to the second weight, and the flag bit base weight corresponding to the third weight respectively; Based on the adjusted dynamic decay factor, the adjusted visual base weight, the adjusted electrical base weight, and the adjusted flag bit base weight, determine a new first weight, a new second weight, and a new third weight.

[0265] In some embodiments, the computer program product may further include a fourth processing module, configured to Store the abnormal information into a circular buffer; Based on the data in the circular buffer, train a target detection model, where the target detection model is used to detect a target indicator light.

[0266] In some embodiments, the computer program product may further include a fifth processing module, configured to: When the amount of data in the circular buffer is greater than the target amount of data, obtain an old task data set and a new task data set based on the acquisition times of the respective data in the circular buffer; Based on the old task data set and the new task data set, train the target detection model.

[0267] In some embodiments, the computer program product may further include a sixth processing module, configured to: Based on the old task data set, train the target detection model to obtain old data model parameters; Calculate the amount of information of the old data model parameters on the old task data set; Train an object detection model based on the old data model parameters, the amount of information, and the new task dataset.

[0268] In the method for detecting an indicator light provided by an embodiment of the present application, the execution subject may be a detection device for the indicator light. In the embodiment of the present application, taking the detection device for the indicator light to execute the method for detecting the indicator light as an example, the detection device for the indicator light provided by the embodiment of the present application is described.

[0269] For the description of the features in the corresponding embodiment of the detection device for the indicator light, reference may be made to the relevant description in the corresponding embodiment of the method for detecting the indicator light, which will not be elaborated here one by one.

[0270] An embodiment of the present application further provides an indicator light detection system.

[0271] As Figure 2 shown, the indicator light detection system includes: a data acquisition layer, a core processing layer, and a feedback loop.

[0272] In this embodiment, the industrial camera, current probe, and status register in the data acquisition layer respectively provide visual, electrical, and register coding data.

[0273] The core processing layer can calculate weights by a dynamic fusion engine and output a decision, and an online learning optimizer can update the model parameters in real time.

[0274] The feedback loop can optimize the object detection model based on the manual recheck result and historical data.

[0275] Among them, the visual recognition module can be used for the LED color and status (constant on / flashing).

[0276] The electrical analysis module is used to monitor parameters such as voltage and current, analyze electrical parameter abnormalities, and output a deviation score.

[0277] The flag bit parsing module is used to read the LED register coding in the hardware, verify the consistency between the register coding and the real-time data, and mark conflict events.

[0278] The dynamic fusion engine is used to synthesize multi-modal data, dynamically calculate weights, and output a comprehensive score.

[0279] The online learning optimizer is used to incrementally update the model parameters based on manual feedback and historical data.

[0280] The alarm module is used to trigger an audible and visual alarm when the comprehensive score < 0.8.

[0281] The detection report generation is used to record the detection result (pass / fail) and detailed parameters.

[0282] The manual recheck result is used for the corrected label confirmed manually and is used to optimize the model.

[0283] The device log is used to store historical detection data and support long-term learning of the model.

[0284] According to the detection system of the indicator light provided by the embodiment of the present application, by acquiring one or more frames of image data, electrical data, and coding data of the target indicator light, and respectively processing the image data, electrical data, and coding data to obtain information entropy, deviation degree, and conflict information, and jointly detecting the target indicator light through various data, a detection score is calculated to improve the accuracy of the detection result.

[0285] The embodiment of the present application also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above embodiments of the detection method of the indicator light.

[0286] The embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above embodiments of the detection method of the indicator light when running.

[0287] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk, or optical disc, and other various media that can store computer programs.

[0288] The embodiment of the present application also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the detection method of the indicator light.

[0289] The embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the detection method of the indicator light.

[0290] Those skilled in the art may further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered as exceeding the scope of this application.

[0291] The above has introduced in detail a method for detecting an indicator light provided by this application. Specific examples have been used herein to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for detecting an indicator light, characterized in that: include: Acquire at least one frame of image data, electrical data, and coded data corresponding to the target indicator light; Based on the at least one frame of image data, the electrical data and the coded data, respectively calculating information entropy corresponding to the at least one frame of image data, a degree of deviation corresponding to the electrical data and conflict information corresponding to the coded data; Determining a detection score corresponding to the target indicator light based on the information entropy, the deviation degree, and the conflict information; Based on the detection score, a detection result corresponding to the target indicator light is determined.

2. The method for detecting an indicator light according to claim 1, characterized in that: The calculating, based on the at least one frame of image data, the electrical data and the coded data, respectively, the information entropy corresponding to the at least one frame of image data, the deviation corresponding to the electrical data and the conflict information corresponding to the coded data, comprises: Based on the at least one frame of image data, determining a color probability distribution and a lighting state corresponding to the target indicator light; Based on the color probability distribution, calculating the information entropy corresponding to the at least one frame of image data; Based on the electrical data and preset standard electrical data, calculating the deviation corresponding to the electrical data; Based on the at least one frame of image data and the coded data, conflict information corresponding to the coded data is determined.

3. The method for detecting an indicator light according to claim 2, characterized in that: The determining, based on the at least one frame of image data, a color probability distribution and a lighting state corresponding to the target indicator light includes: Inputting the at least one frame of image data into a target recognition model to obtain a color probability distribution output by the target recognition model; The lighting state is determined based on brightness changes of a target number of consecutive frame images in the at least one frame of image data.

4. The method for detecting an indicator light according to claim 2, characterized in that: The determining, based on the at least one frame of image data and the coded data, conflict information corresponding to the coded data includes: Performing visual recognition on the at least one frame of image to obtain a recognition result; the recognition result is a lighting result corresponding to the target indicator light; In the case where the recognition result is consistent with the indicator light state corresponding to the coded data, determining that the conflict information is non-conflicting; In the case where the recognition result is inconsistent with the indicator light state corresponding to the coded data, the conflict information is determined to be a conflict.

5. The method for detecting an indicator light according to any one of claims 1 to 4, characterized in that: The determining, based on the information entropy, the deviation degree, and the conflict information, a detection score corresponding to the target indicator light includes: Calculating a first weight corresponding to the visual data, a second weight corresponding to the electrical data, and a third weight corresponding to the coded data based on the information entropy, the deviation, and the conflict information, respectively; The detection score corresponding to the target indicator light is determined by weighting the information entropy by the first weight, weighting the deviation by the second weight, and weighting the conflict information by the third weight.

6. The method for detecting an indicator light according to claim 5, characterized in that: The calculating, based on the information entropy, the deviation and the conflict information, a first weight corresponding to the visual data, a second weight corresponding to the electrical data and a third weight corresponding to the coded data, respectively, comprises: When the degree of change of at least one frame of image data, the electrical data and the coded data within the target time period is less than the first change threshold, the first weight corresponding to the visual data, the second weight corresponding to the electrical data and the third weight corresponding to the coded data are calculated based on the information entropy, the deviation and the conflict information respectively.

7. The method for detecting an indicator light according to claim 5, characterized in that: The calculating, based on the information entropy, the deviation and the conflict information, a first weight corresponding to the visual data, a second weight corresponding to the electrical data and a third weight corresponding to the coded data, respectively, comprises: When at least one of the degrees of change of the at least one frame of image data, the electrical data, and the coded data within the target time period has a degree of change that is not less than a first change threshold, matching the at least one frame of image data, the electrical data, and the coded data with preset standard mode information; Calculating a first weight corresponding to the visual data, a second weight corresponding to the electrical data, and a third weight corresponding to the coded data based on the information entropy, the deviation, and the conflict information respectively; In a case where the at least one frame of image data, the electrical data, and the encoding data match the preset standard mode information, the first weight, the second weight, and the third weight are replaced by preset weights.

8. The method for detecting an indicator light according to claim 5, characterized in that: The calculating, based on the information entropy, the deviation and the conflict information, a first weight corresponding to the visual data, a second weight corresponding to the electrical data and a third weight corresponding to the coded data, respectively, comprises: When at least one of the change degrees of the at least one frame of image data, the electrical data, and the coded data within the target time period is not less than a first change threshold, matching the at least one frame of image data, the electrical data, and the coded data with preset standard mode information; When the at least one frame of image data, the electrical data, and the target data in the coded data do not match the preset standard mode information, obtaining abnormal information confirmed by a user; Based on the abnormal information, the first weight, the second weight and the third weight are adjusted to obtain a new first weight, a new second weight and a new third weight.

9. The method for detecting an indicator light according to claim 8, characterized in that: The adjusting the first weight, the second weight, and the third weight based on the abnormal information to obtain a new first weight, a new second weight, and a new third weight includes: respectively adjusting the dynamic attenuation factor, the visual basic weight corresponding to the first weight, the electrical basic weight corresponding to the second weight, and the flag basic weight corresponding to the third weight; Based on the adjusted dynamic attenuation factor, the adjusted visual basic weight, the adjusted electrical basic weight, and the adjusted flag basic weight, a new first weight, a new second weight, and a new third weight are determined.

10. The method for detecting an indicator light according to claim 8, characterized in that: After obtaining the abnormal information confirmed by the user, the method further includes: Storing the abnormal information in a circular buffer; Based on the data in the circular buffer, a target detection model is trained, and the target detection model is used to detect the target indicator light.

11. The method for detecting an indicator light according to claim 10, characterized in that: The training of the target detection model based on the data in the circular buffer comprises: When the amount of data in the circular buffer is greater than the target amount of data, acquiring an old task data set and a new task data set based on the acquisition time of each data in the circular buffer; The target detection model is trained based on the old task dataset and the new task dataset.

12. The method for detecting an indicator light according to claim 11, characterized in that: The training of the target detection model based on the old task dataset and the new task dataset includes: Based on the old task data set, training the target detection model to obtain old data model parameters; Calculating the information amount of the old data model parameters on the old task data set; The target detection model is trained based on the old data model parameters, the amount of information and the new task data set.

13. A computer program product, characterized in that include: A first processing module, used for acquiring at least one frame of image data, electrical data and coded data corresponding to the target indicator light; A second processing module, configured to calculate, based on the at least one frame of image data, the electrical data and the coded data, information entropy corresponding to the at least one frame of image data, a degree of deviation corresponding to the electrical data and conflict information corresponding to the coded data; A third processing module, configured to determine a detection score corresponding to the target indicator light based on the information entropy, the deviation degree, and the conflict information; The fourth processing module is used to determine the detection result corresponding to the target indicator light based on the detection score.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for detecting an indicator light as claimed in any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for detecting an indicator light according to any one of claims 1 to 12.

Images