State indication lamp monitoring method and device, electronic equipment and storage medium
By using reference template images and grayscale image processing techniques, combined with color thresholding and sliding window methods, the problems of misjudgment and low positioning accuracy in status indicator monitoring have been solved, enabling precise positioning and accurate monitoring in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HONGHU WANLIAN (JIANGSU) TECH DEV CO LTD
- Filing Date
- 2026-04-08
- Publication Date
- 2026-07-03
Smart Images
Figure CN122336634A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of production management and image recognition technology, and in particular to a method, device, electronic device and storage medium for monitoring status indicator lights. Background Technology
[0002] With the continuous development of automation technology, video monitoring has been applied to various industries, especially in the field of industrial production. Since status indicator lights directly reflect the operating status of equipment, the monitoring results of status indicator lights play an important role in production management.
[0003] In existing technologies, the status monitoring of status indicator lights typically involves first performing threshold segmentation on the monitoring image in the RGB (Red Green Blue) color space after acquiring the monitoring image via a camera, thereby generating a binary mask. Then, the position of the status indicator light is determined by finding the outline of the status indicator light in the binary mask. Finally, the color recognition result at that position is obtained based on the position of the status indicator light, and the illuminated color of the status indicator light reflects the operating status of the device.
[0004] However, this method of monitoring status indicator lights is not only easily affected by environmental factors, often leading to misjudgments and omissions in the identification results, but also has low positioning accuracy, making it difficult to quickly and accurately locate the status indicator lights in complex backgrounds. Summary of the Invention
[0005] This invention provides a method, device, electronic device, and storage medium for monitoring status indicator lights, in order to solve the problems of high false alarm rate and low position positioning accuracy of status indicator lights.
[0006] According to another aspect of the present invention, a method for monitoring a status indicator light is provided, comprising: In response to acquiring a video frame image of the status indicator light, the positioning area image of the status indicator light in the video frame image is obtained through the reference position information of the reference template image; Obtain the grayscale image corresponding to the location area image, and determine whether the location area image is overexposed based on the proportion of bright pixels in the grayscale image; If it is determined that the image of the positioning area does not have overexposure, the image of the positioning area is classified into pixels according to the reference color threshold, so as to obtain the lighting status of the status indicator light based on the pixel classification result.
[0007] After obtaining the positioning region image of the status indicator light in the video frame image through the reference position information of the reference template image, the method further includes: determining whether the status indicator light exists in the positioning region image; if it is determined that the status indicator light does not exist, sliding the reference template image in the video frame image to output a similarity matrix between the reference template image and the video frame image; if it is determined that the maximum similarity in the similarity matrix is greater than or equal to a preset similarity threshold, taking the local image corresponding to the maximum similarity in the video frame image as the positioning region image of the status indicator light, and updating the reference position information of the reference template image according to the position information of the local image.
[0008] After determining whether the image of the positioning area is overexposed based on the proportion of bright pixels in the grayscale image, the method further includes: if it is determined that the image of the positioning area is overexposed, obtaining the average brightness of each original color channel in the image of the positioning area, and obtaining the lighting status of the status indicator light based on the average brightness of each original color channel.
[0009] After determining that the image of the positioning area has an overexposure phenomenon, the method further includes: obtaining a non-overexposure area in the positioning area image based on the grayscale image corresponding to the positioning area image and a preset grayscale threshold, and obtaining a valid contour area in the non-overexposure area; classifying the valid contour area into pixels based on a reference color threshold, so as to obtain the lighting state of the status indicator light based on the pixel classification result.
[0010] After obtaining the illumination status of the status indicator light based on the pixel classification result, the process includes: obtaining the illumination status of the status indicator light in a preset number of consecutive video frame images through a sliding window, and determining the actual illumination status of the status indicator light based on the illumination status of the status indicator light in the preset number of consecutive video frame images.
[0011] After obtaining the illumination state of the status indicator light based on the pixel classification result, the method includes: obtaining the timestamp information of multiple transition signals of the status indicator light in the actual illumination state; wherein, the transition signal includes a rising edge signal from off to on and a falling edge signal from on to off; and obtaining the flashing period of the status indicator light in the actual illumination state based on the timestamp information of the multiple transition signals, so as to determine the state operation mode based on the flashing period.
[0012] According to another aspect of the present invention, a monitoring device for a status indicator light is provided, comprising: The positioning area acquisition module is used to acquire the positioning area image of the status indicator in the video frame image in response to the acquisition of the video frame image of the status indicator, by using the reference position information of the reference template image; An overexposure judgment module is used to obtain a grayscale image corresponding to the image of the positioning area, and to determine whether the image of the positioning area is overexposed based on the proportion of bright pixels in the grayscale image. The illumination status acquisition module is used to classify the pixels of the positioning area image according to a reference color threshold if it is determined that there is no overexposure phenomenon in the positioning area image, so as to obtain the illumination status of the status indicator light according to the pixel classification result.
[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the status indicator monitoring method described in any embodiment of the present invention.
[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the status indicator monitoring method described in any embodiment of the present invention.
[0015] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the monitoring method for status indicator lights as described in any embodiment of the present invention.
[0016] The technical solution of this invention, in response to acquiring a video frame image of a status indicator light, obtains a positioning area image of the status indicator light in the video frame image using the reference position information of a reference template image; acquires a grayscale image corresponding to the positioning area image, and determines whether there is overexposure in the positioning area image based on the proportion of bright pixels in the grayscale image; if it is determined that there is no overexposure in the positioning area image, performs pixel classification on the positioning area image according to a reference color threshold, and obtains the illumination status of the status indicator light based on the pixel classification result. Therefore, not only is accurate positioning of the status indicator light in complex backgrounds achieved based on the reference template image, but also potential overexposure is eliminated through the proportion of bright pixels, avoiding interference from environmental factors and improving the accuracy of the illumination status monitoring results.
[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart of a method for monitoring a status indicator light according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of another method for monitoring status indicator lights according to Embodiment 2 of the present invention; Figure 3 This is a flowchart of another method for monitoring a status indicator light according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of a status indicator monitoring device according to Embodiment 4 of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device that implements the status indicator monitoring method of the present invention. Detailed Implementation
[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0022] Example 1 Figure 1 This is a flowchart of a status indicator monitoring method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where the status indicator is located using a reference template image, overexposure is eliminated based on the proportion of bright pixels, and pixel classification is performed based on a reference color threshold. This method can be executed by a status indicator monitoring device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S101. In response to obtaining the video frame image of the status indicator light, obtain the positioning area image of the status indicator light in the video frame image through the reference position information of the reference template image.
[0023] After the industrial camera acquires the monitoring video of the status indicator lights, if the camera is a local camera, the application programming interface (API) of the camera can be directly called using the open-source computer vision library (OpenCV) to obtain the video information; if the industrial camera is a network camera, the video stream can be read based on the Real Time Streaming Protocol (RTSP); then, each frame of the video can be exported as a video frame image, such as a JPEG (Joint Photographic Experts Group) format video frame image, using the ffmepeg command-line tool.
[0024] In the initial monitoring phase, for the acquired initial video frame images, the Region of Interest (ROI) in the video frame image can be identified through a machine learning model, or the complete area of the status indicator light can be selected by manually annotating it with a rectangle, that is, covering all possible luminous ranges of the status indicator light, and the position information of the selected area is recorded. Among them, the image information of the ROI or the selected area is stored as the reference template image, and the position information of the ROI or the selected area is stored as the reference position information.
[0025] Specifically, the reference position information can include the x-coordinate of the top left corner, the y-coordinate of the top left corner, the width and height of the rectangular area, or the x-coordinates of the four vertices, namely the x-coordinates of the top left corner, the bottom left corner, the top right corner, and the bottom right corner. Based on this, each time a video frame image is acquired, the positioning area of the status indicator can be directly determined in the current video frame image according to the reference position information of the reference template image. Compared with determining the location of the status indicator in the entire video frame image, the acquisition of the positioning area image avoids the computational redundancy and noise interference caused by full-image scanning, and improves the positioning efficiency and positioning accuracy of the status indicator.
[0026] Optionally, in this embodiment of the invention, after obtaining the positioning region image of the status indicator in the video frame image through the reference position information of the reference template image, the method further includes: determining whether the status indicator exists in the positioning region image; if it is determined that the status indicator does not exist, sliding the reference template image in the video frame image to output a similarity matrix between the reference template image and the video frame image; if it is determined that the maximum similarity in the similarity matrix is greater than or equal to a preset similarity threshold, using the local image corresponding to the maximum similarity in the video frame image as the positioning region image of the status indicator, and updating the reference position information of the reference template image according to the position information of the local image.
[0027] Specifically, when the camera or status indicator light shifts position, the positioning area of the status indicator light will also deviate. At this time, the status indicator light cannot be identified through the positioning area image. In this case, the reference template image is slid in the video frame image. For example, the sliding step size is set to one pixel. By sliding horizontally first and then vertically or vertically first and then horizontally, each local image in the video frame image is traversed, and the similarity between the reference template image and each local image is recorded, thus forming a similarity matrix.
[0028] If the maximum similarity in the similarity matrix is greater than or equal to a preset similarity threshold (e.g., 80%), it indicates that the reference template image and the local image in the video frame image have successfully matched. The local image corresponding to the maximum similarity is used as the positioning area image of the status indicator. In this way, through the local image matching mechanism, the positioning area of the status indicator can still be accurately obtained even when there is a slight positional shift in the camera or status indicator, thus improving the robustness of the positioning area acquisition process.
[0029] Meanwhile, since the camera or status indicator light has shifted position, it is also necessary to update the reference position information of the reference template image based on the position information of the local image to ensure timely updates of the reference position information. In particular, if the maximum similarity in the similarity matrix is less than the preset similarity threshold, it indicates that the reference template image and the local image in the video frame image have failed to match successfully. That is, the camera or status indicator light has shifted position significantly. Based on this, an alarm is issued to guide maintenance personnel to adjust the placement of the camera or status indicator light.
[0030] S102. Obtain the grayscale image corresponding to the positioning area image, and determine whether the positioning area image is overexposed based on the percentage of bright pixels in the grayscale image.
[0031] The image of the location area is converted into a grayscale image (e.g., an 8-bit grayscale image), and the brightness of each pixel is quantized to an integer between 0 and 255; where 0 represents the darkest, that is, pure black; and the value 255 represents the brightest, that is, pure white. When the grayscale value of a pixel is greater than or equal to a preset grayscale threshold (e.g., 240), it indicates that the brightness of the pixel is relatively high, and it can be regarded as a bright pixel. Therefore, a bright pixel is a pixel whose grayscale value is greater than or equal to the preset grayscale threshold. At the same time, the ratio of the number of bright pixels in the location area image to the total number of pixels is used as the proportion of bright pixels.
[0032] If the proportion of bright pixels is greater than or equal to a preset proportion threshold (e.g., 50%), it can be determined that the image of the location area is overexposed. If the proportion of bright pixels is less than the preset proportion threshold, it can be determined that the image of the location area is not overexposed. Based on this, by using dual judgment conditions, namely comparing the pixel grayscale value with the preset grayscale threshold and comparing the proportion of bright pixels with the preset proportion threshold, the scene difference between small-area reflection and large-area exposure is effectively distinguished, thus improving the reliability of the overexposure judgment result.
[0033] S103. If it is determined that the image of the positioning area does not have overexposure, the image of the positioning area is classified into pixels according to the reference color threshold, so as to obtain the lighting status of the status indicator light according to the pixel classification result.
[0034] The baseline color threshold decomposes each color from the perspectives of Hue, Saturation, and Value. Compared to traditional color classification methods, this is closer to human color perception and reduces interference from changes in lighting. Hue refers to the basic attribute of color and can be measured from 0 to 180 degrees. Saturation refers to the purity of color, with a value range of 0 to 255, where 0 represents the most vivid color and 255 represents no color. Value refers to the lightness or darkness of color, with a value range of 0 to 255, where 255 represents the brightest and 0 represents the darkest.
[0035] In the three evaluation channels of hue, saturation, and brightness, upper and lower thresholds are set to filter pixel areas that meet the conditions, so as to extract status indicator lights of specific colors from the location area image. In particular, red, as the start and end point of the color wheel, exhibits a looping phenomenon. That is, red is distributed in two ranges in the hue: 0 to 10 (i.e., pure red) and 170 to 180 (i.e., dark red). Taking the status indicator light color involving red, yellow, and green as an example, the baseline color thresholds can be shown in Table 1: Table 1. Numerical range of reference color thresholds Based on a reference color threshold, each pixel in the localized area image is classified. For example, if the hue, saturation, and brightness of pixel A all fall within the range of hue, saturation, and brightness of red, then pixel A is identified as red and assigned a value of 255. If the hue, saturation, and brightness of pixel B do not fall within the range of hue, saturation, and brightness of any color, then pixel B is identified as colorless and assigned a value of 0. This achieves binarization output based on the reference color threshold.
[0036] In each color classification result, the number of pixels with a pixel value of 255 is counted, and the ratio of this number to the total number of pixels in the location area image is obtained as the color proportion of that color. The maximum color proportion among all color proportions is obtained. If the maximum color proportion is greater than or equal to a preset proportion threshold (e.g., 5%), then that color can be determined as the current color of the status indicator. The color of the status indicator reflects the illumination status of the status indicator, i.e., whether the status indicator is lit and what color it is lit. The illumination status of the status indicator reflects the operating status of the corresponding monitored device. If the maximum color proportion is less than the preset proportion threshold, it means that the status indicator does not emit any color at this time, thus avoiding color misjudgment. The high-brightness pixel proportion result is not 0, which may be due to environmental noise (e.g., reflection, camera noise).
[0037] The technical solution of this invention, in response to acquiring a video frame image of a status indicator light, obtains a positioning area image of the status indicator light in the video frame image using the reference position information of a reference template image; acquires a grayscale image corresponding to the positioning area image, and determines whether there is overexposure in the positioning area image based on the proportion of bright pixels in the grayscale image; if it is determined that there is no overexposure in the positioning area image, performs pixel classification on the positioning area image according to a reference color threshold, and obtains the illumination status of the status indicator light based on the pixel classification result. Therefore, not only is accurate positioning of the status indicator light in complex backgrounds achieved based on the reference template image, but also potential overexposure is eliminated through the proportion of bright pixels, avoiding interference from environmental factors and improving the accuracy of the illumination status monitoring results.
[0038] Example 2 Figure 2 This is a flowchart of a status indicator monitoring method provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is that the positioning area image exhibits overexposure, such as... Figure 2 As shown, the method specifically includes: S201. In response to obtaining a video frame image of the status indicator light, obtain the positioning area image of the status indicator light in the video frame image through the reference position information of the reference template image.
[0039] S202. Obtain the grayscale image corresponding to the positioning area image, and determine whether the positioning area image is overexposed based on the proportion of bright pixels in the grayscale image.
[0040] S203. If it is determined that the positioning area image has an overexposure phenomenon, obtain the average brightness of each original color channel in the positioning area image, and obtain the lighting status of the status indicator light based on the average brightness of each original color channel.
[0041] When an image is overexposed, the overexposed area will approach pure white, meaning that the values of the three original color channels R (Red), G (Green), and B (Blue) will all approach 255. However, the channel characteristics of the actual colors may still be preserved, meaning that the values of the actual color channels may still be slightly higher than the other channels. For example, after a red light is overexposed, the value of the red (R) channel may be slightly higher than the green (G) channel and the blue (B) channel (e.g., R=255, G=250, B=245).
[0042] At this point, in the image of the positioning area, the average brightness of the three color channels R, G, and B is obtained, and the average brightness of each color channel is compared. If the R channel is greater than the G channel and the B channel (the difference between R and G, and between R and B, is greater than or equal to the first difference threshold), the status indicator light is lit in red. If the G channel is greater than the R channel and the G channel is greater than the B channel (the difference between G and R, and between G and B, is greater than or equal to the first difference threshold), the status indicator light is lit in green. If the B channel is greater than the R channel and the B channel is greater than the G channel (the difference between B and R, and between B and G, is greater than or equal to the first difference threshold), the status indicator light is lit in blue.
[0043] In addition, for other colors besides red, green, and blue, the average brightness of the original color channels can also be obtained. For example, if the R channel is greater than the B channel, and the G channel is greater than the B channel, and the R and G channel values are similar (the difference between R and B and between G and B is greater than or equal to the first difference threshold, and the difference between R and G is less than the first difference threshold), then the color of the status indicator light is yellow.
[0044] If the G channel is greater than the R channel, and the B channel is greater than the R channel, and the values of the G channel and the B channel are similar (the differences between G and R and between B and R are both greater than or equal to the first difference threshold, and the difference between G and B is less than the first difference threshold), then the status indicator light will be illuminated in cyan. If the R channel is greater than the G channel, and the B channel is greater than the G channel, and the values of the R channel and the B channel are similar (the differences between R and G and between B and G are both greater than or equal to the first difference threshold, and the difference between R and B is less than the first difference threshold), then the status indicator light will be illuminated in purple or magenta.
[0045] Optionally, in this embodiment of the invention, after determining that the positioning area image has an overexposure phenomenon, the method further includes: obtaining a non-overexposure area in the positioning area image based on the grayscale image corresponding to the positioning area image and a preset grayscale threshold, and obtaining an effective contour area in the non-overexposure area; classifying the effective contour area into pixels based on a reference color threshold, so as to obtain the lighting state of the status indicator light based on the pixel classification result.
[0046] Specifically, if the image of the location area is overexposed, the bright parts of the image, that is, the areas where the pixel gray value is greater than the preset gray value threshold, may lose details due to strong light reflection and need to be excluded. Only the non-overexposed areas with pixel gray values less than or equal to the preset gray value threshold are retained. Then, the boundaries of the connected regions are extracted by contour detection algorithms, such as contour algorithms based on edge detection, contour algorithms based on region segmentation, and contour tracking algorithms. The fine contours caused by the reflection of the metal shell are removed by morphological operations (e.g., erosion) and area filtering.
[0047] Therefore, after eliminating the influence of edge reflections on the outline of the status indicator, the effective light-emitting area of the status indicator, that is, the effective outline area of the status indicator, can be accurately located. Finally, the effective outline area is classified into pixels according to the reference color threshold, and the illumination color of the status indicator is obtained according to the pixel classification result. This not only avoids the interference of overexposed areas on the illumination color detection result, but also accurately locates the effective outline area based on the outline detection algorithm, thus improving the efficiency of obtaining the illumination color detection result.
[0048] S204. If it is determined that the image of the positioning area does not have overexposure, the image of the positioning area is classified into pixels according to the reference color threshold, so as to obtain the lighting status of the status indicator light according to the pixel classification result.
[0049] The technical solution of this invention, after determining whether there is overexposure in the positioning area image based on the proportion of bright pixels in the grayscale image, if it is determined that there is overexposure, obtains the average brightness of each original color channel in the positioning area image, and obtains the illumination state of the status indicator light based on the average brightness of each original color channel. Therefore, when there is overexposure in the positioning area image, by obtaining the average brightness of each original color channel in the positioning area image, the illumination color of the status indicator light is obtained, ensuring the accurate acquisition of the illumination color of the status indicator light when the image is overexposed, and expanding the number of obtainable illumination colors for the status indicator light.
[0050] Example 3 Figure 3 This is a flowchart of a method for monitoring a status indicator light according to Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that the status operation mode of the status indicator light is determined based on multiple video frame images, such as... Figure 3 As shown, the method specifically includes: S301. In response to obtaining the video frame image of the status indicator light, obtain the positioning area image of the status indicator light in the video frame image through the reference position information of the reference template image.
[0051] S302. Obtain the grayscale image corresponding to the positioning area image, and determine whether the positioning area image is overexposed based on the percentage of bright pixels in the grayscale image.
[0052] S303. If it is determined that the image of the positioning area does not have overexposure, the image of the positioning area is classified into pixels according to the reference color threshold, so as to obtain the lighting status of the status indicator light according to the pixel classification result.
[0053] S304. Obtain the illumination status of the status indicator light in a preset number of consecutive video frame images through a sliding window, and determine the actual illumination status of the status indicator light based on the illumination status of the status indicator light in the preset number of consecutive video frame images.
[0054] A fixed-length (e.g., 5 frames) sliding window can be maintained, and the color that appears most frequently within the sliding window is used as the output result. This allows for time-dimensional filtering of the color recognition results from consecutive video frames, avoiding misjudgments in a single frame. The size of the sliding window can be adaptively adjusted according to the video frame rate. For example, if the video frame rate is 30fps, 5 frames correspond to approximately 0.17 seconds; if the frame rate is 10fps, it can be adjusted to 7 frames, thus balancing real-time performance and filtering effect.
[0055] S305. Obtain the timestamp information of multiple transition signals of the status indicator light in the actual lit state; wherein, the transition signal includes a rising edge signal from off to on, and a falling edge signal from on to off.
[0056] The actual illumination state reflects the actual illuminated color of the status indicator. The actual illuminated color is taken as the target color, and then the color state is converted into a binary signal. For example, the "target color" is 1, and others are 0 (for example, when a red light is detected to be flashing, the red light is 1, and other lights are 0). In reality, the status indicator is considered to be lit only when the target color is present, that is, the value 1 is considered as lit. When there is no color or other colors (considered as misjudgments), the status indicator is considered to be off, that is, the value 0 is considered as off. Based on this, the rising edge signal (i.e., from 0 to 1) and the falling edge signal (from 1 to 0) are identified, and the timestamp of each transition is recorded.
[0057] S306. Based on the timestamp information of the multiple transition signals, obtain the flashing period of the status indicator light in the actual lit state, so as to determine the state operation mode based on the flashing period.
[0058] Based on the time difference between two adjacent transitions of the same type, for example, if the timestamp of the first acquisition of the rising edge signal is A1 and the timestamp of the second acquisition of the rising edge signal is A2, then the transition interval is A2-A1; if the timestamp of the first acquisition of the falling edge signal is B1 and the timestamp of the second acquisition of the falling edge signal is B2, then the transition interval is B2-B1; the time interval of multiple consecutive transitions of the same type is acquired, and the average time interval is calculated as the flashing period. Then, based on the flashing period and the actual illuminated color, the operating mode of the monitored device corresponding to the status indicator can be determined (for example, when the red light is on, if the flashing frequency is A, the monitored device is in a fault state; if the flashing frequency is B, the monitored device is in a standby state).
[0059] The technical solution of this invention obtains the illumination status of the status indicator light in a preset number of consecutive video frames through a sliding window, and determines the actual illumination status of the status indicator light based on the illumination status of the status indicator light in the preset number of consecutive video frames. This avoids single-frame misjudgment and balances the real-time monitoring and filtering effect of the status indicator light illumination status. At the same time, it obtains the timestamp information of multiple transition signals of the status indicator light in the actual illumination status, and obtains the flashing period of the status indicator light in the actual illumination status based on the timestamp information of multiple transition signals. This enables further monitoring of the operating mode of the monitored device and ensures the complete acquisition of multiple operating modes.
[0060] Example 4 Figure 4 This is a structural block diagram of a status indicator monitoring device provided in Embodiment 4 of the present invention. The device specifically includes: The positioning area acquisition module 401 is used to acquire the positioning area image of the status indicator in the video frame image in response to the acquisition of the video frame image of the status indicator, by using the reference position information of the reference template image; The overexposure judgment module 402 is used to obtain the grayscale image corresponding to the positioning area image and judge whether the positioning area image has an overexposure phenomenon based on the proportion of bright pixels in the grayscale image. The illumination status acquisition module 403 is used to classify the pixels of the positioning area image according to a reference color threshold if it is determined that there is no overexposure phenomenon in the positioning area image, so as to obtain the illumination status of the status indicator light according to the pixel classification result.
[0061] The technical solution of this invention, in response to acquiring a video frame image of a status indicator light, obtains a positioning area image of the status indicator light in the video frame image using the reference position information of a reference template image; acquires a grayscale image corresponding to the positioning area image, and determines whether there is overexposure in the positioning area image based on the proportion of bright pixels in the grayscale image; if it is determined that there is no overexposure in the positioning area image, performs pixel classification on the positioning area image according to a reference color threshold, and obtains the illumination status of the status indicator light based on the pixel classification result. Therefore, not only is accurate positioning of the status indicator light in complex backgrounds achieved based on the reference template image, but also potential overexposure is eliminated through the proportion of bright pixels, avoiding interference from environmental factors and improving the accuracy of the illumination status monitoring results.
[0062] Optionally, the monitoring device for the status indicator light is further configured to determine whether the status indicator light exists in the positioning area image; if it is determined that the status indicator light does not exist, the reference template image is slid across the video frame image to output a similarity matrix between the reference template image and the video frame image; if it is determined that the maximum similarity in the similarity matrix is greater than or equal to a preset similarity threshold, the local image corresponding to the maximum similarity in the video frame image is used as the positioning area image of the status indicator light, and the reference position information of the reference template image is updated according to the position information of the local image.
[0063] Optionally, the monitoring device for the status indicator light is further configured to, if it is determined that the image of the positioning area is overexposed, obtain the average brightness of each original color channel in the image of the positioning area, and obtain the lighting status of the status indicator light based on the average brightness of each original color channel.
[0064] Optionally, the monitoring device for the status indicator light is further configured to obtain the non-overexposed area in the positioning area image based on the grayscale image corresponding to the positioning area image and a preset grayscale threshold, and obtain the effective contour area in the non-overexposed area; classify the effective contour area into pixels based on a reference color threshold, so as to obtain the lighting state of the status indicator light based on the pixel classification result.
[0065] Optionally, the monitoring device for the status indicator light is further configured to acquire the illumination status of the status indicator light in a preset number of consecutive video frame images through a sliding window, and determine the actual illumination status of the status indicator light based on the illumination status of the status indicator light in the preset number of consecutive video frame images.
[0066] Optionally, the monitoring device for the status indicator light is further configured to acquire timestamp information of multiple transition signals of the status indicator light in the actual lit state; wherein, the transition signal includes a rising edge signal from off to on and a falling edge signal from on to off; and based on the timestamp information of the multiple transition signals, the flashing period of the status indicator light in the actual lit state is acquired, so as to determine the state operation mode based on the flashing period.
[0067] The above-described device can execute the status indicator monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the status indicator monitoring method provided in any embodiment of the present invention.
[0068] Example 5 Figure 5A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0069] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0070] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0071] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method of monitoring status indicator lights.
[0072] In some embodiments, the method for monitoring status indicator lights may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the status indicator light monitoring method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the status indicator light monitoring method by any other suitable means (e.g., by means of firmware).
[0073] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0074] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0075] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0076] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).
[0077] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0078] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0079] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0080] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for monitoring a status indicator light, characterized in that, include: In response to acquiring a video frame image of the status indicator light, the positioning area image of the status indicator light in the video frame image is obtained through the reference position information of the reference template image; Obtain the grayscale image corresponding to the location area image, and determine whether the location area image is overexposed based on the proportion of bright pixels in the grayscale image; If it is determined that there is no overexposure in the image of the positioning area, the image of the positioning area is classified into pixels according to the reference color threshold, so as to obtain the lighting status of the status indicator light based on the pixel classification result.
2. The method for monitoring status indicator lights according to claim 1, characterized in that, After obtaining the positioning area image of the status indicator light in the video frame image using the reference position information of the reference template image, the method further includes: Determine whether the status indicator light exists in the image of the positioning area; If it is determined that the status indicator light does not exist, the reference template image is slid across the video frame image to output a similarity matrix between the reference template image and the video frame image; If the maximum similarity in the similarity matrix is determined to be greater than or equal to a preset similarity threshold, the local image corresponding to the maximum similarity in the video frame image is used as the positioning area image of the status indicator light, and the reference position information of the reference template image is updated according to the position information of the local image.
3. The method for monitoring status indicator lights according to claim 1, characterized in that, After determining whether the image of the location region is overexposed based on the proportion of bright pixels in the grayscale image, the method further includes: If it is determined that the image of the positioning area is overexposed, the average brightness of each original color channel in the image of the positioning area is obtained, and the lighting status of the status indicator is obtained based on the average brightness of each original color channel.
4. The method for monitoring status indicator lights according to claim 3, characterized in that, After determining that the image of the location area is overexposed, the method further includes: Based on the grayscale image corresponding to the location area image and a preset grayscale threshold, obtain the non-overexposed area in the location area image, and obtain the effective contour area in the non-overexposed area. The effective contour region is classified into pixels according to a reference color threshold, and the illumination status of the status indicator is obtained based on the pixel classification result.
5. The method for monitoring status indicator lights according to claim 1, characterized in that, After obtaining the illumination status of the status indicator light based on the pixel classification result, the process includes: The status indicator light's illumination status is obtained by using a sliding window to capture a preset number of consecutive video frame images. Based on the illumination status of the status indicator light in the preset number of consecutive video frame images, the actual illumination status of the status indicator light is determined.
6. The method for monitoring status indicator lights according to claim 5, characterized in that, After obtaining the illumination status of the status indicator light based on the pixel classification result, the process includes: The timestamp information of multiple transition signals of the status indicator light in the actual lit state is obtained; wherein, the transition signal includes a rising edge signal from off to on, and a falling edge signal from on to off; Based on the timestamp information of multiple transition signals, the flashing period of the status indicator light in the actual lit state is obtained, so as to determine the state operation mode based on the flashing period.
7. A monitoring device for status indicator lights, characterized in that, include: The positioning area acquisition module is used to acquire the positioning area image of the status indicator in the video frame image in response to the acquisition of the video frame image of the status indicator, by using the reference position information of the reference template image; The overexposure judgment module is used to obtain the grayscale image corresponding to the image of the positioning area, and to determine whether the image of the positioning area is overexposed based on the proportion of bright pixels in the grayscale image. The illumination status acquisition module is used to classify the pixels of the positioning area image according to a reference color threshold if it is determined that there is no overexposure phenomenon in the positioning area image, so as to obtain the illumination status of the status indicator light according to the pixel classification result.
8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the monitoring method of the status indicator light according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the monitoring method for the status indicator light as described in any one of claims 1-6.
10. A computer program product comprising a computer program that, when executed by a processor, implements the monitoring method for a status indicator light as described in any one of claims 1-6.