A method and device for identifying the light-emitting state of an indicator light, and a terminal device

Abstract

This invention relates to the field of image processing technology and provides a method, device, and terminal equipment for recognizing the illumination state of an indicator light. The recognition method includes: acquiring a video to be recognized; using a first recognition model to select a target indicator light in the image to be recognized, obtaining a first intermediate image; inputting the video to be recognized into a second recognition model, and using the second recognition model to perform motion region detection on the target indicator light in the video to be recognized, obtaining a second intermediate image; performing contour detection on the second intermediate image, obtaining a third intermediate image; and determining the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image. Because this invention separately recognizes the rectangular selection box and the indicator light contour, and combines the rectangular selection box and the indicator light contour to comprehensively determine the illumination state of the target indicator light, it has multiple recognition dimensions and therefore high recognition accuracy.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, and particularly relates to a method, device and terminal equipment for recognizing the illumination status of an indicator light. Background Technology

[0002] Indicator lights are typically used to reflect the operating status of circuits (energized or de-energized), the operating status of electrical equipment (running, shut down, or under test), and its position status (closed or open). Indicator lights have a wide range of applications, often used for equipment status monitoring. For example, in substation operation and maintenance, monitoring the operational status of equipment is essential. By monitoring the flashing status of equipment indicator lights, it is possible to determine whether the substation equipment is operating normally, allowing for timely detection and correction of abnormal conditions, thereby improving the safety of substation operation.

[0003] Monitoring the flashing status of indicator lights on substation equipment is primarily done manually, through on-site inspections or video monitoring to determine equipment operating status. In substations with numerous devices, this method is extremely resource-intensive and inefficient. With the development of image recognition technology, it has been increasingly applied to indicator light identification. However, traditional image recognition techniques suffer from a high error rate in identifying indicator light status, failing to accurately determine their condition. This is a critical technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, device, terminal device, and computer-readable storage medium for identifying the illumination status of an indicator light, in order to solve the technical problem that traditional image recognition technology has a high error rate in identifying the status of indicator lights and cannot accurately identify the status of indicator lights.

[0005] A first aspect of the present invention provides a method for identifying the illumination state of an indicator light, comprising:

[0006] Collect the video to be identified;

[0007] The image to be identified in the video to be identified or the video to be identified is input into the first recognition model. The target indicator light in the image to be identified is selected by the first recognition model to obtain a first intermediate image. The first intermediate image includes a rectangular selection box of the target indicator light.

[0008] The video to be identified is input into the second recognition model, and the second recognition model performs motion region detection on the target indicator light in the video to be identified to obtain a second intermediate image;

[0009] Perform contour detection on the second intermediate image to obtain the third intermediate image;

[0010] The illumination state of the target indicator light is determined based on the positional relationship between the rectangular selection box in the first intermediate image and the outline in the third intermediate image.

[0011] Further, the step of determining the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image includes:

[0012] If the outline in the third intermediate image is within the rectangular selection box in the first intermediate image, then the target indicator light is determined to be in a flashing state.

[0013] If the outline is not present within the rectangular selection box in the first intermediate image, the illumination state of the target indicator light is determined based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified.

[0014] Further, the step of determining the illumination state of the target indicator light based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified if the outline does not exist within the rectangular selection box in the first intermediate image includes:

[0015] If the outline does not exist within the rectangular selection box in the first intermediate image, then calculate the first pixel mean of the rectangular selection box and the second pixel mean of the image to be identified;

[0016] If the average value of the first pixel is greater than the average value of the second pixel within a preset time period, then the target indicator light is determined to be in a constantly lit state.

[0017] If the average value of the first pixel is less than the average value of the second pixel within a preset time period, then the target indicator light is determined to be in an off state.

[0018] If the relationship between the average value of the first pixel and the average value of the second pixel changes within a preset time period, then the target indicator light is determined to be in a flashing state.

[0019] Furthermore, the identification method further includes:

[0020] If the contour in the second intermediate image exceeds the boundary of the rectangular selection box in the first intermediate image, then the initial image in the video to be recognized is segmented by median to obtain the fourth intermediate image.

[0021] The illumination state of the target indicator light is determined based on the first intermediate image and the fourth intermediate image.

[0022] Furthermore, the step of determining the illumination state of the target indicator light based on the first intermediate image and the fourth intermediate image further includes:

[0023] If the outline in the fourth intermediate image is within the rectangular selection box in the first intermediate image, then the target indicator light is determined to be in a flashing state.

[0024] If the outline is not present within the rectangular selection box in the fourth intermediate image, the illumination state of the target indicator light is determined based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified.

[0025] Furthermore, the second recognition model is a Gaussian mixture model;

[0026] The Gaussian mixture model is as follows:

[0027]

[0028] Where, χ T ={x (t) ,...,x (t-T)} represents the video to be identified at time t. For a motion estimation model that includes both background and foreground, M represents the number of Gaussian models in the Gaussian mixture model. This represents the weights, which are non-negative and sum to 1. This is an estimate of the mean of a Gaussian distribution. Here, N is the variance estimate of the Gaussian distribution, N is the covariance matrix, and I is the identity matrix. It is a video frame vector.

[0029] Further, the step of inputting the video to be identified into the second recognition model, and using the second recognition model to perform motion region detection on the target indicator light in the video to be identified, to obtain the second intermediate image, includes:

[0030] The video to be identified is input into the second recognition model, and the second recognition model detects the motion change region in the video to be identified.

[0031] The second intermediate image is obtained by binarizing the motion-changing region and the non-motion-changing region using the second recognition model.

[0032] A second aspect of the present invention provides a device for identifying the illumination state of an indicator light, comprising:

[0033] The acquisition unit is used to acquire the video to be identified;

[0034] The first processing unit is configured to input the image to be identified in the video to be identified or the video to be identified into a first recognition model, and to select the target indicator light in the image to be identified by the first recognition model to obtain a first intermediate image; the first intermediate image includes a rectangular selection box of the target indicator light;

[0035] The second processing unit is used to input the video to be identified into the second recognition model, and to perform motion region detection on the target indicator light in the video to be identified through the second recognition model to obtain a second intermediate image.

[0036] The third processing unit is used to perform contour detection on the second intermediate image to obtain a third intermediate image;

[0037] The determining unit is used to determine the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the second intermediate image.

[0038] A third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0040] The beneficial effects of this invention compared to existing technologies are as follows: This invention involves: acquiring a video to be identified; inputting the image to be identified or the video itself into a first recognition model; using the first recognition model to select a target indicator light in the image to be identified, obtaining a first intermediate image; the first intermediate image includes a rectangular selection box for the target indicator light; inputting the video to be identified into a second recognition model; using the second recognition model to perform motion region detection on the target indicator light in the video to be identified, obtaining a second intermediate image; performing contour detection on the second intermediate image, obtaining a third intermediate image; and determining the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image. Due to the above scheme, the first recognition model selects the image position of the image to be identified to obtain the first intermediate image. The second recognition model then performs motion region detection and contour recognition on the target indicator light in the video to be identified to obtain the second intermediate image. Furthermore, contour recognition is performed on the second intermediate image to obtain the third intermediate image. Finally, the positional relationship between the rectangular selection box and the contour of the target indicator light determines the illumination state of the target indicator light. Because this invention identifies the rectangular selection box and the indicator light outline separately, and combines the rectangular selection box and the indicator light outline to comprehensively determine the illumination state of the target indicator light, it has multiple recognition dimensions and therefore has high recognition accuracy. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A schematic flowchart of a method for identifying the illumination state of an indicator light provided by the present invention is shown;

[0043] Figure 2 The diagram shows a detailed schematic flowchart of step 103 in a method for identifying the illumination state of an indicator light provided by the present invention.

[0044] Figure 3 A schematic diagram of the indicator light contour recognition provided by the present invention is shown;

[0045] Figure 4 The diagram shows a schematic flowchart of step 104 in a method for identifying the illumination state of an indicator light provided by the present invention.

[0046] Figure 5A schematic flowchart illustrating step 1042 of a method for identifying the illumination state of an indicator light provided by the present invention is shown.

[0047] Figure 6 A schematic flowchart illustrating another method for identifying the illumination state of an indicator light provided by the present invention is shown.

[0048] Figure 7 A schematic diagram showing the indicator light provided by the present invention exhibiting a halo effect is shown;

[0049] Figure 8 A schematic diagram illustrating the removal of halos from indicator lights provided by the present invention is shown;

[0050] Figure 9 A schematic flowchart illustrating step 1044 of a method for identifying the illumination state of an indicator light provided by the present invention is shown.

[0051] Figure 10 A schematic diagram illustrating the overall flow of a method for identifying the illumination status of an indicator light provided by the present invention is shown.

[0052] Figure 11 A schematic diagram of an indicator light illumination status recognition device provided by the present invention is shown;

[0053] Figure 12 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0054] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0055] It should be noted that this solution can be applied to multiple application scenarios, such as automated factories, computer hangars, or power plants, where the illumination status of indicator lights needs to be identified. This invention does not limit the application scenarios; however, to better understand this invention, a power plant scenario is used as an example to explain the technical solution. Technical solutions for other application scenarios can be derived by analogy and will not be elaborated upon here.

[0056] First, this invention provides a method for identifying the illumination status of an indicator light. Please refer to [link / reference]. Figure 1 , Figure 1 A schematic flowchart of a method for identifying the illumination state of an indicator light provided by the present invention is shown. Figure 1As shown, the identification method may include the following steps:

[0057] Step 101: Collect the video to be identified.

[0058] The video to be identified refers to video data containing indicator lights that indicate a status to be identified. The video to be identified can be collected by an inspection robot that moves to a preset location based on navigation and positioning information. Alternatively, the video to be identified can be collected by a camera at a specific location. The video to be identified can also be video data directly input by the user.

[0059] The entity that collects the video to be identified and the entity that executes the technical solution of this invention can be the same device or different devices (for example, a robot collects the video to be identified and sends the video to be identified to a server, and the server executes the technical solution of this invention).

[0060] As an optional embodiment of the present invention, before recognizing the video to be recognized, image processing can be performed on the video, such as noise reduction or brightness adjustment, etc., to improve the image quality of the video to be recognized, thereby improving the recognition accuracy.

[0061] Step 102: Input the image to be identified in the video to be identified or the video to be identified into the first recognition model, and use the first recognition model to select the target indicator light in the image to be identified to obtain a first intermediate image; the first intermediate image includes a rectangular selection box of the target indicator light.

[0062] The first recognition model is used for object detection. This first recognition model can use object detection networks such as SSD or YOLO. The feature extraction backbone network in the object detection model can be replaced with a lightweight MobileNet network.

[0063] Before performing step 102, the first recognition model needs to be trained. The training process is as follows:

[0064] ① First training step: Collect a large number of training sample images or training sample videos of device indicator lights:

[0065] For solutions that incorporate deep learning algorithms, extensive data collection is a prerequisite for implementation. For example, the detection of indicator lights on substation equipment requires a large number of training sample images or videos to train the first recognition model.

[0066] ② Second training step: Annotation processing of training sample images or training sample videos:

[0067] During the acquisition of training sample images or videos, the image includes not only indicator light areas but also other non-indicator light areas. In practical applications, inspection robots need to identify the status of indicator light areas in the on-site images. Therefore, it is necessary to annotate the training sample images or videos to obtain annotation data. The annotation data refers to selecting the target indicator lights in the training sample images or videos to obtain rectangular selection boxes.

[0068] ③ Third training step: Model training:

[0069] For each training sample image or training sample video, the following steps are repeated sequentially to obtain the first recognition model: Output the training sample image or training sample video to the first initial model to obtain the recognition result output by the first initial model. Calculate the first loss value between the recognition result and the labeled data. Adjust the network parameters of the first initial model based on the first loss value. When the recognition performance of the first initial model converges or the training iterations reach their maximum value, model training is stopped, and the currently obtained first initial model is used as the first recognition model.

[0070] When performing step 102, only the pre-stored first recognition model needs to be retrieved from the memory. The image or video to be recognized is then input into the first recognition model. The target indicator light in the image to be recognized is selected using the first recognition model to obtain a first intermediate image. The first intermediate image includes a rectangular selection box corresponding to the target indicator light (the rectangular selection box is represented by four coordinate values). The image to be recognized is a frame from the video to be recognized.

[0071] Step 103: Input the video to be identified into the second recognition model, and use the second recognition model to detect the motion region of the target indicator light in the video to be identified, and obtain the second intermediate image.

[0072] The second recognition model is a Gaussian mixture model, used to identify the motion region of moving objects (blinking indicator lights) in the video to be recognized.

[0073] The Gaussian mixture model is as follows:

[0074]

[0075] Where, χ T ={x (t) ,...,x (t-T)} represents the video to be identified at time t. For a motion estimation model that includes both background and foreground, M represents the number of Gaussian models in the Gaussian mixture model. This represents the weights, which are non-negative and sum to 1. This is an estimate of the mean of a Gaussian distribution. Here, N is the variance estimate of the Gaussian distribution, N is the covariance matrix, and I is the identity matrix. It is a video frame vector.

[0076] It is worth noting that the first recognition model can accept either an image or a video as input. The second recognition model, however, can only accept videos.

[0077] Specifically, step 103 includes steps 1031 to 1032. For example... Figure 2 As shown, Figure 2 The diagram shows a schematic flowchart of step 103 in a method for identifying the illumination state of an indicator light provided by the present invention.

[0078] Step 1031: Input the video to be identified into the second recognition model, and use the second recognition model to detect the motion change region in the video to be identified.

[0079] The motion change area refers to the image area corresponding to the indicator light flashing. The image area corresponding to the indicator light being constantly on or off is not considered the motion change area.

[0080] Step 1032: Binarize the motion-changing region and the non-motion-changing region using the second recognition model to obtain the second intermediate image.

[0081] Image binarization refers to setting the grayscale value of pixels in an image to 0 or 255, thus presenting the entire image with only black and white visual effects. Binarization can be used to perform contour recognition on flashing indicator lights. If a moving object appears in the video to be recognized, the second recognition model will detect the moving object as the foreground region, while targets that remain stable over a period of time will be detected as the background region.

[0082] The second recognition model determines whether a pixel value belongs to the background region in the following way:

[0083]

[0084] Among them, c thr The threshold is determined for background pixels, where χ is the video to be identified. This is the background model.

[0085] Step 104: Perform contour detection on the second intermediate image to obtain the third intermediate image.

[0086] The indicator light outline is represented by multiple coordinate values. Please refer to [link / reference]. Figure 3 , Figure 3 A schematic diagram of the indicator light contour recognition provided by the present invention is shown. Figure 3As shown, the white area represents the area of ​​motion change, i.e., the outline of the indicator light. The black area represents the area of ​​no motion change. Contour detection technology is a traditional technology and will not be elaborated upon here.

[0087] Step 105: Determine the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image.

[0088] Since the rectangular selection box in the first intermediate image is used to identify the position of the indicator light, while the indicator light outline in the third intermediate image is used to identify the area of ​​moving objects (the flashing indicator light), the positional relationship between the rectangular selection box and the outline can be used to determine whether the target indicator light is flashing.

[0089] Specifically, step 104 includes steps 1041 to 1042. For example... Figure 4 As shown, Figure 4 The diagram shows a schematic flowchart of step 104 in a method for identifying the illumination status of an indicator light provided by the present invention.

[0090] Step 1041: If the contour in the third intermediate image is within the rectangular selection box in the first intermediate image, then the target indicator light is determined to be in a flashing state.

[0091] That is, when the position area of ​​the rectangular selection box is contained within the position area of ​​the outline, it can be determined that the target indicator light is in a flashing state.

[0092] Step 1042: If the outline does not exist within the rectangular selection box in the first intermediate image, the illumination state of the target indicator light is determined based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified.

[0093] If there is no outline within the rectangular selection box in the first intermediate image, the target indicator light may be constantly on or off. The illumination state of the target indicator light can be determined based on the average first pixel value of the rectangular selection box and the average second pixel value of the image to be identified (or the video to be identified). Specifically:

[0094] Specifically, step 1042 includes steps A1 to A4. For example... Figure 5 As shown, Figure 5 The diagram shows a schematic flowchart of step 1042 in a method for identifying the illumination status of an indicator light provided by the present invention.

[0095] Step A1: If the outline does not exist within the rectangular selection box in the first intermediate image, then calculate the first pixel mean of the rectangular selection box and the second pixel mean of the image to be identified.

[0096] The first pixel mean refers to the average pixel value of all pixels within the rectangular selection area. The second pixel mean refers to the average pixel value of all pixels in the image or video to be identified.

[0097] Step A2: If the average value of the first pixel is greater than the average value of the second pixel within a preset time period, then the target indicator light is determined to be in a constantly lit state.

[0098] Because the brightness of the target indicator light area differs significantly from the brightness of the surrounding image area when the indicator light is constantly on and off, the illumination state of the target indicator light can be determined based on the first and second pixel averages.

[0099] If the average value of the first pixel is greater than the average value of the second pixel, it means that the average brightness of the rectangular selection box is greater than the average brightness of the image or video to be identified. Therefore, it can be determined that the target indicator light is in a constant-on state.

[0100] Step A3: If the average value of the first pixel is less than the average value of the second pixel within a preset time period, then the target indicator light is determined to be in the off state.

[0101] If the average value of the first pixel is greater than the average value of the second pixel, it means that the average brightness of the rectangular selection box is less than the average brightness of the image or video to be identified. Therefore, it can be determined that the target indicator light is in the off state.

[0102] Step A4: If the relationship between the average value of the first pixel and the average value of the second pixel changes within a preset time period, then the target indicator light is determined to be in a flashing state.

[0103] When the average value of the first pixel and the average value of the second pixel fluctuate in size within a preset time period (i.e., the size relationship between the average value of the first pixel and the average value of the second pixel changes), the target indicator light is determined to be in a flashing state.

[0104] It is understood that, in this embodiment, the average value of the second pixel of the image or video to be identified is used as a threshold and compared with the average value of the first pixel to distinguish the illumination state of the target indicator light.

[0105] Optionally, step 104 may also include steps 1043 and 1044. For example... Figure 6 As shown, Figure 6 A schematic flowchart illustrating another method for identifying the illumination status of an indicator light provided by the present invention is shown.

[0106] Step 1043: If the contour in the third intermediate image exceeds the boundary of the rectangular selection box in the first intermediate image, then the initial image in the video to be recognized is segmented at the median to obtain the fourth intermediate image.

[0107] Because substation equipment indicator lights come in various types, and their images are affected by factors such as angle, lighting, density of indicator lights, and high brightness, some indicator lights in the video to be identified may be subject to strong halos. The flickering of these halos can lead to incorrect identification of the status of these indicator lights. When indicator lights are densely distributed or have high brightness, the halos can severely interfere with the identification of adjacent indicator lights, resulting in incorrect status recognition. When the outline in the third intermediate image exceeds the boundary of the rectangular selection box in the first intermediate image, it indicates that the indicator lights in the video to be identified are experiencing halos. This can be eliminated or reduced by segmenting the image to obtain a fourth intermediate image.

[0108] The principle of median segmentation is as follows: First, all pixels are mapped to the RGB color space. Within this three-dimensional space, subspaces are repeatedly created. Finally, the average of the pixels in each subspace is used as the extraction result. When segmenting blocks, the largest block (those with the longest side, largest volume, or most pixels) is selected. The cutting point should be located along the edge, ensuring that the two resulting blocks each contain half the pixels. Starting with the entire image as a cuboid, the longest side of the RGB color space is cut in half along the median, resulting in two cuboids containing the same number of pixels. This process is repeated until the final number of cuboids equals the number of main colors.

[0109] To better understand the difference between the third and fourth intermediate images, please refer to... Figure 7 and Figure 8 , Figure 7 A schematic diagram showing the indicator light provided by the present invention exhibiting a halo effect is shown. Figure 8 A schematic diagram illustrating the removal of halos from indicator lights provided by the present invention is shown. For example... Figure 7 As shown, a halo effect appears around the target indicator light in the center of the image. After being split into two parts, the result is as follows: Figure 8 The image shown is a screenshot.

[0110] Step 1044: Determine the illumination state of the target indicator light based on the first intermediate image and the fourth intermediate image.

[0111] Specifically, step 1044 includes steps B1 to B2. For example... Figure 9 As shown, Figure 9 The diagram shows a schematic flowchart of step 1044 in a method for identifying the illumination status of an indicator light provided by the present invention.

[0112] Step B1: If the outline in the fourth intermediate image is within the rectangular selection box in the first intermediate image, then the target indicator light is determined to be in a flashing state.

[0113] Step B2: If the outline does not exist within the rectangular selection box in the fourth intermediate image, the illumination state of the target indicator light is determined based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified.

[0114] The execution principles of steps B1 and B2 are similar to those of... Figure 4 Steps 1041 and 1042 in the illustrated embodiment are the same. For detailed execution process, please refer to steps 1041 and 1042, which will not be repeated here.

[0115] As an optional embodiment of the present invention, after the automatic identification of the indicator light status is completed, the present invention can automatically generate a report containing indicator light information and its flashing status. Monitoring personnel can intuitively and efficiently understand the operating status of each device by viewing the report.

[0116] To better understand the technical solution of the present invention, a general description of the technical solution of the present invention will be provided here in conjunction with the accompanying drawings. Please refer to... Figure 10 , Figure 10 A schematic diagram illustrating the overall flow of a method for identifying the illumination state of an indicator light provided by the present invention is shown. Figure 10 As shown, the image or video to be identified (i.e., the indicator light image in the figure) is input into the first recognition model (i.e., the indicator light detection model) and the second recognition model (i.e., the Gaussian mixture model), respectively. The first recognition model obtains an image containing the indicator light's rectangular frame (i.e., the first intermediate image). The second recognition model obtains a binarized motion image (i.e., the second intermediate image). Contour detection is performed on the second intermediate image to obtain a contour detection image (i.e., the third intermediate image). The rectangular frame in the first intermediate image and the contour in the third intermediate image are matched. If the rectangular frame includes the contour, the indicator light is in a flashing state. If the rectangular frame does not include the contour, the first pixel mean of the image or video to be identified is calculated, and the second pixel mean within the rectangular frame is calculated. The first pixel mean is used as a threshold. If the second pixel mean is greater than the threshold, the indicator light is determined to be in a constantly lit state. If the second pixel mean is less than the threshold, the indicator light is determined to be in an off state. If the relationship between the second pixel mean and the threshold is inconsistent, the indicator light is determined to be in a flashing state. If the contour exceeds the rectangular frame, the initial image in the video to be identified is segmented by median to obtain a fourth intermediate image. The rectangle in the first intermediate image is compared with the outline in the fourth intermediate image (the comparison process is the same as that for the third intermediate image).

[0117] This embodiment involves: acquiring a video to be identified; inputting the image to be identified from the video, or the video itself, into a first recognition model; using the first recognition model to select a target indicator light in the image to be identified, obtaining a first intermediate image; the first intermediate image includes a rectangular selection box for the target indicator light; inputting the video to be identified into a second recognition model; using the second recognition model to perform motion region detection on the target indicator light in the video to be identified, obtaining a second intermediate image; performing contour detection on the second intermediate image, obtaining a third intermediate image; and determining the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image. In this scheme, the first recognition model selects the image position of the image to be identified to obtain the first intermediate image. The second recognition model then performs motion region detection on the target indicator light in the video to be identified to obtain the second intermediate image. Contour recognition is then performed on the second intermediate image to obtain the third intermediate image. Finally, the positional relationship between the rectangular selection box and the contour of the target indicator light determines the illumination state of the target indicator light. Because this invention identifies the rectangular selection box and the indicator light outline separately, and combines the rectangular selection box and the indicator light outline to comprehensively determine the illumination state of the target indicator light, it has multiple recognition dimensions and therefore has high recognition accuracy.

[0118] like Figure 11 This invention provides a device 11 for recognizing the illumination status of an indicator light. Please refer to [link / reference]. Figure 11 , Figure 11 A schematic diagram of an indicator light illumination status recognition device provided by the present invention is shown, as follows: Figure 11 The device for identifying the illumination status of an indicator light includes:

[0119] Acquisition unit 111 is used to acquire the video to be identified;

[0120] The first processing unit 112 is used to input the image to be identified in the video to be identified or the video to be identified into a first recognition model, and to select the target indicator light in the image to be identified by the first recognition model to obtain a first intermediate image; the first intermediate image includes a rectangular selection box of the target indicator light;

[0121] The second processing unit 113 is used to input the video to be identified into the second recognition model, and to perform motion region detection on the target indicator light in the video to be identified through the second recognition model to obtain a second intermediate image.

[0122] The third processing unit 114 is used to perform contour detection on the second intermediate image to obtain a third intermediate image;

[0123] The matching unit 115 is used to determine the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image.

[0124] This invention provides a device for recognizing the illumination state of an indicator light. The device acquires a video to be recognized; inputs the image to be recognized from the video, or the video itself, into a first recognition model; the first recognition model selects a target indicator light in the image to be recognized to obtain a first intermediate image; the first intermediate image includes a rectangular selection box of the target indicator light; the video to be recognized is input into a second recognition model; the second recognition model performs motion region detection on the target indicator light in the video to be recognized to obtain a second intermediate image; contour detection is performed on the second intermediate image to obtain a third intermediate image; the illumination state of the target indicator light is determined based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image. Due to the above scheme, the first recognition model selects the image position of the image to be recognized to obtain the first intermediate image. The second recognition model performs motion region detection on the target indicator light in the video to be recognized to obtain the second intermediate image. Contour recognition is then performed on the second intermediate image to obtain the third intermediate image. The illumination state of the target indicator light is determined based on the positional relationship between the rectangular selection box and the contour of the target indicator light. Because this invention identifies the rectangular selection box and the indicator light outline separately, and combines the rectangular selection box and the indicator light outline to comprehensively determine the illumination state of the target indicator light, it has multiple recognition dimensions and therefore has high recognition accuracy.

[0125] Figure 12 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 12 As shown, a terminal device 12 in this embodiment includes: a processor 120, a memory 121, and a computer program 122 stored in the memory 121 and executable on the processor 120, such as a method program for recognizing the illumination state of an indicator light. When the processor 120 executes the computer program 122, it implements the steps in the various embodiments of the method for recognizing the illumination state of an indicator light described above, for example... Figure 1 Steps 101 to 104 are shown. Alternatively, when the processor 120 executes the computer program 122, it implements the functions of each unit in the above-described device embodiments, for example... Figure 11 The functions of units 111 to 115 are shown.

[0126] For example, the computer program 122 can be divided into one or more units, which are stored in the memory 121 and executed by the processor 120 to complete the present invention. The one or more units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 122 in the terminal device 12. For example, the specific functions of each unit of the computer program 122 can be divided as follows:

[0127] The acquisition unit is used to acquire the video to be identified;

[0128] The first processing unit is configured to input the image to be identified in the video to be identified or the video to be identified into a first recognition model, and to select the target indicator light in the image to be identified by the first recognition model to obtain a first intermediate image; the first intermediate image includes a rectangular selection box of the target indicator light;

[0129] The second processing unit is used to input the video to be identified into the second recognition model, and to perform motion region detection on the target indicator light in the video to be identified through the second recognition model to obtain a second intermediate image.

[0130] The third processing unit is used to perform contour detection on the second intermediate image to obtain a third intermediate image;

[0131] The determining unit is used to determine the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the second intermediate image.

[0132] The terminal device includes, but is not limited to, a processor 120 and a memory 121. Those skilled in the art will understand that... Figure 12 This is merely an example of a terminal device 12 and does not constitute a limitation on a terminal device 12. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0133] The processor 120 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0134] The memory 121 can be an internal storage unit of the terminal device 12, such as a hard disk or memory of the terminal device 12. The memory 121 can also be an external storage device of the terminal device 12, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 12. Furthermore, the memory 121 can include both internal and external storage units of the terminal device 12. The memory 121 is used to store the computer program and other programs and data required by the roaming control device. The memory 121 can also be used to temporarily store data that has been output or will be output.

[0135] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0136] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0138] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0139] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0141] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0142] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0143] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units.

[0145] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0146] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0147] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0148] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0149] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0150] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for identifying the illumination status of an indicator light, characterized in that, The identification method includes: Collect the video to be identified; The image to be identified in the video to be identified or the video to be identified is input into the first recognition model. The target indicator light in the image to be identified is selected by the first recognition model to obtain a first intermediate image. The first intermediate image includes a rectangular selection box of the target indicator light. The video to be identified is input into the second recognition model, and the second recognition model performs motion region detection on the target indicator light in the video to be identified to obtain a second intermediate image; Perform contour detection on the second intermediate image to obtain the third intermediate image; The illumination state of the target indicator light is determined based on the positional relationship between the rectangular selection box in the first intermediate image and the outline in the third intermediate image; The step of determining the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image includes: If the outline in the third intermediate image is within the rectangular selection box in the first intermediate image, then the target indicator light is determined to be in a flashing state. If the outline does not exist within the rectangular selection box in the first intermediate image, the illumination state of the target indicator light is determined based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified. The step of determining the illumination state of the target indicator light based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified if the outline does not exist within the rectangular selection box in the first intermediate image includes: If the outline does not exist within the rectangular selection box in the first intermediate image, then calculate the first pixel mean of the rectangular selection box and the second pixel mean of the image to be identified; If the average value of the first pixel is greater than the average value of the second pixel within a preset time period, then the target indicator light is determined to be in a constantly lit state. If the average value of the first pixel is less than the average value of the second pixel within a preset time period, then the target indicator light is determined to be in an off state. If the relationship between the average value of the first pixel and the average value of the second pixel changes within a preset time period, then the target indicator light is determined to be in a flashing state.

2. The identification method as described in claim 1, characterized in that, The identification method further includes: If the contour in the third intermediate image exceeds the boundary of the rectangular selection box in the first intermediate image, then the initial image in the video to be recognized is segmented at the median to obtain the fourth intermediate image. The illumination state of the target indicator light is determined based on the first intermediate image and the fourth intermediate image.

3. The identification method as described in claim 2, characterized in that, The step of determining the illumination state of the target indicator light based on the first intermediate image and the fourth intermediate image further includes: If the outline in the fourth intermediate image is within the rectangular selection box in the first intermediate image, then the target indicator light is determined to be in a flashing state. If the outline is not present within the rectangular selection box in the fourth intermediate image, the illumination state of the target indicator light is determined based on the first pixel average of the rectangular selection box and the second pixel average of the image to be identified.

4. The identification method as described in claim 1, characterized in that, The second recognition model is a Gaussian mixture model; The Gaussian mixture model is as follows: ;in, For the video to be identified at time t, For a motion estimation model that includes both background and foreground, M represents the number of Gaussian models in the Gaussian mixture model. This represents the weights, which are non-negative and sum to 1. This is an estimate of the mean of a Gaussian distribution. Here, represents the variance estimate of the Gaussian distribution, and N is the covariance matrix. It is the identity matrix. It is a video frame vector.

5. The identification method as described in claim 1, characterized in that, The step of inputting the video to be identified into the second recognition model, and using the second recognition model to perform motion region detection on the target indicator light in the video to be identified, to obtain the second intermediate image, includes: The video to be identified is input into the second recognition model, and the second recognition model detects the motion change region in the video to be identified. The second intermediate image is obtained by binarizing the motion-changing region and the non-motion-changing region using the second recognition model.

6. A device for identifying the illumination status of an indicator light, characterized in that, A method for identifying the illumination state of an indicator light according to any one of claims 1 to 5, the identification device comprising: The acquisition unit is used to acquire the video to be identified; The first processing unit is configured to input the image to be identified in the video to be identified or the video to be identified into a first recognition model, and to select the target indicator light in the image to be identified by the first recognition model to obtain a first intermediate image; the first intermediate image includes a rectangular selection box of the target indicator light; The second processing unit is used to input the video to be identified into the second recognition model, and to perform motion region detection on the target indicator light in the video to be identified through the second recognition model to obtain a second intermediate image. The third processing unit is used to perform contour detection on the second intermediate image to obtain a third intermediate image; A matching unit is used to determine the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the third intermediate image; The determining unit is used to determine the illumination state of the target indicator light based on the positional relationship between the rectangular selection box in the first intermediate image and the contour in the second intermediate image.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images