Lamp detection method, device and equipment and storage medium

[0083] Exemplary embodiments will be described in detail herein, and examples are illustrated in the drawings. The following description is related to the drawings, unless otherwise indicated, the same figures in the different drawings represent the same or similar elements. The embodiments described in the exemplary embodiments will not represent all embodiments consistent with the present invention. In contrast, examples of apparatus and methods consistent with some aspects of the invention, as detailed in the appended claims.

[0084] The terms used in the present invention are only for the purpose of describing particular embodiments, not intended to limit the invention. "One", "one", "one", "" "and" "" as used in the present invention and the appended claims are also intended to include most forms unless the context clearly represents other meanings. It should also be understood that the terms "and / or" as used herein refer to any or more of any or all possible combinations of one or more associated listing items.

[0085] It should be understood that various information may be described in terms of the terms first, second, third, etc., but this information should not be limited to these terms. These terms are used only to distinguish the same type of information from each other. For example, the first information may be referred to as a second information without departing from the scope of the invention, and the second information can also be referred to as the first information. Depending on the context, if the words used herein "If" can be interpreted as "in ..." or "when ..." or "in response to determination".

[0086] In the first aspect, at least one embodiment of the present invention provides a method of detecting a lamp, refer to the attachment figure 1 It shows the process of the method, including steps S101 to S103.

[0087] The method can be used to detect the landscape lamp of the building, or the other type of luminaire is detected, and the landscape lamp is mainly described as an example of the detection method provided in this embodiment, but it is understood that this is not The type of lamp type is targeted for the detection method provided in the present disclosure.

[0088] The landscape lamp is generally at a higher position, so the object targeted by the detection method can be captured in the high-altitude wide-angle camera that has a high-serving wide-angle camera, because the horizons of the high-altitude wide-angle camera can be photographed in a wide range of cities. High-rise building. The camera or video taken by the high-altitude wide-angle camera can be photographed at night, thereby enabling the image or video, and the landscape lamp detected, the landscape light detected is light, or Scene light in bright state.

[0089] In addition, the method can be performed by an electronic device such as a terminal device or a server, and the terminal device can be a user equipment (UE Equipment, UE), mobile device, user terminal, terminal, cellular telephone, cordless telephone, personal digital processing (Personal Digital Assistant, PDA) Handheld devices, computing devices, in-vehicle devices, wearable devices, etc., which can be implemented by the method of calling computer readable instructions stored in the memory. Alternatively, the method can be performed by the server, the server can be a local server, a cloud server, and the like.

[0090] In step S101, an image to be detected, wherein the scene to be detected is in the field including a luminaire.

[0091] Wherein, the image to be detected can be an image captured by the above-mentioned high-altitude wide-angle camera, or a frame image recorded by the high-altitude wide-angle camera. Since the scene in the shooting angle of the high-altitude wide-angle camera includes a landscape lamp, the image of the landscape lamp is included in the resulting image. The high-altitude wide-angle camera can take an image at a certain time interval, obtain an image per taken as the image to be detected; the high-altitude wide-angle camera can continuously shoot a video, and the video frame can be extracted from the video in the video according to a certain time interval, and will Subscribed video frames as the image to be detected.

[0092] In step S102, the image to be detected is borid to obtain a binarized image, and the luminaire candidate box is determined based on the binarized image.

[0093] If the image to be detected is a grayscale image, the detected image is used directly to perform binarization processing. If the image to be detected is a color image, you can first convert the image to the grayscale image, that is, the corresponding brightness value can be determined according to the red (R), yellow (G), blue (B) pixel value according to the red (R) value of each pixel. .

[0094] When bisarrative processing, a brightness threshold (e.g., 230) can be provided, and the brightness of the pixel greater than or equal to the brightness threshold is 255, and the brightness of the pixel smaller than the brightness threshold is adjusted to 0, thereby enabling each pixel One of the brightness is 255 (bright pixels) and 0 (dark pixels), adjacent multiple bright pixels can constitute a communication area, and adjacent multiple dark pixels can also form a communication area, and thus the entire binarized image. A communication region consisting of a bright pixel consisting of a bright pixel consisting of a communication region consisting of a dark pixel. Since the image to be detected is taken at night, the brightness of the landscape lamp (in bright state) is higher than the other objects in the scene, so the communication area consisting of bright pixels is a large probability of landscape lamp, so it can be determined In the binarized image, the communication area consisting of bright pixels is the lighting candidate frame.

[0095] After binarization treatment, the binarized image can also be image corrosion treatment, that is, the peripheral pixel (the range of the range is determined according to the pre-set dimension, for example, the dimension of 2 * 2) is highlights bright pixels ( It is to be kept at 255), and the bright pixels that include dark pixels are adjusted to dark pixels (adjust their brightness to 0). Through the image corrosion processing, the noise point in the binarized image can be eliminated, and the edges of the communication area in the binarized image are smooth.

[0096] After bisarrative treatment, it is also possible to remove the binarized image, which is less than a communication area of ​​the preset area threshold, since these communication regions that are less than the area threshold may be noise points. Since the communication area is identified as the area in which the landscape lamp is located, the area threshold can be determined according to the area of ​​the area of ​​the landscape lamp. By removing a small area of ​​communication, the noise point can be reduced, making the subsequent detection of the candidate area more targeted, lowered load, improve efficiency, and accuracy.

[0097] It will be appreciated that the above two types of removal of noise (i.e., processing of image corrosion processing and noise point of removal area), can be used, or it can be used; when using two processes, it is possible to perform images. Corrosion treatment, remove less area of ​​noise.

[0098] In step S103, the candidate area of ​​the lamp candidate frame to be detected in the detected image is identified to determine the lamp target box of the image to be detected, wherein the lamp target frame is the position where the lamp is located.

[0099] Among them, each of the candidate zone can be input to a classification neural network pre-completed training, the classified neural network outputs a classification result of each of the candidate zone, wherein the classification results include a luminaire and a non-lamp; The classification result is the candidate area of ​​the luminaire to determine the light fixture.

[0100] The training set can be prepared in advance, the sample image in the training set can be selected, and the area of ​​the image of the luminaire is labeled as a luminaire, and the remaining local area is labeled as a comparison box; then sample the sample In the image input to the classified neural network, the prediction result of each partial area is output, and the network loss value is determined based on the forecast results, and the network parameters of the classified neural network are adjusted according to the network loss value until the classification neural network convergence.

[0101]The lamp target box can be a minimum rectangle frame that can frame like a luminaire. There is a luminaire in each of the lamp target boxes, and the state of the lamp is a bright state. In other words, the lamp target box to be detected in this step is to be detected, in the bright state in the image to be detected. picture.

[0102] According to the above embodiment, by acquiring the image to be detected, the binarization image is obtained, thereby obtaining a binarized image, and then determines the lamp candidate frame based on the binarized image, and then treating the lighting candidate box in the detection image. The candidate zone is identified to determine if there is a luminaire in each candidate area, that is, determine that the lamp target box in the image to be detected, the lighting target box is the position where the luminaire is located. Since the lamp candidate box is selected by the binarization process, it is necessary to perform targeted identification of the candidate area corresponding to the luminaire candidate box, which reduces the scope of the identification, improves the accuracy of the identification, and increases the accuracy and robustness of the lamp detection.

[0103] In some embodiments of the present disclosure, the image to be detected is a frame image to be detected, and the scene to detect the video includes a luminaire;

[0104] Based on this, please refer to the attachment figure 2 After step S101, the luminaire detecting method further includes step S104: The brightness of the light fixture corresponding to the light fixture of the reference image is determined according to the light fixture frame of the light fixture to the image to be detected. Wherein, the reference image is a frame image before the video to be detected by the reference image.

[0105] However, the detection method provided in this embodiment is to repeat the loop operation for each acquired image, that is, each time it acquires from step S101 to S103 after the image to be detected is obtained from step S101 to S103. Target block. Since the last acquired image is also determined that the lamp target box is determined, it can be determined by comparing the target block of the image to be detected and the target box to be detected, determine each of the target blocks to be detected. The light and dimming state of the corresponding luminaire, for example, a target box in the last time to detect is also a target box in the corresponding area in the image to be detected, and the light fixture corresponding to the target box is considered to be bright, ie in two The lamp is not extinguished between the image to be detected; a target box in the image to be detected does not have a target box in the corresponding area in the image to be detected, and the light fixture corresponding to the target box is considered to be dark. That is, the luminaire is extinguished between the two acquisition images.

[0106] It should be noted that in order to distinguish the image acquired two times, the last acquired image can be referred to as a reference image, and the object to be detected in this time is continued to be detected. Therefore, the image to be detected and the reference image is an image of the two frames in the video to be detected, and the reference image is before, the image to be detected is behind.

[0107] Since the image to be detected each time, after determining the lamp target box, it is necessary to determine the lamp target box after the next acquired image is determined, and therefore, the image to be detected each time is determined to detect the target target. Save it after the frame is saved. Further, since each acquired image is only used as a reference image, after step S104 determines the brightness of the light fixture corresponding to each of the light fixtures of the reference image, the reference image can be deleted. The image to be detected is saved as a reference image, and saves the lamp target box to be detected image, which is to update the reference image at any time, thereby reducing memory occupancy, and increasing the definite precision of the lights shade.

[0108] It will be appreciated that when the image to be detected is acquired from the image to be detected, step S104 is not executed after determining the image of the image to be detected, but the image to be detected is stored as a reference image. And save the lamp target box for the image to be detected.

[0109] In one example, the image to be detected can be obtained at a certain time interval, i.e., when the image to be detected is obtained from the video to be detected, the time interval of the reference image is a frame image of the preset time long, determined as The image to be detected. The preset time can be 3s, which can be repeatedly obtained in accordance with the preset time.

[0110] In one example, you can follow image 3 As shown, according to the light fixture of the light fixture to the image to be detected, it is determined that the lighting of the light fixture corresponding to the light fixture of the reference image, including steps S301 to S303.

[0111] In step S301, each of the light fixtures of the reference image and each of the lamp target frames to be detected an image are determined.

[0112] Take each of the light fix target boxes in the image in order, and after each get the luminaire target box, determine the mating ratio of each of the lamp target frames to be detected to be detected.

[0113] In step S302, when the signal target frame of the reference image is smaller than the intersecting ratio of each of the lamp target frames to be detected, determine the reference image. The light and dark state of the luminaire corresponding to the lamp target box is dark.

[0114] In step S303, in the case where the light fixture frame of the reference image is greater than or equal to the first ratio threshold in the case where the interstrous ratio of any of the lamp target frames to be detected, it is determined. The light-dimensional state of the light fixture corresponding to the light fixture frame is bright.

[0115] Reference The image of the lamp target box is greater than the preset first proportional threshold, which is the same, or corresponding to the same one of the two luminaires, or corresponding to the same one of the two luminaires. Lamp, that is, the light fixture in the reference image is in the image in the image to be detected. The bright and dark state is bright.

[0116] Compared, a light fixture of the reference image and the interstitial ratio of any of the lamp target frames to be detected, and is less than the preset ratio threshold, the table is laminated to detect the lamp target block that matches the lamp target box in the image. . Therefore, it is possible to determine that the brightness of the light fixture corresponding to the light fixture of the reference image is dark.

[0117] In this example, the integration ratio of the luminaire target box determines the brightness of the luminaire, since the luminaire target frame is the position where the fixture is located, this way is determined that the lighting is simple and dark, and it is more accurate.

[0118] In another example, you can follow Figure 4 As shown, according to the lamp target block to which the image to be detected, the light fixture of the fixture corresponding to the light fixture of the reference image is determined, including steps S401 to S403.

[0119] In step S401, it is determined that the pixel value average value of all the pixels in the light fixture frame of the reference image is a first average value, and determines that all pixels in the light fixture in the image to be detected. The pixel value average is the second average.

[0120] In step S402, in the case where the difference between the first average is greater than the preset first pixel threshold value, it is determined that the bright and dark state of the fixture corresponding to the light fixture is dark.

[0121] In step S403, in the case where the difference between the first average is less than or equal to the preset first pixel threshold, determine the bright and dark state of the fixture corresponding to the lamp to be fixed. .

[0122] Among them, the first pixel threshold can be preset to 100. In the dark state, it is taken after the moment of the reference image corresponds to the reference image.

[0123] In this embodiment, by the average pixel difference between the corresponding target frame, it is accurately determined whether the fixture is turned off, thereby improving the recognition accuracy of the light-shaking state.

[0124] It can be understood that the above is image 3 The method of determining the brightness of the lamp in the example shown, and Figure 4 The method of determining the light and dark state of the fixture in the example shown, can be used or used. When combining two ways to determine the light and dark state of the fixture, you can use image 3 The way shown is different, when step S302 is executed, the intersection ratio of the lamp target frame of the reference image and the lamp target frame to detect the image is less than the preset first In the case of a proportional threshold, determine the light fixture frame of the reference image as a candidate target box; then execute Figure 4 As shown, the difference is that only the pixel values ​​of all the pixels in the candidate target box are only determined in step S401, and only the light fixtures corresponding to the candidate target box are determined to be detected. The pixel value average of all the pixels is the second average, and only the first average value and the second average are determined by the first average value and the second average value, ie at the first average value. The difference between the second average is greater than the preset first pixel threshold, determines the bright and dark state of the luminaire corresponding to the lamp candidate box, in the first average value and the second average value. The value is smaller than or equal to the preset first pixel threshold, determines the bright and dark state of the lamps corresponding to the lamp candidate frame to be bright.

[0125] In combination, it is judged that the brightness of the luminaire corresponding to the light fixture frame is first, first utilized the interstitial comparison between the target box, the target, the candidate box, the candidate box, and then focus on the pixel comparison of the candidate box to determine the lamp. Small and dark state. It has improved the accuracy of the light-shaping and dark state judgment, and avoids excessive improvement of the calculation load.

[0126] In some embodiments of the present disclosure, the to-detect image can be detected in the following manner: First, the pixel pair of pixels of each pixel pair to be detected and each pixel between the reference image may be determined, wherein One pixel point in the pixel pair is in the position in which the image to be detected, and the other pixel point is the same in the reference image; Next, in all pixel pairs, the pixel difference is greater than the preset. The first number of pixel pairs of the second pixel threshold; Value processing.

[0127] Among them, the pixel difference can be subtracted by the corresponding two pixels, and the difference is absolutely worth it. The size of the pixel difference can be used to form a matching degree of the scene corresponding to the detected image and the reference image corresponding to the reference image, and less than or equal to the pixel difference of the second pixel threshold, characterizes the same scene in the two images. , The pixel difference greater than the second pixel threshold is characterized by the scene corresponding to the pixel in the two images. Further, the ratio of the number of first numbers to all of the pixel pairs is greater than the preset second ratio threshold, which characterizes the same scene corresponding to the two images, that is, the high-altitude wide-angle camera is the same when it acquires the two images. After obtaining the reference image, there is no movement, so in this case, the detection image can be detected by binaryization, and further in accordance with the subsequent steps determine the bright and dark state of the light fixture corresponding to the target block of the reference image.

[0128] Comparison, the ratio of the number of the first number and all of the pixel pairs is less than or equal to the preset second ratio threshold, and the corresponding scene corresponding to the two images is different, that is, the high-altitude wide angle camera is acquired. The angle of the image is different, and moves after the reference image is acquired. In this case, the detection image is not a binarization process, that is, discarding the image to be detected, further re-re-acquire the image.

[0129]In this embodiment, after each time the image to be detected is obtained, it is determined whether the image to be detected is the same as the scene corresponding to the reference image, that is, if the high-altitude wide-angle camera is moved, if the scene is different, if the scene is the same, it is further determined the brightness of the lamp. The state, thereby avoiding the problem of improving the detection result of the high-altitude wide-angle camera movement, and improves the detection accuracy of the lamp target frame and the definite precision of the dimming state of the lamp.

[0130] According to a second aspect of the embodiment of the present invention, a luminaire detecting device is provided. Please refer to the attachment Figure 5 It shows the structure of the device, including:

[0131] The acquisition module 501 is used to obtain an image to be detected, wherein the scene corresponding to the image is detected includes a lamp;

[0132] The processing module 502 is configured to detect the image to be detected, to obtain a binarized image, and determine the lamp candidate frame based on the binarized image;

[0133] Determine module 503 for identifying a candidate area corresponding to the lamp candidate box in the to-detect image, determining the lamp target box of the image to be detected, wherein the light fixture is located in the position of the lamp. .

[0134] In one example, the image to be detected is a frame image to be detected, and the scene to be detected includes a luminaire;

[0135] The apparatus also includes a judgment module for:

[0136] According to the light fixture frame of the image to be detected, it is determined that the lighting state of the light fixture corresponding to the light fixture of the reference image, wherein the reference image is in the detected video, the A frame image before the image is detected.

[0137] In one example, the judgment module is specifically used for:

[0138] Determine each of the light fixtures of each of the light fixtures of the reference image and each of the lamp target frames to be detected;

[0139] At the intersecting ratio of the lamp target frame of the reference image, each of the lamp target frames to be detected, it is less than the preset first proportional threshold, determines the brightness of the light fixture corresponding to the light fixture of the reference image. The state is dark;

[0140] In the case where the lamp target frame of the reference image is equal to or equal to or equal to the first proportional threshold, it is determined corresponding to the first proportional threshold. The bright and dark state of the lamp is bright.

[0141] In one example, the judgment module is specifically used for:

[0142] It is determined that the pixel value average of all the pixels in the light fixture frame of the reference image is a first average value, and determines the pixel value average of all pixels in the light fixture in the image to be detected. For the second average;

[0143] In the case where the difference between the first average is greater than the preset first pixel threshold, it is determined that the bright and dark state of the landscape lamp corresponding to the light fixture frame is dark;

[0144] In the case where the difference between the first average is less than or equal to the preset first pixel threshold, it is bright to determine the bright and dark state of the landscape lamp corresponding to the light fixture.

[0145] In one example, the processing module is specifically used for:

[0146] Determine the pixel pair of pixel pairs to be detected and each pixel pair between the reference images, wherein a pixel point in the pixel pair is in the position of the image to be detected, and another pixel point is in the The location in the reference image is the same;

[0147] In all pixel pairs, the first number of pixel pairs greater than the preset second pixel threshold is determined;

[0148] In the case where the ratio of the first number is greater than the predetermined ratio threshold value, the to be detected is borid to be detected.

[0149] In one example, the processing module is also used in:

[0150] The to detect image is converted to a grayscale image before the detected image is borid.

[0151] In one example, the processing module is also used in:

[0152] The binarized image performs image corrosion processing based on the dicing candidate box based on the binarized image.

[0153] In one example, the processing module is used to determine the lamp candidate frame based on the binarized image, specifically for:

[0154] Determining the binarized image, a communication area consisting of bright pixels is the lighting candidate frame.

[0155] In one example, the processing module is also used in:

[0156] Based on the binarized image to determine the luminated image, the communication area is less than the preset area threshold.

[0157] In one example, the determination module is specifically used for:

[0158] Each of the candidate zone is input to a pre-completed classification neural network, the classified neural network outputs a classification result of each of the candidate zones, wherein the classification results include a lamp and a non-lamp;

[0159] Determine the candidate area of ​​the luminaire as the lighting target box.

[0160] In one example, the acquisition module is specifically used in:

[0161] A frame image to be detected is determined in the time interval of the reference image to be detected in the video to be detected.

[0162] In one example, a cache module is also included for:

[0163] Remove the reference image, saving the image to be detected as a reference image, and saves the light fixture frame to be detected image.

[0164] Regarding the apparatus in the above embodiment, the specific manner of each module performs a detailed description of the embodiments of the method in the first aspect, which will not be described in detail herein.

[0165] In a third aspect, at least one embodiment of the present invention provides a device, please refer to the attachment Image 6 It shows the structure of the device, the apparatus including a memory, a processor, which is configured to store computer instructions that can be run on the processor, the processor for executing the computer instruction based on the first The method described in terms of any of the methods is detected.

[0166] In a fourth aspect, at least one embodiment of the present invention provides a computer readable storage medium that stores a computer program that implements the method of any one of the first aspects when executed by the processor.

[0167] In the present invention, the term "first", "second" is used only for the purpose, and cannot be understood as an indication or implies relative importance. The term "multiple" refers to two or more unless otherwise clear.

[0168] Other embodiments of the invention will be readily apparent to those skilled in the art. The present invention is intended to cover any variations, uses, or adaptive changes of the invention, and these variations, applications, or adaptive changes, follow the general principles of the invention, and include known common sense or customary techniques in the art of the present invention. . The instructions and examples are considered only as exemplary, and the true scope and spirit of the invention are pointed out by the following claims.

[0169] It should be understood that the present invention is not limited to the exact structure described above and shown in the drawings, and various modifications and changes can be made without departing from their extent. The scope of the invention is limited only by the appended claims.