Method and device for determining abnormal state of signal lamp, storage medium and electronic device

Abstract

The embodiment of the invention provides a signal lamp abnormal state determination method and device, a storage medium and an electronic device.The method comprises the steps that under the condition that camera shooting parameters of camera shooting equipment meet preset conditions, a detection frame image collected by the camera shooting equipment is obtained; determining first signal lamp brightness of a target signal lamp of a target color included in the detection frame image; adjusting the first signal lamp brightness based on camera shooting parameters and equipment parameters of camera shooting equipment when the detection frame image is collected to obtain second signal lamp brightness; and determining an abnormal state of the target signal lamp based on the second signal lamp brightness. According to the invention, the technical problem of poor accuracy of determining the abnormal state of the signal lamp in the prior art is solved, and the effect of improving the accuracy of determining the abnormal state of the signal lamp is achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of communications, and in particular, to a method, device, storage medium, and electronic device for determining an abnormal state of a signal light. Background Art

[0002] An important basis for monitoring and capturing is the color state of the signal light. After a long period of use, the brightness of the signal light will gradually dim, affecting the recognition of the video signal light and thus affecting the recognition effect of the capture. In related technologies, two preset cameras with fixed parameters are usually used to detect the corresponding brightness and compare them, but the detection environment needs to be stable and the adaptability is poor.

[0003] It can be seen from this that the related art has a technical problem of poor accuracy in determining the abnormal state of a traffic light.

[0004] In view of the above problems existing in the related technologies, no effective solutions have been proposed yet. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, storage medium and electronic device for determining an abnormal state of a signal light, so as to at least solve the problem of poor accuracy in determining an abnormal state of a signal light existing in the related art.

[0006] According to one embodiment of the present invention, a method for determining an abnormal state of a traffic light is provided, comprising: obtaining a detection frame image captured by a camera device when a camera parameter of the camera device satisfies a predetermined condition; determining a first signal light brightness of a target signal light of a target color included in the detection frame image; adjusting the first signal light brightness based on the camera parameters of the camera device and device parameters when capturing the detection frame image to obtain a second signal light brightness; and determining the abnormal state of the target signal light based on the second signal light brightness.

[0007] In an exemplary embodiment, adjusting the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the brightness of the second signal light includes: determining the exposure time, gain parameter and aperture value included in the camera parameters; determining the transmittance included in the device parameters; adjusting the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value and the transmittance to obtain the brightness of the second signal light.

[0008] In an exemplary embodiment, adjusting the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value, and the transmittance to obtain the brightness of the second signal light includes: determining a first ratio of the gain parameter to a first constant; determining the first ratio power of a second constant as a first value; determining the square value of the aperture value as a second value; determining a first product of the exposure time and the first value, the second value, and the transmittance; and determining a second ratio of the brightness of the first signal light to the first product as the brightness of the second signal light.

[0009] In an exemplary embodiment, after adjusting the first signal light brightness based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the second signal light brightness, the method also includes: determining a first minimum signal light brightness and a first maximum signal light brightness acquired by the camera device under preset parameters; adjusting the first minimum signal light brightness based on the preset parameters to obtain a second minimum signal light brightness; adjusting the first maximum signal light brightness based on the preset parameters to obtain a second maximum signal light brightness; normalizing the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain a third signal light brightness; and updating the second signal light brightness to the third signal light brightness.

[0010] In an exemplary embodiment, normalizing the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain the third signal light brightness includes: when the second signal light brightness is less than the second minimum signal light brightness, determining a first set value as the third signal light brightness; when the second signal light brightness is greater than the second maximum signal light brightness, determining a second set value as the third signal light brightness; when the second signal light brightness is greater than or equal to the second minimum signal light brightness and the second signal light brightness is less than or equal to the second maximum signal light brightness, determining a first difference between the second signal light brightness and the second minimum signal light brightness, determining a second difference between the second maximum signal light brightness and the second minimum signal light brightness, determining a third ratio of the first difference to the second difference, determining a third difference between the second set value and the first set value, determining a second product of the third ratio and the third difference, and determining the sum of the second product and the first set value as the third signal light brightness, wherein the second set value is greater than the first set value.

[0011] In an exemplary embodiment, determining a first signal light brightness of a target signal light of a target color included in the detection frame image comprises: determining a signal light image included in the detection frame image; determining pixel points included in the signal light image and belonging to the target signal light; and determining an average brightness of the pixel points as the first signal light brightness.

[0012] In an exemplary embodiment, before acquiring the detection frame image captured by the camera device, the method also includes: acquiring the target image captured by the camera device; determining the average brightness value of the signal light of a predetermined color included in the target image; when the average brightness value is within a preset brightness range, determining that the camera parameters satisfy the predetermined conditions; when the average brightness value is less than the minimum value of the preset brightness range, increasing the exposure time included in the camera parameters; when the average brightness value is greater than the maximum value of the preset brightness range, reducing the exposure time included in the camera parameters.

[0013] In an exemplary embodiment, the predetermined color is red.

[0014] In an exemplary embodiment, the abnormal state of the target signal light is determined based on the brightness of the second signal light: when the brightness of the second signal light is less than a brightness threshold, the target signal light is determined to have abnormal brightness in the detection frame image; a target number of times the target signal light has abnormal brightness in the detection frame and other detection frame images is determined, wherein the other detection frame images are images captured after the detection frame image; when the target number is greater than a preset number, the abnormal state of the target signal light is determined to be abnormal signal light brightness.

[0015] According to another embodiment of the present invention, there is provided a device for determining an abnormal state of a traffic light, comprising: an acquisition module, for acquiring a detection frame image captured by a camera device when a camera parameter of the camera device satisfies a predetermined condition; a first determination module, for determining a first signal light brightness of a target signal light of a target color included in the detection frame image; an acquisition module, for adjusting the first signal light brightness based on the camera parameters of the camera device and device parameters when capturing the detection frame image to obtain a second signal light brightness; and a second determination module, for determining the abnormal state of the target signal light based on the second signal light brightness.

[0016] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0017] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0018] According to yet another embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0019] Through the present invention, the camera parameters of the camera device can be adjusted, and the camera device can be used to collect detection frame images under the condition that the camera parameters meet predetermined conditions. Determine the first signal brightness of the target signal light of the target color in the detection frame image collected by the camera device, and record the corresponding camera parameters of the camera device. Then, the second signal light brightness after removing environmental interference can be determined through the first signal brightness and the camera parameters of the camera device. Based on the second signal light brightness, the abnormal state of the target signal light can be determined. Since the camera parameters of the camera device are comprehensively considered, environmental interference can be eliminated. In addition, a separate detection frame is set to be responsible for the signal light effect, which can make the statistical signal light brightness more accurate. Therefore, the technical problem of poor accuracy in determining the abnormal state of the signal light in the related art can be solved, thereby achieving the effect of improving the accuracy of determining the abnormal state of the signal light. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for determining an abnormal state of a signal light according to an embodiment of the present invention;

[0021] Figure 2 is a flow chart of a method for determining an abnormal state of a signal light according to an embodiment of the present invention;

[0022] Figure 3 is a signal light area according to an embodiment of the present invention;

[0023] Figure 4 is a flow chart of an image segmentation process according to an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of a process of adjusting and counting the brightness of a raw signal light according to an embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of a process for determining insufficient brightness of a target signal light according to an embodiment of the present invention;

[0026] Figure 7 is a flow chart of a method for determining an abnormal state of a signal light according to a specific embodiment of the present invention;

[0027] Figure 8 4 is a structural block diagram of a device for determining an abnormal state of a signal light according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0030] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for determining an abnormal state of a signal light according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the abnormal state of a signal light in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] In this embodiment, a method for determining an abnormal state of a signal light is provided. Figure 2 is a flow chart of a method for determining an abnormal state of a signal light according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0034] Step S202, when the camera parameters of the camera device meet the predetermined conditions, obtaining the detection frame image captured by the camera device;

[0035] Step S204, determining the brightness of a first signal light of a target signal light of a target color included in the detection frame image;

[0036] Step S206, adjusting the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image, to obtain the brightness of the second signal light;

[0037] Step S208: determining the abnormal state of the target signal light based on the brightness of the second signal light.

[0038] In the above embodiment, all frames captured by the camera device can be divided into video frames and detection frames. The video frames can be used to monitor and capture illegal behaviors, and the detection frames can be responsible for detecting the effect of traffic lights. Therefore, the signal lights in the detection frames can be mainly used to detect insufficient brightness. Among them, the camera device can be any device with shooting capabilities, such as a camera and a webcam. For the detection frame, the camera parameters of the camera device that collects the detection frame can be adjusted according to the brightness of the signal light in the raw (RAW Domain) domain, which can ensure that the brightness of the signal light on the raw is within a reasonable range, that is, most of the pixels in the signal light are not overexposed or too dark. Among them, the detection frame image can be a raw data image, that is, raw domain data, and the raw domain can be understood as the raw data output from the sensor without any processing.

[0039] In the above embodiment, it is possible to determine whether the camera parameters of the camera device meet the predetermined conditions based on the image captured by the camera device before the detection frame image is captured. If the predetermined conditions are not met, the camera parameters of the camera device are adjusted until the camera parameters of the camera device meet the predetermined conditions. Wherein, determining whether the camera parameters of the camera device meet the predetermined conditions may include: acquiring N frames of images captured by the camera device before the detection frame image is captured, determining the average brightness of the signal light in each frame of the N frames, and determining that the camera parameters of the camera device meet the predetermined conditions when the average brightness is within the set range, and determining that the camera parameters of the camera device do not meet the predetermined conditions when the average brightness is outside the set area.

[0040] In the above embodiment, before determining the target traffic light of the target color, the target traffic light image can be determined in the detection frame image. For example, a rectangular frame input through the target interface can be received, and the image corresponding to the rectangular frame area can be determined as the traffic light area. After determining the traffic light area, the foreground area and the background area can also be determined in the traffic light area, wherein the foreground area can be the luminous area, and the background area can be the lamp panel area of ​​the traffic light. The schematic diagram of the traffic light area can be seen in Figure 3 , Figure 3 is a schematic diagram of a signal light area according to an embodiment of the present invention. Figure 3 As shown in the figure, the larger rectangular box is the signal light area that can be defined by the user. The signal light recognition module will recognize this area and output the color status of the signal lights in the signal light group and the circumscribed rectangular area of ​​the signal lights. In the circumscribed rectangular area of ​​the signal lights, the foreground points of the signal lights can be segmented. Since the rectangular box contains the signal lights and the light panel area, if the histogram distribution is used, the result is basically in the shape of two peaks. Considering the related issues of real-time detection and time-consuming for the brightness statistics of the signal lights, the global threshold segmentation method can be selected to achieve it.

[0041] In the above embodiment, Figure 4 is a flow chart of an image segmentation process according to an embodiment of the present invention. Figure 4As shown, after obtaining the coordinates of the light frame of the target signal light, the grayscale mean Me of the pixels in the frame can be calculated, and the grayscale mean Me can be set as the initial segmentation threshold T, that is, T=Me. Initialize the number of iterations count to 0, and determine the relationship between the current number of iterations and M. If it is less than M, enter the next iteration. At this time, the number of iterations count is increased by 1. If it is not less than M, the process ends. Among them, M can be understood as the number of light groups. Based on the initial segmentation threshold T, the pixels in the frame can be divided into foreground points and background points, and the grayscale mean Me1 of all foreground points in the frame and the grayscale mean Me2 of all background points in the frame are calculated respectively. The new segmentation threshold Tnext can be calculated by Me1 and Me2: Tnext=0.5*(Me1+Me2). Among them, foreground points can be understood as pixels greater than or equal to the initial segmentation threshold T, and background points can be understood as pixels less than the initial segmentation threshold T. The new segmentation threshold Tnext is compared with the initial segmentation threshold T. If the difference between the two is less than or equal to Tc, the final segmentation threshold T=Tnext, and the iteration process ends; if the difference between the two is greater than Tc, the next iteration is entered. Tc can be 1, 2, 5, but not limited to this.

[0042] In the above embodiment, when it is determined that the parameters of the camera device meet the predetermined conditions, the detection frame image captured by the camera device at this time can be obtained, and the brightness of the first signal light of the target color light in the target signal light can be determined in the detection frame image. The brightness of the second signal light can be adjusted according to the brightness of the first signal light and the camera parameters of the camera device when collecting the detection frame image, and the abnormal state of the target signal light can be determined by the brightness of the second signal light obtained after adjustment. Among them, the target color can be yellow, red, and green.

[0043] Through the present invention, the camera parameters of the camera device can be adjusted, and the camera device can be used to collect detection frame images under the condition that the camera parameters meet predetermined conditions. Determine the first signal brightness of the target signal light of the target color in the detection frame image collected by the camera device, and record the corresponding camera parameters of the camera device. Then, the second signal light brightness after removing environmental interference can be determined through the first signal brightness and the camera parameters of the camera device. Based on the second signal light brightness, the abnormal state of the target signal light can be determined. Since the camera parameters of the camera device are comprehensively considered, environmental interference can be eliminated. In addition, a separate detection frame is set to be responsible for the signal light effect, which can make the statistical signal light brightness more accurate. Therefore, the technical problem of poor accuracy in determining the abnormal state of the signal light in the related art can be solved, thereby achieving the effect of improving the accuracy of determining the abnormal state of the signal light.

[0044] Optionally, the executor of the above steps may be a background processor, or other devices with similar processing capabilities, or a machine that integrates at least an image acquisition device and a data processing device, wherein the image acquisition device may include a graphics acquisition module such as a camera, and the data processing device may include a computer, a mobile phone or other terminal, but is not limited thereto.

[0045] In an exemplary embodiment, adjusting the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the brightness of the second signal light includes: determining the exposure time, gain parameter and aperture value included in the camera parameters; determining the transmittance included in the device parameters; adjusting the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value and the transmittance to obtain the brightness of the second signal light.

[0046] In the above embodiment, the camera parameters of the camera device may include exposure time, gain parameter, aperture value, and the device parameters may include transmittance corresponding to the filter type. Among them, the exposure time can be understood as the shutter time corresponding to the exposure of the camera device; the gain parameter can be understood as the process of increasing the image brightness by electronically amplifying the signal; the aperture value can be understood as the size of the opening of the camera lens to control the amount of light entering the camera device, which can affect the amount of light entering; the filter type can be a MICR micro filter, a polarizer, a neutral gray filter, etc., and different types of filters have different transmittances. According to the determined exposure time, gain parameter, aperture value and transmittance of the camera device, the brightness of the first signal light can be adjusted to obtain the brightness of the second signal light.

[0047] In an exemplary embodiment, adjusting the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value, and the transmittance to obtain the brightness of the second signal light includes: determining a first ratio of the gain parameter to a first constant; determining the first ratio power of a second constant as a first value; determining the square value of the aperture value as a second value; determining a first product of the exposure time and the first value, the second value, and the transmittance; and determining a second ratio of the brightness of the first signal light to the first product as the brightness of the second signal light.

[0048] In the above embodiment, taking the transmittance of the filter Micr as Kicr, the exposure time as cur_shut (in ms), the gain parameter as cur_gain (in db), and the aperture value Viris as an example, the brightness Y2 of the second signal light can be calculated by the following formula: , where Y1 can be understood as the brightness of the first signal light, That is, the first ratio of the above gain parameter to the first constant, That is, the first value determined by the first ratio power of the second constant. That is, the second value determined by the square value of the aperture value.

[0049] In the above embodiment, the brightness of the second signal light can also be calculated through a functional relationship: when the aperture value is fixed, the brightness of the second signal light is inversely proportional to the conversion multiple of the exposure gain and the transmittance of the filter; when the aperture value and the transmittance of the filter are fixed, the brightness of the second signal light is inversely proportional to the conversion multiple of the exposure gain. In addition, the brightness of the second signal light can also be calculated through a fitting relationship that is inversely proportional to the exposure time, exposure gain, the square of the aperture value (corresponding to the incoming light), and the transmittance of the filter.

[0050] In an exemplary embodiment, after adjusting the first signal light brightness based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the second signal light brightness, the method also includes: determining a first minimum signal light brightness and a first maximum signal light brightness acquired by the camera device under preset parameters; adjusting the first minimum signal light brightness based on the preset parameters to obtain a second minimum signal light brightness; adjusting the first maximum signal light brightness based on the preset parameters to obtain a second maximum signal light brightness; normalizing the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain a third signal light brightness; and updating the second signal light brightness to the third signal light brightness.

[0051] In the above embodiment, the preset parameters of the camera device can be understood as the preset exposure time cur_shuts (in ms), gain parameter cur_gains (in db), aperture value Vs, filter type Ms (transmittance Ks). The maximum value Y of the signal light brightness in the detection frame collected by the camera device under the above preset parameters can be determined. 1smax (i.e. the brightness of the first maximum signal light mentioned above) and the minimum value Y 1smin (i.e., the first minimum signal light brightness mentioned above), the first maximum signal light brightness and the first minimum signal light brightness can be adjusted according to the preset parameters to obtain the second minimum signal light brightness Y 2smin and the second maximum signal light brightness Y 2smax , where Y 2smin and Y 2smax It can be calculated by the following formula: . Take advantage of Y 2smin and Y 2smaxThe second signal light brightness Y2 is normalized to a specific range to obtain the third signal light brightness Y3, and the second signal light brightness is updated to the third signal light brightness. The normalization process can be understood as a process of converting a dimensional expression into a dimensionless expression through mathematical transformation, that is, to simplify calculations and eliminate differences between different dimensional quantities.

[0052] In an exemplary embodiment, normalizing the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain the third signal light brightness includes: when the second signal light brightness is less than the second minimum signal light brightness, determining a first set value as the third signal light brightness; when the second signal light brightness is greater than the second maximum signal light brightness, determining a second set value as the third signal light brightness; when the second signal light brightness is greater than or equal to the second minimum signal light brightness and the second signal light brightness is less than or equal to the second maximum signal light brightness, determining a first difference between the second signal light brightness and the second minimum signal light brightness, determining a second difference between the second maximum signal light brightness and the second minimum signal light brightness, determining a third ratio of the first difference to the second difference, determining a third difference between the second set value and the first set value, determining a second product of the third ratio and the third difference, and determining the sum of the second product and the first set value as the third signal light brightness, wherein the second set value is greater than the first set value.

[0053] In the above embodiment, the brightness Y2 of the second signal light can be normalized to the first set value Y within a specific range. 3min To the second set value Y 3max Inside: When the second signal light brightness Y2 is less than the second minimum signal light brightness Y 2smin In the case of 3min When the second signal light brightness Y2 is greater than the second maximum signal light brightness Y 2smax In the case of 3max When the second signal light brightness Y2 is at the second minimum signal light brightness Y 2smin With the second maximum signal light brightness Y 2smax In the case of a range between , it can be determined by linear mapping, that is, the brightness of the third signal light Y3 can be calculated by the following formula: Y3= ,in, It can be understood as the first difference between the brightness of the second signal light and the brightness of the second minimum signal light. It can be understood as the second difference between the second maximum signal light brightness and the second minimum signal light brightness. It can be understood as the third ratio of the first difference to the second difference, It can be understood as the third difference between the second set value and the first set value.

[0054] In an exemplary embodiment, determining a first signal light brightness of a target signal light of a target color included in the detection frame image comprises: determining a signal light image included in the detection frame image; determining pixel points included in the signal light image and belonging to the target signal light; and determining an average brightness of the pixel points as the first signal light brightness.

[0055] In the above embodiment, when the camera parameters of the camera device meet the predetermined conditions, the foreground points in each target signal light pixel can be screened out in combination with the color information of the signal light recognition module and the signal light coordinate information, and the average brightness of the signal light can be calculated. Since the bits of different sensors are inconsistent, the above average brightness can be unified to a specific bit, such as 8bit, 16bit, etc., and this average brightness can be determined as the first signal light brightness Y1, and the camera parameters of the camera device at this time are recorded at the same time.

[0056] In the above embodiment, a traffic light image can be identified from a detection frame image by machine learning, and the traffic light image is the traffic light area in the detection frame image. The traffic light area can also be determined by receiving a rectangular frame input through a target interface. For example, an image at the position where the rectangular frame is located in the detection frame image is determined as a traffic light image. After determining the traffic light image, the foreground area and the background area in the traffic light image can be determined, and the pixels in the foreground area are the pixels of the target traffic light.

[0057] In the above embodiment, the signal light image may include images of multiple signal light groups. When the signal light image includes images of multiple signal light groups, the average brightness of the target signal lights in the multiple signal light groups is determined as the first signal light brightness. For example, the signal light image includes three signal light groups, the first signal light group includes red light 1, green light 1, and yellow light 1, the second signal light group includes red light 2, green light 2, and yellow light 2, and the third signal light group includes red light 3, green light 3, and yellow light 3. When the target signal light is a red light, the first signal light brightness is the average brightness of red light 1, red light 2, and red light 3. When the signal light image includes only one signal light group, such as only red light 1, green light 1, and yellow light 1 included in the first signal light group, when the target signal light is a red light, the first signal light brightness is the brightness of red light 1. When determining the brightness of each signal light, the average brightness of all pixels of the signal light may be determined, and the average brightness of all pixels is determined as the signal light brightness.

[0058] In an exemplary embodiment, before acquiring the detection frame image captured by the camera device, the method also includes: acquiring the target image captured by the camera device; determining the average brightness value of the signal light of a predetermined color included in the target image; when the average brightness value is within a preset brightness range, determining that the camera parameters satisfy the predetermined conditions; when the average brightness value is less than the minimum value of the preset brightness range, increasing the exposure time included in the camera parameters; when the average brightness value is greater than the maximum value of the preset brightness range, reducing the exposure time included in the camera parameters.

[0059] In the above embodiment, before the camera device collects the detection frame image, the camera parameters of the camera device can be adjusted for different environments: the brightness average value of the signal light included in the target image collected by the camera device in this environment can be compared with the preset brightness range, and the exposure time of the camera device can be adjusted to achieve that the brightness of the signal light in the captured image can be within the preset brightness range, so as to ensure that the brightness average value of the signal light collected by the camera device in this environment is moderate. Since the camera device can adaptively adjust the camera parameters for different shooting environments, it is ensured that the camera device can better adapt to various lighting conditions, whether it is a sunny day with high brightness or a cloudy day with low illumination, the brightness exposure of the signal light captured by the camera device can be moderate. In addition, through adaptive adjustment, the camera device can better handle the scene of the dynamic range, retain the details of the bright and dark parts, obtain a clearer image, and improve the shooting efficiency, that is, in a rapidly changing environment, it can quickly adapt to the change of light. Thereby improving the environmental adaptability of the method for determining the abnormal state of the signal light, solving the problem of poor environmental adaptability caused by only being able to collect images according to fixed parameters in the related art.

[0060] In the above embodiment, see Figure 3 , in the traffic light frame, the brightness of the predetermined color can be counted according to the segmented foreground points, and then the average brightness of the predetermined color can be obtained. Among them, the predetermined colors can be red, yellow, and green. If the average brightness value is within the preset brightness range, it is determined that the camera parameters meet the predetermined conditions, that is, the brightness of the traffic light is within a reasonable range, and most of the pixels in the traffic light are not overexposed or too dark; if the average brightness value is less than the minimum value of the preset brightness range, the exposure time of the camera device is increased; if the average brightness value is greater than the maximum value of the preset brightness range, the exposure time of the camera device is reduced. Among them, to determine whether the camera parameters meet the predetermined conditions, please refer to Figure 5 , Figure 5 is a process diagram of raw signal light brightness adjustment and statistics according to an embodiment of the present invention, the process comprising:

[0061] Step S502, start;

[0062] Step S504, determining whether the current frame is a detection frame, if yes, executing step S506, if no, executing step S524;

[0063] Step S506, obtaining the signal light recognition module result: signal light frame coordinates + color information;

[0064] Step S508, determining whether there is a traffic light currently, if yes, executing step S510, if no, executing step S524;

[0065] Step S510, determining whether there is a red light at present, if yes, executing step S512, if no, executing step S524;

[0066] Step S512, the image is screened for foreground points;

[0067] Step S514, counting the average brightness of each red light;

[0068] Step S516, calculating the average brightness of all red lights;

[0069] Step S518, determining whether the average brightness of the red light is within a reasonable range, if yes, executing step S522, if no, executing step S520;

[0070] Step S520, adjusting detection frame camera parameters;

[0071] Step S522, screening the foreground points of each signal light and counting the brightness;

[0072] Step S524, end.

[0073] In the above embodiment, the results of the processing by the detection frame signal light recognition module can be obtained first, which may include the signal light frame coordinates and color information. Within the circumscribed rectangular area of ​​the signal light, the signal light foreground points can be segmented, and the brightness mean of all red lights can be counted and calculated. If it is within a reasonable range, the brightness of the signal light foreground points is filtered and counted. If the brightness mean is not within a reasonable range, the camera parameters of the detection frame camera device are adjusted.

[0074] In an exemplary embodiment, the predetermined color is red. In this embodiment, considering that red traffic lights are more easily overexposed than lights of the other two colors, the exposure parameters of the detection frame camera device can be adjusted according to the average brightness of red (i.e., the predetermined color) traffic lights to ensure that the average brightness is within a reasonable range and the brightness of most traffic lights is moderate. In the traffic light frame, the brightness of the red light can be counted according to the segmented foreground points, and then the average brightness of the red light can be obtained.

[0075] In an exemplary embodiment, the abnormal state of the target signal light is determined based on the brightness of the second signal light: when the brightness of the second signal light is less than a brightness threshold, the target signal light is determined to have abnormal brightness in the detection frame image; a target number of times the target signal light has abnormal brightness in the detection frame and other detection frame images is determined, wherein the other detection frame images are images captured after the detection frame image; when the target number is greater than a preset number, the abnormal state of the target signal light is determined to be abnormal signal light brightness.

[0076] In the above embodiment, the abnormal state of the target signal light may be insufficient brightness of the target signal light. By setting the upper layer open insufficient brightness threshold T and the number of detection frames N, the process of judging the abnormal brightness of the target signal light can be seen in Figure 6 As shown, Figure 6 is a schematic diagram of a process for determining insufficient brightness of a target signal light according to an embodiment of the present invention. Figure 6 As shown, first, the configuration information sent down by the upper layer can be received, which may include the number of light groups M, the threshold value of insufficient brightness T and the number of detection frames N. The number of insufficient brightness frames of the three target colors of red, yellow and green of the above M light groups is initialized, that is, nr[M]=ny[M]=ng[M]={0}, where nr[M], ny[M] and ng[M] can be understood as the number of red light group frames, yellow light group frames and green light group frames respectively. The coordinate information and color information of the signal light frame can be obtained through the results of the signal light recognition module. The detection starts from the light group with light group number m=0. Each time the detection is performed, it can be determined whether the light group number m is less than the number of light groups M. If so, it can be indicated that all light groups have not been fully detected, and the light group with the current number will continue to be processed. If m is equal to or greater than M, the detection data of the next frame will be processed.

[0077] In the above embodiment, the brightness of the signal light of the light group number m is detected. By judging the color of the signal light, the three target colors of red, yellow and green can be processed respectively. The brightness of the second signal light in the light group number m can be calculated to judge whether the brightness of the second signal light is lower than the brightness threshold T. If so, the number of insufficient brightness detection frames of the corresponding color light of the light group number m can be increased by 1 to determine that the brightness of the detected target signal light in the detection frame image is abnormal. If not, the number of insufficient brightness detection frames of the corresponding color light of the light group number m can be cleared to 0. It is judged whether the number of insufficient brightness detection frames (i.e. the above target number) is lower than the number of detection frames N (i.e. the above preset number), as shown in the formula, then the light group number is increased by 1, and the next light group is detected. If not, it means that the brightness of the corresponding color light of the current light group number is abnormal, which can be reported to the cloud platform for subsequent operations, and the brightness detection of the signal light of the next light group is also performed at the same time.

[0078] In the above embodiment, for example, the number of light groups M=10, and the number of detection frames N is 10. First, the brightness of the signal light of light group number m=0 can be detected. Since the number m is less than the number of light groups M, the current light group can be processed, and the brightness of the lights of red, yellow, and green colors can be detected respectively, that is, the brightness of the third signal light of light group m=0 under the three colors of red, yellow, and green can be determined respectively. If it is less than the insufficient brightness threshold T, the insufficient brightness detection frame number of the corresponding color of light group m is increased by 1. When the detected insufficient brightness detection frame number is less than N, it means that the brightness of the signal light of light group m=0 is normal, and the brightness of the signal light of light group m=1 can be continuously detected. If the detected insufficient brightness detection frame number is greater than N, it can be said that the brightness of the signal light of light group m0 is abnormal, and the signal light group of light group m=1 is processed while reporting. When the light group number m=10 is processed, the next detection frame can be returned for processing.

[0079] It should be noted that the predetermined conditions, preset parameters, first set values, second set values, predetermined colors, predetermined brightness ranges, preset times and other pre-set values ​​or parameters in the aforementioned embodiments can all be customized through the target interface, thereby improving the stability and environmental adaptability of the detection scheme.

[0080] The following is a description of the method for determining the abnormal state of a signal light in conjunction with a specific implementation method:

[0081] Figure 7 is a flow chart of a method for determining an abnormal state of a signal light according to a specific embodiment of the present invention. Figure 7 As shown, the process includes the following steps:

[0082] Step S702, all frames are divided into video frames and detection frames, the video frames are used for monitoring and capturing, and the detection frames are responsible for the signal light effect;

[0083] Step S704, adjusting the camera parameters of the detection frame based on the raw domain signal light brightness to ensure that the signal light brightness is within a reasonable range, and obtaining a first signal light brightness;

[0084] Step S706, calculating the brightness of the signal light after eliminating environmental interference, that is, the brightness of the second signal light, based on the exposure time of the current detection frame, the aperture size, the filter type, and the brightness of the first signal light;

[0085] Step S708, normalizing the brightness of the second signal light to a specific range to obtain the brightness of the third signal light;

[0086] Step S710: When the number of frames in which the brightness of the third signal light is continuously lower than the threshold exceeds a certain number, reporting a state of insufficient brightness of the signal light.

[0087] In the above embodiment, under different ambient brightness, factors such as exposure gain, aperture, and filter are comprehensively considered to calculate the brightness of the traffic light, eliminating the impact of environmental changes. In addition, the upper interface opens the interface for insufficient brightness threshold and detection frame number, which improves the stability and environmental adaptability of the detection scheme. Pulling out a separate detection frame to be responsible for the traffic light effect can ensure that the brightness of the traffic light is within a reasonable range and is not overexposed or too dark, which affects the brightness statistics of the traffic light. In addition, the brightness of the traffic light is counted by raw data, which is not affected by the ISP (Image Signal Processor), and the brightness statistics can be more accurate.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0089] In this embodiment, a device for determining an abnormal state of a signal light is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made are not repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0090] Figure 8 is a structural block diagram of a device for determining an abnormal state of a signal light according to an embodiment of the present invention. Figure 8 As shown, the device comprises:

[0091] The acquisition module 802 is used to acquire the detection frame image captured by the camera device when the camera parameters of the camera device meet the predetermined conditions;

[0092] A first determination module 804 is used to determine a first signal light brightness of a target signal light of a target color included in the detection frame image;

[0093] An adjustment module 806, configured to adjust the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image, so as to obtain the brightness of the second signal light;

[0094] The second determining module 808 is configured to determine the abnormal state of the target signal light based on the brightness of the second signal light.

[0095] In an exemplary embodiment, the adjustment module 806 adjusts the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the brightness of the second signal light: determines the exposure time, gain parameter and aperture value included in the camera parameters; determines the transmittance included in the device parameters; adjusts the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value and the transmittance to obtain the brightness of the second signal light.

[0096] In an exemplary embodiment, the adjustment module 806 can adjust the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value and the transmittance to obtain the brightness of the second signal light in the following manner: determine a first ratio of the gain parameter to a first constant; determine the first ratio power of the second constant as a first value; determine the square value of the aperture value as a second value; determine a first product of the exposure time and the first value, the second value and the transmittance; and determine a second ratio of the brightness of the first signal light to the first product as the brightness of the second signal light.

[0097] In an exemplary embodiment, the device can be used to adjust the first signal light brightness based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the second signal light brightness: determine a first minimum signal light brightness and a first maximum signal light brightness acquired by the camera device under preset parameters; adjust the first minimum signal light brightness based on the preset parameters to obtain the second minimum signal light brightness; adjust the first maximum signal light brightness based on the preset parameters to obtain the second maximum signal light brightness; normalize the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain a third signal light brightness; and update the second signal light brightness to the third signal light brightness.

[0098] In an exemplary embodiment, the device can normalize the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain the third signal light brightness in the following manner: when the second signal light brightness is less than the second minimum signal light brightness, determine the first set value as the third signal light brightness; when the second signal light brightness is greater than the second maximum signal light brightness, determine the second set value as the third signal light brightness; when the second signal light brightness is greater than or equal to the second minimum signal light brightness and the second signal light brightness is less than or equal to the second maximum signal light brightness, determine a first difference between the second signal light brightness and the second minimum signal light brightness, determine a second difference between the second maximum signal light brightness and the second minimum signal light brightness, determine a third ratio of the first difference to the second difference, determine a third difference between the second set value and the first set value, determine a second product of the third ratio and the third difference, and determine the sum of the second product and the first set value as the third signal light brightness, wherein the second set value is greater than the first set value.

[0099] In an exemplary embodiment, the first determination module 804 can determine the first signal light brightness of the target signal light of the target color included in the detection frame image in the following manner: determine the signal light image included in the detection frame image; determine the pixel points included in the signal light image and belonging to the target signal light; and determine the average brightness of the pixel points as the first signal light brightness.

[0100] In an exemplary embodiment, the device can be used to: acquire a target image captured by the camera device before acquiring a detection frame image captured by the camera device; determine the average brightness value of a signal light of a predetermined color included in the target image; determine that the camera parameters satisfy the predetermined conditions when the average brightness value is within a preset brightness range; increase the exposure time included in the camera parameters when the average brightness value is less than the minimum value of the preset brightness range; and reduce the exposure time included in the camera parameters when the average brightness value is greater than the maximum value of the preset brightness range.

[0101] In an exemplary embodiment, the predetermined color that the apparatus is configured to determine is red.

[0102] In an exemplary embodiment, the second determination module 808 may determine the abnormal state of the target signal light based on the brightness of the second signal light in the following manner: when the brightness of the second signal light is less than a brightness threshold, determine that the brightness of the target signal light in the detection frame image is abnormal; determine a target number of times that the target signal light has abnormal brightness in the detection frame and other detection frame images, wherein the other detection frame images are images captured after the detection frame image; and when the target number is greater than a preset number, determine that the abnormal state of the target signal light is abnormal signal light brightness.

[0103] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0104] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0105] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0106] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0107] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method in each embodiment of the present application when the computer program is executed by a processor.

[0108] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0109] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining an abnormal state of a signal light, characterized in that: include: When the camera parameters of the camera device meet the predetermined conditions, acquiring the detection frame image captured by the camera device; Determine a first signal light brightness of a target signal light of a target color included in the detection frame image; adjusting the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image, to obtain the brightness of the second signal light; An abnormal state of the target signal light is determined based on the brightness of the second signal light.

2. The method according to claim 1, characterized in that The brightness of the first signal light is adjusted based on the camera parameters of the camera device and the device parameters when the detection frame image is acquired, and the brightness of the second signal light is obtained, comprising: Determining the exposure time, gain parameter and aperture value included in the camera parameters; Determining the transmittance included in the device parameters; The brightness of the first signal light is adjusted based on the exposure time, the gain parameter, the aperture value, and the transmittance to obtain the brightness of the second signal light.

3. The method according to claim 2, characterized in that Adjusting the brightness of the first signal light based on the exposure time, the gain parameter, the aperture value, and the transmittance to obtain the brightness of the second signal light includes: determining a first ratio of the gain parameter to a first constant; determining the first ratio power of a second constant as a first value; Determine the square value of the aperture value as a second value; Determine a first product of the exposure time, the first value, the second value, and the transmittance; A second ratio of the first signal light brightness to the first product is determined as the second signal light brightness.

4. The method according to claim 1, characterized in that: After adjusting the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image to obtain the brightness of the second signal light, the method further includes: Determine a first minimum signal light brightness and a first maximum signal light brightness captured by the camera device under preset parameters; Adjusting the first minimum signal light brightness based on the preset parameters to obtain a second minimum signal light brightness; Adjusting the first maximum signal light brightness based on the preset parameters to obtain a second maximum signal light brightness; Normalizing the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain a third signal light brightness; The second signal light brightness is updated to the third signal light brightness.

5. The method according to claim 4, characterized in that Normalizing the second signal light brightness based on the second minimum signal light brightness and the second maximum signal light brightness to obtain a third signal light brightness includes: When the brightness of the second signal light is less than the brightness of the second minimum signal light, determining the first set value as the brightness of the third signal light; When the second signal light brightness is greater than the second maximum signal light brightness, determining the second set value as the third signal light brightness; When the second signal light brightness is greater than or equal to the second minimum signal light brightness, and the second signal light brightness is less than or equal to the second maximum signal light brightness, determine a first difference between the second signal light brightness and the second minimum signal light brightness, determine a second difference between the second maximum signal light brightness and the second minimum signal light brightness, determine a third ratio of the first difference to the second difference, determine a third difference between the second set value and the first set value, determine a second product of the third ratio and the third difference, and determine the sum of the second product and the first set value as the third signal light brightness, wherein the second set value is greater than the first set value.

6. The method according to claim 1, characterized in that Determining the first signal light brightness of the target signal light of the target color included in the detection frame image includes: Determining a traffic light image included in the detection frame image; Determining the pixel points included in the traffic light image and belonging to the target traffic light; The average brightness of the pixels is determined as the brightness of the first signal light.

7. The method according to claim 1, characterized in that Before acquiring the detection frame image captured by the camera device, the method further includes: Acquire a target image captured by the camera device; Determining an average brightness of a signal light of a predetermined color included in the target image; When the brightness average value is within a preset brightness range, determining that the camera parameter meets the predetermined condition; When the average brightness value is less than the minimum value of the preset brightness range, increasing the exposure time included in the camera parameters; When the brightness average value is greater than the maximum value of the preset brightness interval, the exposure time included in the imaging parameters is reduced.

8. The method according to claim 7, characterized in that The predetermined color is red.

9. The method according to claim 1, characterized in that: Determining the abnormal state of the target signal light based on the brightness of the second signal light includes: When the brightness of the second signal light is less than a brightness threshold, determining that the brightness of the target signal light in the detection frame image is abnormal; Determine a target number of times that the target signal light has abnormal brightness in the detection frame and other detection frame images, wherein the other detection frame images are images acquired after the detection frame image; When the target number of times is greater than a preset number of times, it is determined that the abnormal state of the target signal light is abnormal brightness of the signal light.

10. A device for determining abnormal state of a signal light, characterized in that: include: An acquisition module, used for acquiring a detection frame image captured by the camera device when the camera parameters of the camera device meet a predetermined condition; A first determining module, configured to determine a first signal light brightness of a target signal light of a target color included in the detection frame image; An adjustment module, configured to adjust the brightness of the first signal light based on the camera parameters and device parameters of the camera device when acquiring the detection frame image, so as to obtain the brightness of the second signal light; The second determining module is configured to determine an abnormal state of the target signal light based on the brightness of the second signal light.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 9 when executed.

12. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images