A method and device for color restoration of an image
Through area recognition based on pixel saturation and brightness and HSV color space filtering processing, the colors of overexposed objects in the image are accurately restored, and the problems of discontinuity and accuracy of signal light color restoration in the prior art are solved, thereby achieving efficient video signal light Color restoration effect.
Patent Information
- Application Number
- CN201811517272.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-12-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2038-12-12
AI Technical Summary
The prior art is difficult to accurately restore the colors of overexposed objects (such as traffic lights) in images, resulting in difficulties in evidence collection and discontinuity of color.
By acquiring the image to be processed, the overexposure area is determined according to the saturation and brightness of the pixels, the binarization process and the connection area recognition are performed, the hue and saturation are adjusted to restore the color, and the filtering process is performed in combination with the HSV color space to improve the recognition accuracy.
Accurate color restoration of overexposed objects in the image is achieved, the difficulty in obtaining evidence caused by overexposure of signal lights is solved, and the continuity and accuracy of color restoration is improved.
Smart Images

Figure CN111311500B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to a method and device for color restoration of an image. Background Art
[0002] Image overexposure is a common phenomenon that may lead to a series of problems. For example, in a traffic video surveillance system, a surveillance camera working in electronic police mode can capture red light signals and illegal vehicles at the same time as evidence of the offender's violation. However, in the case of low background ambient light illumination (such as dusk or night scenes), when the surveillance camera takes pictures of the red light signal to collect evidence, the camera often needs to increase the shutter, gain and aperture size. If the gain, shutter or aperture is increased excessively, the captured traffic light will be overexposed (for example, the red light will turn yellow or white), resulting in the captured image cannot be used as evidence of the violation.
[0003] At present, there are several methods for achieving signal light color restoration, which can be divided into hardware methods and software methods. Hardware method: A camera with an ultra-wide dynamic range can be used to eliminate the color distortion of signal lights caused by strong light. This is because the camera with an ultra-wide dynamic range has a large dynamic range of brightness detection and can restore image details in high-contrast brightness scenes. However, on the one hand, ultra-wide dynamic cameras are relatively expensive; on the other hand, since the signal light area usually occupies a small proportion of the entire screen, the ultra-wide dynamic range camera has limited effect on color restoration of overexposed signal lights under low illumination. Software method 1: The signal light area can be identified based on the red, green, blue (RGB) (color) space of the image, and then the color of the signal light can be restored. However, the RGB color space cannot reflect the brightness information of the signal light well, which may cause the gradient phenomenon of the signal light area after color restoration to be discontinuous with the surrounding image. Software method 2: The signal light area in the image can be identified based on the deep learning method and color enhancement can be performed. However, the deep learning-based method has low accuracy in recognizing the outline of the signal light, so the color restoration area of the signal light is prone to be discontinuous with the surrounding image.
[0004] Therefore, there is an urgent need for a more accurate method for restoring the color of overexposed objects in an image (such as overexposed traffic lights). Summary of the invention
[0005] The embodiment of the present application provides an image processing method that can accurately restore the color of overexposed objects in an image (e.g., traffic lights). Furthermore, it can solve a series of problems caused by overexposure of objects in an image (e.g., the difficulty in obtaining evidence caused by overexposure of traffic lights in current electric police monitoring scenarios).
[0006] In a first aspect, an embodiment of the present application provides a method for color restoration of an image, comprising: obtaining a first image to be processed, the first image comprising an overexposed first target object; determining a first region of the first image according to the saturation and brightness of pixels of the first image, the saturation of pixels of the first region being lower than an average value of the saturation of pixels of the first image, and the brightness of pixels of the first region being higher than an average value of the brightness of pixels of the first image; the first region corresponds to a region where the first target object is (located); binarizing the first region to obtain a binary image corresponding to the first region; determining at least one connected region in the binary image whose area is greater than or equal to a first threshold; the contour of at least one connected region corresponds to the contour of the first target object; and restoring the color of at least one connected region.
[0007] Based on the method provided in the embodiment of the present application, after obtaining the first image to be processed, the first area of the first image can be determined according to the saturation and brightness of the pixels of the first image (the first area can be used as the area where the overexposed first target object is located). Then, the first area can be binarized to obtain a binary image corresponding to the first area; then, at least one connected area in the binary image whose area is greater than or equal to the first threshold is determined (the outline of at least one connected area corresponds to the outline of the first target object), and then the color of the at least one connected area is restored (that is, the color of the overexposed first target object is restored). It can be seen that the method provided in the embodiment of the present application can accurately restore the color of the overexposed object in the image (that is, the first target object, such as a traffic light). Furthermore, a series of problems caused by overexposure of objects in the image can be solved (for example, the difficulty in obtaining evidence caused by overexposure of traffic lights in the current electric police monitoring scenario).
[0008] In a possible implementation, before determining the first region of the first image according to the saturation and brightness of the first image, the method further includes: converting the first image from a first space to a hue saturation value (HSV) space, the first space being any one of a luminance chromaticity YUV space, an RGB space, or a hue saturation brightness hue saturation brightness (HSL) space. That is, if the first image to be processed is in a first space, the first space may be a YUV space, an RGB space, or an HSL space. At this time, in order to obtain the saturation component and the brightness component of the pixels of the first image, the first image needs to be converted from the first space to the HSV space.
[0009] In one possible implementation, when the first target object is a traffic light, restoring the color of at least one connected area includes: acquiring color information of the traffic light; when the traffic light is red, adjusting the hue of pixels of at least one connected area to a red range, and increasing and decreasing the saturation and brightness of pixels of at least one connected area respectively; when the traffic light is yellow, adjusting the hue of pixels of at least one connected area to a yellow range, and increasing and decreasing the saturation and brightness of pixels of at least one connected area respectively; when the traffic light is green, adjusting the hue of pixels of at least one connected area to a green range, and increasing and decreasing the saturation and brightness of pixels of at least one connected area respectively.
[0010] In an embodiment of the present application, increasing and decreasing the saturation and brightness of pixels of at least one connected area may be respectively increasing and decreasing the saturation and brightness of pixels of at least one connected area linearly, or may be respectively increasing and decreasing the saturation and brightness of pixels of at least one connected area nonlinearly.
[0011] In one possible implementation, when the first target object is a traffic light, restoring the color of at least one connected area includes: converting at least one connected area to RGB space; obtaining color information of the traffic light; when the traffic light is red, adjusting the red component of the pixels of at least one connected area to a first preset range, and reducing the blue and green components of the pixels of at least one connected area; when the traffic light is yellow, adjusting the red and green components of the pixels of at least one connected area to a second preset range, and reducing the blue component of the pixels of at least one connected area; when the traffic light is green, adjusting the green component of the pixels of at least one connected area to a third preset range, and reducing the red and blue components of the pixels of at least one connected area.
[0012] The reducing of the blue and green components of the pixels of the at least one connected region may be a linear reduction or a nonlinear reduction of the blue and green components of the pixels of the at least one connected region.
[0013] In a possible implementation, before determining the first region of the first image according to the saturation and brightness of the first image, the method further includes: filtering the saturation component and the brightness component of the pixels of the first image. In this way, the outline of the overexposed first object can be more stably and accurately identified.
[0014] In a possible implementation manner, acquiring the first image to be processed includes: taking the second area selected by the user on the second image as the first image to be processed.
[0015] In this way, compared with directly processing the second image, selecting the first image from the second image and processing only the first image can reduce the amount of calculation in subsequent steps on the one hand, and improve the recognition accuracy of the area where the first target object (for example, a traffic light) is located on the other hand.
[0016] In a second aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition unit, used to acquire a first image to be processed, the first image comprising an overexposed first target object; a determination unit, used to determine a first area of the first image according to the saturation and brightness of pixels of the first image, the saturation of pixels of the first area being lower than the average saturation of pixels of the first image, and the brightness of pixels of the first area being higher than the average brightness of pixels of the first image; the first area corresponds to a first target object area; a processing unit, used to binarize the first area to obtain a binary image corresponding to the first area; the determination unit, further used to determine at least one connected area in the binary image whose area is greater than or equal to a first threshold; the contour of the first target object corresponds to the contour of the at least one connected area; the processing unit, further used to restore the color of the at least one connected area.
[0017] In a possible implementation manner, the processing unit is further used to: convert the first image from a first space to an HSV space, where the first space is any one of a YUV space, an RGB space, or an HSL space.
[0018] In one possible implementation, when the first target object is a traffic light, the processing unit is used to: obtain color information of the traffic light through an acquisition unit; when the traffic light is red, adjust the hue of pixels of at least one connected area to a red range, and increase and decrease the saturation and brightness of pixels of at least one connected area respectively; when the traffic light is yellow, adjust the hue of pixels of at least one connected area to a yellow range, and increase and decrease the saturation and brightness of pixels of at least one connected area respectively; when the traffic light is green, adjust the hue of pixels of at least one connected area to a green range, and increase and decrease the saturation and brightness of pixels of at least one connected area respectively.
[0019] In one possible implementation, when the first target object is a traffic light, the processing unit is used to: convert at least one connected area into an RGB space; obtain color information of the traffic light through an acquisition unit; when the traffic light is red, adjust the red component of the pixels of at least one connected area to a first preset range, and reduce the blue and green components of the pixels of at least one connected area; when the traffic light is yellow, adjust the red and green components of the pixels of at least one connected area to a second preset range, and reduce the blue component of the pixels of at least one connected area; when the traffic light is green, adjust the green component of the pixels of at least one connected area to a third preset range, and reduce the red and blue components of the pixels of at least one connected area.
[0020] In a possible implementation manner, the processing unit is further configured to: perform filtering processing on a saturation component and a brightness component of pixels of the first image.
[0021] In a possible implementation manner, the acquisition unit is configured to: use the second area selected by the user on the second image as the first image to be processed.
[0022] The technical effects of the second aspect and its various possible implementations can refer to the technical effects of the first aspect and its various possible implementations, and will not be repeated here.
[0023] On the third aspect, an embodiment of the present application provides a device, which exists in the product form of a chip. The structure of the device includes a processor and a memory. The memory is used to couple with the processor to store the necessary program instructions and data for the device. The processor is used to execute the program instructions stored in the memory, so that the device performs the functions of the image processing device in the above method.
[0024] In a fourth aspect, an embodiment of the present application provides an image processing device, which can implement the functions performed by the above-mentioned image processing device, and the functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.
[0025] In one possible design, the structure of the image processing device includes a processor and a communication interface, and the processor is configured to support the image processing device to perform the corresponding functions in the above method. The communication interface is used to support communication between the image processing device and other devices (such as a signal light detector on a traffic light). The image processing device may also include a memory, which is used to couple with the processor and store the necessary program instructions and data of the image processing device.
[0026] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute any one of the methods provided in the first aspect.
[0027] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any one of the methods provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 A schematic diagram of a system architecture of a method for color restoration of an image provided in an embodiment of the present application;
[0029] Figure 2 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;
[0030] Figure 3 A flowchart of a method for color restoration of an image provided in an embodiment of the present application;
[0031] Figure 4 A schematic diagram of a user selecting a first image to be processed provided in an embodiment of the present application;
[0032] Figure 5 A schematic diagram of a first region provided in an embodiment of the present application;
[0033] Figure 6 A schematic diagram of a binary image corresponding to a first region provided in an embodiment of the present application;
[0034] Figure 7 A schematic diagram of a connected area provided in an embodiment of the present application;
[0035] Figure 8 A schematic diagram of a first region, a binary image corresponding to the first region, and a connected region determined according to the binary image provided in an embodiment of the present application;
[0036] Fig. 9 A schematic diagram of the structure of another image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] The embodiments of the present application provide a method and device for color (color) restoration of an image, which is applied to the scene of color restoration of an overexposed image. It is understandable that the embodiments of the present application can also be applied to the scene of color restoration of a video including one or more frames of overexposed images. Specifically, it can be applied to the process of color restoration of an overexposed object (or overexposed area) in an image. For example, in the process of color restoration of an overexposed traffic light in an image or video taken by an electronic police (camera or monitor).
[0038] The present application embodiment takes the scene where an electronic police camera captures images of traffic violations as an example. Figure 1 As shown, a system architecture diagram of a method for color restoration of an image (a violation image captured by an electronic police camera) is provided, including an electronic police camera and an image processing device connected to the electronic police camera (the image processing device may also be integrated in the electronic police camera), the image processing device may perform color restoration processing on the image or video captured by the electronic police camera. The image processing device may also be connected to a traffic light to obtain the (historical) color change of the light from a light detector on the traffic light.
[0039] Embodiments of the present application Figure 1 The image processing device in the embodiment of the present application may be implemented by a device or a functional module in a device, and the present application embodiment does not specifically limit this. It is understandable that the above functions can be network elements in hardware devices, software functions running on dedicated hardware, virtualization functions instantiated on a platform (e.g., a cloud platform), or chip systems. In the embodiment of the present application, the chip system can be composed of chips, or it can include chips and other discrete devices.
[0040] For example, the apparatus for realizing the functions of the image processing device provided in the embodiment of the present application can be Figure 2 It is implemented by the device 200 in FIG. Figure 2 The hardware structure diagram of the device 200 provided in the embodiment of the present application is shown. The device 200 includes at least one processor 201, which is used to implement the functions of the image processing device provided in the embodiment of the present application. The device 200 may also include a bus 202 and at least one communication interface 204. The device 200 may also include a memory 203.
[0041] In the embodiments of the present application, the processor may be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor may also be any other device having a processing function, such as a circuit, a device, or a software module.
[0042] The bus 202 may be used to transmit information between the above-mentioned components.
[0043] The communication interface 204 is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 204 can be an interface, circuit, transceiver or other device capable of communication, which is not limited in this application. The communication interface 204 can be coupled to the processor 201. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.
[0044] In the embodiment of the present application, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently or be coupled to the processor, for example, through bus 202. The memory may also be integrated with the processor.
[0045] The memory 203 is used to store program instructions, and can be controlled by the processor 201 to execute, so as to implement the method for color restoration of an image provided in the following embodiments of the present application. The processor 201 is used to call and execute the instructions stored in the memory 203, so as to implement the method for color restoration of an image provided in the following embodiments of the present application.
[0046] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.
[0047] Optionally, the memory 203 may be included in the processor 201 .
[0048] In a specific implementation, as an embodiment, the processor 201 may include one or more CPUs, such as Figure 2 CPU0 and CPU1 in.
[0049] In a specific implementation, as an embodiment, the apparatus 200 may include multiple processors, such as Figure 2 201 and processor 207 in the embodiment of the present invention. Each of these processors may be a single-CPU processor or a multi-CPU processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0050] In a specific implementation, as an embodiment, the apparatus 200 may further include an output device 205 and an input device 206. The output device 205 is coupled to the processor 201 and may display information in a variety of ways. For example, the output device 205 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 206 is coupled to the processor 201 and may receive user input in a variety of ways. For example, the input device 206 may be a camera, a mouse, a keyboard, a touch screen device, or a sensor device.
[0051] The apparatus 200 may be a general purpose device or a dedicated device. In a specific implementation, the image processing device 200 may be a camera, a camera, a monitor, a video display device, a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device or a computer with a processor. Figure 2The embodiment of the present application does not limit the type of the device 200.
[0052] In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant concepts or technologies is first given:
[0053] Color space: Color is usually described by three independent attributes. The three independent variables work together to form a space coordinate, namely the color space. Color space can include RGB (color) space, YUV (color) space, HSV (color) space and HSL (color) space, etc. Different color spaces can measure the color of the same object from different angles. In different processing processes, the emphasis on color processing is different, so various color spaces can be converted to each other to meet different processing requirements. Among them:
[0054] RGB space: R represents the red component (red channel), G represents the green component (green channel), and B represents the blue component (blue channel). Various colors are obtained by changing the red, green, and blue color channels and superimposing them on each other. In each component, the smaller the value, the lower the brightness, and the larger the value, the higher the brightness. When the components are mixed, the brightness of the mixture is equal to the sum of the brightness of the components.
[0055] YUV space: Y represents brightness, that is, grayscale value. U and V represent chrominance, which is used to describe the color and saturation of the image, and can specify the color of the pixel. If there is only a Y signal component but no U and V components, the image represented in this way is a black and white grayscale image. YUV space is mainly used to optimize the transmission of color video signals to make it backward compatible with old black and white TVs.
[0056] HSV space: HSV is a method of representing points in the RGB color space in an inverted cone. Among them, H represents color information, that is, the position of the spectral color, which can be measured by angle, ranging from 0° to 360°. Red is 0°, green is 120°, and blue is 240°. S represents the ratio between the saturation of the selected color and the maximum saturation of the color, ranging from 0 to 1. When S=0, there is only grayscale. V represents the brightness of the color, ranging from 0 to 1.
[0057] HSL space: HSL is similar to HSV, and the first two parameters in this model are the same as HSV. L represents the brightness of the color, which can be used to control the light and dark changes of the color. The value range of L is 0% to 100%. The smaller the value, the darker the color and the closer it is to black; the larger the value, the brighter the color and the closer it is to white.
[0058] The technical scheme in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. Wherein, in the description of the present application, unless otherwise specified, "at least one" refers to one or more, and "a plurality of" refers to two or more than two. In addition, in order to facilitate the clear description of the technical scheme of the embodiment of the present application, in the embodiment of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit necessarily different.
[0059] For ease of understanding, the method for color restoration of an image provided by an embodiment of the present application is specifically introduced below with reference to the accompanying drawings.
[0060] like Figure 3 As shown, the embodiment of the present application provides a method for color restoration of an image, comprising:
[0061] 301. Acquire a first image to be processed, where the first image includes an overexposed first target object.
[0062] The first target object refers to one or more objects (or areas where the objects are located) that are overexposed in the first image. The objects may be, for example, traffic lights (of various shapes and colors), vehicles, traffic signs, and the like.
[0063] In a possible design, the second area selected by the user on the second image can be used as the first image to be processed. Figure 4 As shown, when the user believes that the traffic light (first target object) in a certain image (second image) is overexposed, the user can perform a first operation on the second image through the input device of the image processing device (for example, a mouse or a touch screen), and the first operation is used to select the area (second area) where the overexposed traffic light is located. After the image processing device recognizes the user's first operation on the second image, the second area can be cut from the second image according to the coordinate information of the second area selected by the user to obtain a third image and the first image to be processed. Among them, the first image to be processed includes the second area, and the third image includes other areas except the second area. That is, the first image to be processed includes the second area selected by the user on the second image.
[0064] In this way, compared with directly processing the second image, selecting the first image from the second image and processing only the first image can, on the one hand, reduce the amount of calculation in subsequent steps, and on the other hand, improve the recognition accuracy of the area where the first target object (for example, a traffic light) is located.
[0065] 302. Optionally, filter the saturation component and the brightness component of the pixels of the first image.
[0066] It should be noted that if the second image is an image in HSV format, that is, the second image is in the HSV space, then the first image to be processed is also in the HSV space, and the saturation component and brightness component of the pixels of the first image can be directly filtered so that the outline of the overexposed first object (for example, an overexposed traffic light) can be more stably and accurately identified in subsequent steps.
[0067] If the second image is in the first space, which may be a YUV space, an RGB space, or an HSL space, then the first image to be processed is also in the first space. At this time, in order to obtain the saturation component and the brightness component of the pixels of the first image, the first image needs to be converted from the first space to the HSV space. Then, the saturation component and the brightness component of the pixels of the first image are filtered, so that the outline of the overexposed first object can be more stably and accurately identified in subsequent steps.
[0068] 303. Determine a first region of the first image according to saturation and brightness of pixels of the first image.
[0069] In the embodiment of the present application, the first area corresponds to the area where the first target object is located. The area where the overexposed first target object (eg, an overexposed traffic light) is located in the first image can be determined according to the saturation and brightness of the pixels of the first image.
[0070] In a possible design, the saturation component and brightness component of the pixels of the first image after filtering can be analyzed to determine that the area where the saturation component is lower than the average saturation of the pixels of the first image and the brightness component is higher than the average brightness of the pixels of the first image is the area where the first target object is located. That is, the saturation of the pixels in the first area is lower than the average saturation of the pixels in the first image, and the brightness of the pixels in the first area is higher than the average brightness of the pixels in the first image. Figure 5 Shown is a schematic diagram of a first area.
[0071] 304. Perform binarization processing on the first region to obtain a binary image corresponding to the first region.
[0072] For example, Figure 6 As shown, Figure 5 After the first region shown is binarized, a binary image corresponding to the first region can be obtained. The binary image corresponding to the first region can include three connected regions (respectively a, b and c). The connected region a is the region where the traffic light is located, and the connected regions b and c can be the regions where the halos generated by the traffic light are located.
[0073] 305. Determine at least one connected region in the binary image whose area is greater than or equal to a first threshold.
[0074] In a possible design, the contour of at least one connected region in the binary image whose area is greater than or equal to the first threshold corresponds to the contour of the first target object. The contour of at least one connected region in the binary image whose area is greater than or equal to the first threshold can be used as the contour of the first target object. It should be noted that the contour of at least one connected region whose area is greater than or equal to the first threshold may be larger or smaller than the actual (real) contour of the first target object. For example, when the first target object is a signal light, since the (lit) signal light will generate a halo, the at least one connected region may include not only the area where the signal light itself is located, but also the area where the halo generated by the signal light is located. Therefore, the contour of at least one connected region whose area is greater than or equal to the first threshold may be larger than the actual contour of the signal light. For another example, if a part of the signal light is damaged and cannot emit light, the at least one connected region may only include the area where the part of the signal light that emits light is located. At this time, the contour of the at least one connected region is smaller than the actual contour of the signal light.
[0075] The first threshold may be determined according to the area of the largest connected region in the binary image. For example, the first threshold may be equal to the area of the largest connected region in the binary image, or the first threshold may be N% (e.g., 30%) of the area of the largest connected region in the binary image, where N is a positive number.
[0076] There are many connected regions of different sizes in the binary image corresponding to the first region. Figure 6 As shown, the binary image corresponding to the first area may include three connected areas (respectively a, b and c). Based on the binary image, all connected areas in the binary image are searched. All the found connected areas are sorted according to the size of the area (for example, all the connected areas are arranged in order from large to small in area). Among these connected areas of different sizes, the connected areas with smaller areas can be filtered out, the connected areas with larger areas can be retained, and at least one connected area with a larger area can be used as the area of the first target object (for example, a traffic light) (for example, the connected area with the largest area can be used as the area where the traffic light is located, or multiple connected areas with larger areas can be used as the area where the traffic light is located).
[0077] For example, we can first filter out the connected region with the largest area in the binary image. Assuming that its area is max, we can set the first threshold to max / (x+2) to filter out the connected regions with an area smaller than max / (x+2). Figure 7As shown, the connected regions (b and c) can be filtered out, and the connected region a is retained. The contour of the connected region a corresponds to the contour of the first target object.
[0078] Among them, the value range of parameter x can be 0-9, and parameter x can be used to adjust the size of the first threshold. The larger the value of x, the fewer connected areas with smaller areas are filtered out, that is, the more connected areas with smaller areas are retained, so that the contour of the first target object corresponds to the contours of more connected areas. For example, if the first target object is a traffic light, the area where the traffic light is located spreads outward (that is, the area where the traffic light is located not only includes the traffic light, but also includes the halo generated by the traffic light). The smaller the value of x, the more connected areas with smaller areas are filtered out, that is, the fewer connected areas with smaller areas are retained, so that the contour of the first target object corresponds to the contours of fewer connected areas. For example, if the first target object is a traffic light, the area where the traffic light is located shrinks inward (that is, the area where the traffic light is located only includes the traffic light, not including the halo generated by the traffic light).
[0079] For another example, if the first image to be processed includes an arrow-shaped indicator light, the image processing device determines that the first area of the first image is Figure 8 As shown in (a) of FIG. 1 , after the first region is binarized, as shown in FIG. 1 , Figure 8 As shown in (b) in FIG. 1 , the binary image corresponding to the first region may include four connected regions (d, e, f and g respectively), as shown in FIG. Figure 8 As shown in (c), at least one connected region in the binary image whose area is greater than or equal to the first threshold value may be connected regions d and e. That is, the connected regions (f and g) are filtered out, and the connected regions d and e are retained. The contours of the connected regions d and e may be used as the contours of the first target object.
[0080] 306. Restore the color of at least one connected region.
[0081] In the embodiment of the present application, restoring the color of at least one connected area can also be considered as correcting the color of at least one connected area. Restoring or correcting the color of at least one connected area is to restore or correct the color of the overexposed first target object. Figure 7 As shown, restoring or correcting the color of the connected area a is to restore or correct the color of the overexposed first target object.
[0082] Exemplarily, when the first target object is a traffic light, the color information of the traffic light may be first acquired based on a traffic light detector externally connected to the traffic light or based on an image recognition status light.
[0083] In a possible design, the color of at least one connected region can be restored in the HSV space. Specifically, when (determined) the traffic light is red, the hue of the pixels of at least one connected region is adjusted to the red range, and the saturation and brightness of the pixels of at least one connected region are increased and decreased respectively; when (determined) the traffic light is yellow, the hue of the pixels of at least one connected region is adjusted to the yellow range, and the saturation and brightness of the pixels of at least one connected region are increased and decreased respectively; when (determined) the traffic light is green, the hue of the pixels of at least one connected region is adjusted to the green range, and the saturation and brightness of the pixels of at least one connected region are increased and decreased respectively.
[0084] In an embodiment of the present application, the saturation and brightness of the pixels of at least one connected region are increased and decreased respectively by linearly increasing and linearly decreasing the saturation and brightness of the pixels of at least one connected region, or the saturation and brightness of the pixels of at least one connected region are increased and decreased nonlinearly. Linearly increasing the saturation can increase the saturation of each pixel in the pixels of at least one connected region by the same amount (value); nonlinearly increasing the saturation can increase the saturation of different pixels in the pixels of at least one connected region by different amounts (values). Linearly decreasing the brightness can reduce the brightness of each pixel in the pixels of at least one connected region by the same amount (value); nonlinearly decreasing the brightness can reduce the brightness of different pixels in the pixels of at least one connected region by different amounts (values). It should be noted that after the saturation and brightness of the pixels of at least one connected region are increased and decreased respectively, the saturation and brightness of the at least one connected region will have a strong correlation. Before the saturation and brightness of the pixels of at least one connected region are increased and decreased respectively, the saturation and brightness of the at least one connected region do not have a strong correlation.
[0085] In another possible design, the color of at least one connected region may be restored in the RGB space. Specifically, when (determined) the traffic light is red, the red component of the pixels of the at least one connected region may be adjusted to a first preset range, and the blue and green components of the pixels of the at least one connected region may be reduced.
[0086] For example, assuming that an 8-bit encoded RGB image is restored to color, if the red component (R value) of a pixel is Rx, the R value of the pixel can be adjusted to R=(200+55*x / 9+Rx) / 2 (first preset range). The parameter x can range from 0 to 9, and the parameter x can be used to adjust the size of the R value.
[0087] Wherein, reducing the blue and green components of the pixels of at least one connected region may be a linear reduction or a nonlinear reduction of the blue and green components of the pixels of at least one connected region. The linear reduction may reduce the blue and green components of each pixel in the pixels of at least one connected region by the same amount (value); the nonlinear reduction may reduce the blue and green components of different pixels in the pixels of at least one connected region by different amounts (values).
[0088] When (determined) the traffic light is yellow, the red and green components of the pixels of at least one connected area are increased, and the blue component of the pixels of at least one connected area is reduced; when (determined) the traffic light is green, the green component of the pixels of at least one connected area is increased, and the red and blue components of the pixels of at least one connected area are reduced. The specific process can refer to the above-mentioned relevant processing process when the traffic light is red, which will not be repeated here.
[0089] 307. Merge the first image and the third image.
[0090] If the first image is converted from the first space to the HSV space in step 302, the first image after color restoration can be converted from the HSV space to the first space again, so that the first image and the third image (i.e., the remaining portion after the first image is cut off from the second image) are combined to obtain a complete image (the complete image can be considered to be the second image after color restoration). If the operation of converting the first image from the first space to the HSV space is not performed in step 302, the first image after color restoration and the third image can be directly combined to obtain a complete image.
[0091] In one possible design, during the process of restoring the color of the overexposed first target object in the video, the user can select the overexposed object before the video starts or when it is paused. During the subsequent playback of the video, the method of the embodiment of the present application can be used to perform corresponding processing on each frame image of the video, thereby restoring the color of the overexposed first target object in the entire video.
[0092] Based on the method provided in the embodiment of the present application, after obtaining the first image to be processed, the first area of the first image can be determined according to the saturation and brightness of the pixels of the first image (the first area can be used as the area where the overexposed first target object is located), and then, the first area is binarized to obtain a binary image corresponding to the first area; then, at least one connected area in the binary image whose area is greater than or equal to the first threshold is determined (the outline of the at least one connected area can be used as the outline of the first target object), and then, the color of the at least one connected area is restored (that is, the color of the overexposed first target object is restored). It can be seen that the method provided in the embodiment of the present application can accurately restore the color of overexposed objects in the image (for example, traffic lights). Furthermore, a series of problems caused by overexposure of objects in the image can be solved (for example, the difficulty in obtaining evidence caused by overexposure of traffic lights in the current electric police monitoring scenario).
[0093] In the prior art, a camera with an ultra-wide dynamic range can be used to eliminate the color distortion of signal lights caused by strong light, but ultra-wide dynamic cameras are expensive and have limited effect on the color restoration of overexposed signal lights under low illumination. The method provided in the embodiment of the present application does not require the addition of additional equipment, is inexpensive, can achieve the effect of color restoration, and has high environmental adaptability, and can restore the color of overexposed signal lights under low illumination. In the prior art, the color restoration of signal lights can also be performed based on the RGB space of the image, or the signal lights in the image can be regionally identified and color enhanced based on a deep learning method, but this may cause a discontinuous gradient phenomenon between the signal light area after color restoration and the surrounding image. The method provided in the embodiment of the present application is based on the HSV space, which better reflects the brightness information of the physical signal light, so that the identification of the signal light area is more stable and accurate, and the signal light area after color restoration is more continuous with the surrounding image, and the effect is more realistic to the naked eye. In addition, the method provided in the embodiment of the present application is efficient and reliable, and can meet the color restoration effect of signal lights in the video.
[0094] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the image processing device. It is understandable that, in order to realize the above functions, the image processing device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and software. Whether a function is executed in a hardware or software-driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.
[0095] The embodiment of the present application can divide the functional modules of the image processing device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0096] In the case of dividing each functional module into corresponding functional modules, Fig. 9 A possible structural diagram of the image processing device 9 involved in the above embodiment is shown, and the image processing device includes: an acquisition unit 901, a determination unit 902 and a processing unit 903. The acquisition unit 901 is used to support the image processing device to execute Figure 3 The determining unit 902 is used to support the image processing device to perform Figure 3 Process 303 and 305 in the process. The processing unit 903 is used to support the image processing device to perform Figure 3 302, 304 and 306. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
[0097] The steps of the method or algorithm described in conjunction with the disclosure of the present application may be implemented in hardware or by executing software instructions by a processor. The software instructions may be composed of corresponding software modules, and the software modules may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, mobile hard disks, read-only optical disks, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an application specific integrated circuit (ASIC). In addition, the ASIC may be located in a core network interface device. Of course, the processor and the storage medium may also exist in a core network interface device as discrete components.
[0098] Those skilled in the art should be aware that in one or more of the above examples, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in an image processing device readable medium or transmitted as one or more instructions or codes on an image processing device readable medium. The image processing device readable medium includes an image processing device storage medium and a communication medium, wherein the communication medium includes any medium that facilitates the transmission of an image processing device program from one place to another. The storage medium can be any available medium that a general or special image processing device can access.
[0099] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation methods of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.
Claims
1. A method for color restoration of an image, characterized in that: include: Acquire a first image to be processed, wherein the first image includes an overexposed first target object; determining a first region of the first image according to the saturation and brightness of the pixels of the first image, wherein the saturation of the pixels of the first region is lower than an average value of the saturation of the pixels of the first image, and the brightness of the pixels of the first region is higher than an average value of the brightness of the pixels of the first image; the first region corresponds to a region of the first target object; Performing binarization processing on the first region to obtain a binary image corresponding to the first region; Determine at least one connected region in the binary image whose area is greater than or equal to a first threshold; the contour of the at least one connected region corresponds to the contour of the first target object; The color of the at least one connected region is restored.
2. The method for color restoration of an image according to claim 1, characterized in that: Before determining the first area of the first image according to the saturation and brightness of the first image, the method further includes: The first image is converted from a first space to a hue-saturation-lightness (HSV) space, where the first space is any one of a brightness-chromaticity (YUV) space, a red-green-blue (RGB) space, or a hue-saturation-lightness (HSL) space.
3. The method for color restoration of an image according to claim 1 or 2, characterized in that: When the first target object is a traffic light, restoring the color of the at least one connected area includes: Obtaining color information of the traffic light; When the traffic light is red, adjusting the hue of the pixels of the at least one connected area to a red range, and increasing and decreasing the saturation and brightness of the pixels of the at least one connected area respectively; When the traffic light is yellow, adjusting the hue of the pixels of the at least one connected area to a yellow range, and increasing and decreasing the saturation and brightness of the pixels of the at least one connected area respectively; When the traffic light is green, the hue of the pixels of the at least one connected area is adjusted to a green range, and the saturation and brightness of the pixels of the at least one connected area are increased and decreased respectively.
4. The method for color restoration of an image according to claim 1 or 2, characterized in that: When the first target object is a traffic light, restoring the color of the at least one connected area includes: Converting the at least one connected region into RGB space; Obtaining color information of the traffic light; When the traffic light is red, adjusting the red component of the pixels of the at least one connected area to a first preset range, and reducing the blue and green components of the pixels of the at least one connected area; When the traffic light is yellow, adjusting the red and green components of the pixels of the at least one connected area to a second preset range, and reducing the blue component of the pixels of the at least one connected area; When the traffic light is green, the green component of the pixels of the at least one connected area is adjusted to a third preset range, and the red and blue components of the pixels of the at least one connected area are reduced.
5. The method for color restoration of an image according to claim 1, characterized in that: Before determining the first area of the first image according to the saturation and brightness of the first image, the method further includes: The saturation component and the brightness component of the pixels of the first image are filtered.
6. The method for color restoration of an image according to claim 1, characterized in that: The acquiring of a first image to be processed comprises: The second area selected by the user on the second image is used as the first image to be processed.
7. An image processing device, characterized in that: include: An acquisition unit, configured to acquire a first image to be processed, wherein the first image includes an overexposed first target object; a determining unit, configured to determine a first region of the first image according to saturation and brightness of pixels of the first image, wherein the saturation of pixels of the first region is lower than an average value of saturation of pixels of the first image, and the brightness of pixels of the first region is higher than an average value of brightness of pixels of the first image; the first region corresponds to a region of the first target object; A processing unit, configured to perform a binarization process on the first region to obtain a binary image corresponding to the first region; The determining unit is further used to determine at least one connected region in the binary image whose area is greater than or equal to a first threshold; the contour of the at least one connected region corresponds to the contour of the first target object; The processing unit is further used to restore the color of the at least one connected area.
8. The image processing device according to claim 7, characterized in that The processing unit is also used for: The first image is converted from a first space to a hue-saturation-lightness (HSV) space, where the first space is any one of a brightness-chromaticity (YUV) space, a red-green-blue (RGB) space, or a hue-saturation-lightness (HSL) space.
9. The image processing device according to claim 7 or 8, characterized in that: When the first target object is a traffic light, the processing unit is used to: Acquiring the color information of the traffic light by the acquisition unit; When the traffic light is red, adjusting the hue of the pixels of the at least one connected area to a red range, and increasing and decreasing the saturation and brightness of the pixels of the at least one connected area respectively; When the traffic light is yellow, adjusting the hue of the pixels of the at least one connected area to a yellow range, and increasing and decreasing the saturation and brightness of the pixels of the at least one connected area respectively; When the traffic light is green, the hue of the pixels of the at least one connected area is adjusted to a green range, and the saturation and brightness of the pixels of the at least one connected area are increased and decreased respectively.
10. The image processing device according to claim 7 or 8, characterized in that: When the first target object is a traffic light, the processing unit is used to: Converting the at least one connected region into RGB space; Acquiring the color information of the traffic light by the acquisition unit; When the traffic light is red, adjusting the red component of the pixels of the at least one connected area to a first preset range, and reducing the blue and green components of the pixels of the at least one connected area; When the traffic light is yellow, adjusting the red and green components of the pixels of the at least one connected area to a second preset range, and reducing the blue component of the pixels of the at least one connected area; When the traffic light is green, the green component of the pixels of the at least one connected area is adjusted to a third preset range, and the red and blue components of the pixels of the at least one connected area are reduced.
11. The image processing device according to claim 7, characterized in that: The processing unit is also used for: The saturation component and the brightness component of the pixels of the first image are filtered.
12. The image processing device according to claim 7, characterized in that The acquisition unit is used for: The second area selected by the user on the second image is used as the first image to be processed.
13. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a computer, the computer is enabled to execute the method for color restoration of an image as claimed in any one of claims 1 to 6.
