Network camera system

Abstract

Appropriate image adjustments should be made regardless of the shooting environment. [Solution] A network camera system comprising multiple imaging devices, wherein each imaging device includes an image correction unit that calculates image correction values ​​necessary for image adjustment based on image data acquired by an imaging unit, a storage unit that stores the image correction values, a shared mode setting unit that accepts the setting of a shared mode for the image correction values, and a communication unit. If the shared mode setting unit does not accept the setting of a shared mode for the image correction values, the system causes the image correction unit to correct the image data based on the image correction values. If the shared mode setting unit accepts the setting of a shared mode for the image correction values ​​and this imaging device is set as the transmitting side for the image correction values, the system causes the other imaging devices set as the receiving side to share the image correction values ​​and correct the image data. If this imaging device is set as the receiving side for the image correction values, the system causes the image data to correct based on the image correction values.

Description

Technical Field

[0001] The present disclosure relates to a network camera system, and more particularly, to a network camera system that displays images from a plurality of network cameras.

Background Art

[0002] Conventionally, in an imaging apparatus such as a camera, a technique is known for adjusting white balance so that a white subject appears white in the captured image, regardless of the color of light in the surrounding environment. For example, when objects of various colors appear in a captured image, a method is known in which the light source color is estimated under the assumption that the average color is an achromatic color, and the white balance is adjusted by correcting the color to a standard light source color.

[0003] Further, as an imaging apparatus that performs such white balance adjustment, for example, an imaging apparatus having white balance adjustment means characterized by performing white balance adjustment on a captured image based on weather information, shooting date and time, shooting location information, and map information is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0005] However, when most of a captured image is occupied by a specific object, the average color is biased toward the color of that object, so the light source color cannot be correctly estimated, and incorrect correction may be performed. For example, in a surveillance camera that captures an empty lot, when most of the captured image is occupied by green weeds, the image is erroneously estimated to be illuminated by a green light source, and as a result, the image is corrected to a reddish-brown color.

[0006] Traditionally, a technique has been known to switch to manual correction when such misestimations occur. However, manual correction has the problem that, because the correction value is fixed, it cannot provide appropriate correction in situations where the light source color constantly changes depending on changes in shooting time, weather, etc., such as with imaging devices installed outdoors.

[0007] This disclosure has been made in consideration of the circumstances described above, and its purpose is to provide a network camera system that can appropriately adjust images regardless of the shooting environment. [Means for solving the problem]

[0008] This disclosure relates to a network camera system comprising a plurality of imaging devices, each imaging device comprising: an imaging unit that captures images and acquires image data; an image correction unit that calculates predetermined image correction values ​​necessary for image adjustment based on the image data; a storage unit that stores the image correction values; a shared mode setting unit that accepts the setting of a shared mode for the image correction values; a communication unit that sends and receives data with other imaging devices; and a control unit that controls the imaging unit, the image correction unit, the storage unit, the shared mode setting unit and the communication unit, wherein if the shared mode setting unit has not accepted the setting of a shared mode for the image correction values, the control unit will determine the image data based on the image correction values ​​acquired by the imaging unit. The present invention provides a network camera system characterized in that, if the sharing mode setting unit accepts the setting of the sharing mode for the image correction value and the imaging device is set as the transmitting side for the image correction value, the image correction value acquired by the imaging device is shared with the other imaging device set as the receiving side, and the image data is corrected by the image correction unit based on the image correction value, and on the other hand, if the sharing mode setting unit accepts the setting of the sharing mode for the image correction value and the imaging device is set as the receiving side for the image correction value, the image data is corrected by the image correction unit based on the image correction value shared with the other imaging device set as the transmitting side. [Effects of the Invention]

[0009] According to this disclosure, a network camera system can be realized that accepts the setting of a shared mode for image correction values, and when this imaging device is set as the receiver of image correction values, it corrects image data based on the image correction values ​​shared with other imaging devices set as the transmitter, thereby enabling appropriate image adjustment regardless of the shooting environment. [Brief explanation of the drawing]

[0010] [Figure 1] This block diagram shows the schematic configuration of the network camera system according to Embodiment 1 of this disclosure. [Figure 2] Figure 1 is a block diagram showing the schematic configuration of the camera. [Figure 3] Figure 1 is a block diagram showing the schematic configuration of the image display device. [Figure 4] Figure 1 is a flowchart illustrating an example of the white balance adjustment process for a network camera system. [Figure 5] This diagram illustrates the problems with the white balance adjustment process when the sharing mode for correction results is not set in the network camera system shown in Figure 1. [Figure 6] This is an explanatory diagram showing an example of the results of white balance adjustment processing when the correction result sharing mode is set in the network camera system shown in Figure 1. [Figure 7] Figure 1 is an explanatory diagram showing an example of white balance adjustment for image data received from multiple cameras in a modified example of the network camera system. [Modes for carrying out the invention]

[0011] Furthermore, preferred embodiments of this disclosure will be described.

[0012] In the network camera system according to this disclosure, the control unit may store the light source color estimated based on the image correction value acquired by the imaging unit in the storage unit, and if the light source color changes beyond a predetermined range from the previous time, it may cause the image correction unit to correct the image data.

[0013] In this way, a network camera system can be realized that can appropriately adjust images regardless of the shooting environment by correcting the image data only when the light source color estimated based on the image correction value acquired by the imaging unit has changed by more than a predetermined range from the light source color previously stored in the memory unit.

[0014] In the network camera system according to this disclosure, the control unit may, when the shared mode setting unit receives the setting of the shared mode for the image correction value and the imaging device and one or more other imaging devices are set as the transmitting side for the image correction value, share the average value of the image correction value acquired by the imaging unit and the image correction value acquired by the other imaging devices set as the transmitting side with the other imaging devices set as the receiving side.

[0015] In this way, if this imaging device and one or more other imaging devices are set as the transmitting side for image correction values, the average value of the image correction values ​​acquired by the imaging unit and the image correction values ​​acquired by the other imaging devices set as transmitting sides can be shared with the other imaging devices set as receiving sides, thereby realizing a network camera system that can appropriately adjust images regardless of the shooting environment.

[0016] In the network camera system described herein, the image adjustment may be white balance adjustment.

[0017] With this configuration, it is possible to realize a network camera system that accepts setting of a sharing mode for image correction values, and when the present imaging apparatus is set as the image correction value receiving side, corrects image data based on the image correction values shared with another imaging apparatus set as the transmitting side, thereby enabling appropriate white balance adjustment regardless of the shooting environment.

[0018] The network camera system according to this disclosure may further include an image display apparatus that displays an image based on the image data received from the plurality of imaging apparatuses, and the control unit may cause the communication unit to transmit the corrected image data to the image display apparatus.

[0019] With this configuration, it is possible to realize a network camera system that accepts setting of a sharing mode for image correction values, and when the present imaging apparatus is set as the image correction value receiving side, corrects image data based on the image correction values shared with another imaging apparatus set as the transmitting side, thereby being capable of transmitting image data that has been appropriately adjusted regardless of the shooting environment to an image display apparatus.

[0020] Hereinafter, this disclosure will be described in further detail with reference to the drawings. It should be noted that the following description is illustrative in all respects and should not be construed as limiting this disclosure.

[0021] [Embodiment 1] <Schematic Configuration of Network Camera System 100> The network camera system 100 according to Embodiment 1 of this disclosure will be described based on FIG. 1 to FIG. 3. FIG. 1 is a block diagram showing a schematic configuration of the network camera system 100 according to Embodiment 1 of this disclosure.

[0022] As shown in Figure 1, the network camera system 100 according to Embodiment 1 of this disclosure consists of camera 1A, camera 1B (hereinafter collectively referred to as camera 1), image display device 2, and network 3. The network camera system 100 may also be equipped with any number of cameras 1 and image display devices 2.

[0023] Camera 1 is an imaging device that is installed to capture images of any location. Each camera 1 is assigned a unique IP address. In the example in Figure 1, camera 1A is assigned the IP address (192.168.0.11), and camera 1B is assigned the IP address (192.168.0.22). Image data captured by camera 1 is transmitted to image display device 2 via network 3.

[0024] Figure 2 is a block diagram showing the schematic configuration of camera 1 in Figure 1. As shown in Figure 2, the camera 1 comprises a control unit 10, an imaging unit 11, a storage unit 12, an image correction unit 13, a communication unit 14, and a shared mode setting unit 15. The following describes each component of Camera 1.

[0025] The control unit 10 comprehensively controls the camera 1 and consists of at least one CPU (Central Processing Unit), at least one SoC (System on a Chip), at least one RAM (Random Access Memory), at least one ROM (Read-only memory), at least one control circuit, various interface circuits, etc. The control unit 10 monitors and controls all loads on each part in order to control the operation of the entire camera 1.

[0026] The imaging unit 11 is equipped with a lens and has various mechanisms such as a shutter, aperture, focus, and zoom, which are driven by a drive circuit (not shown). The imaging unit 11 is equipped with an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and converts the optical image formed on the image sensor through a lens into an electric charge corresponding to the amount of light by photoelectric conversion, and stores it in the memory unit 12.

[0027] The memory unit 12 is an element or storage medium that stores information necessary to realize the various functions of the camera 1, as well as control programs. For example, semiconductor elements such as RAM and ROM, storage media such as hard disks, flash memory units, and SSDs are used. Furthermore, the data may be stored in different devices, such as a hard disk drive for the data storage area and a flash memory unit for the program storage area.

[0028] The image correction unit 13 calculates the image correction values ​​necessary for white balance adjustment based on the image data stored in the memory unit 12, and corrects the image data. The specific process involves analyzing image data to estimate the light source color, and if it exceeds a predetermined range from a predetermined standard light source color (e.g., white), it replaces it with the standard light source color to return it to white. This correction data is vector data represented by the coordinate values ​​(X,Y) of the estimated light source color.

[0029] This type of image data correction makes it possible to display image data using the color of an object as it would appear under white light, even if, for example, an object illuminated by a setting sun appears orange. As a result, the color of a person's face and clothing captured by a security camera can be accurately estimated regardless of the light source color.

[0030] Incidentally, the colors we actually see are the result of multiplying the light source color by the spectral reflectance characteristics of the object (the object's color). However, the color distribution of objects in an image includes a variety of colors, and it is thought that the centroid of this distribution is achromatic.

[0031] However, if the color of an object in the image data is extremely biased towards a particular color, there is a risk of misestimating that it is illuminated by a light source of that color. Therefore, to solve the problem of such misestimation of the light source, white balance adjustment is performed using the shared mode setting described later.

[0032] The communication unit 14 transmits the image data stored in the storage unit 12 and corrected by the image correction unit 13 to the image display device 2.

[0033] The shared mode setting unit 15 accepts settings for a shared mode in which the correction results of a specific camera 1 are referenced among multiple cameras 1 during white balance adjustment. When the setting to shared mode is accepted, camera 1A, which is set as the transmitting side, is configured to share its correction results with camera 1B, which is set as the receiving side. Then, camera 1B performs white balance adjustment by referring to the correction results of camera 1A.

[0034] The image display device 2 displays image data received from camera 1 via network 3. Alternatively, the image display device 2 may access the web management screen of camera 1 to configure camera 1.

[0035] Figure 3 is a block diagram showing the schematic configuration of the image display device 2 shown in Figure 1. As shown in Figure 3, the image display device 2 comprises a control unit 20, a communication unit 21, an image processing unit 22, a storage unit 23, a display unit 24, and an operation unit 25. The following describes each component of the image display device 2.

[0036] The control unit 20 comprehensively controls the image display device 2 and consists of a CPU, RAM, ROM, various interface circuits, and the like. The control unit 20 monitors and controls all loads on each component in order to control the operation of the entire image display device 2.

[0037] The communication unit 21 receives image data from the camera 1 via the network 3.

[0038] The image processing unit 22 is the part that converts the image data received from the camera 1 into an appropriate electrical signal and processes it so that it is suitable for output to the display unit 24.

[0039] The memory unit 23 is an element or storage medium that stores information and control programs necessary to realize various functions of the image display device 2. For example, semiconductor elements such as RAM and ROM, storage media such as hard disks, flash memory units, and SSDs are used. Furthermore, the data may be stored in different devices, such as a hard disk drive for the data storage area and a flash memory unit for the program storage area.

[0040] The display unit 24 is a display device such as a monitor that displays images based on image data received from the camera 1. The display unit 24 is composed of, for example, a CRT display, a liquid crystal display, or an EL display.

[0041] The operation unit 25 is an interface for operating the image display device 2 and is the part that receives settings from the user.

[0042] Network 3 refers to wired or wireless networks such as LANs, wide area networks (WANs) like the Internet, and dedicated communication lines.

[0043] <White balance adjustment process of network camera system 100 according to Embodiment 1 of this disclosure> Next, the white balance adjustment process of the network camera system 100 according to Embodiment 1 of this disclosure will be described based on Figures 4 to 6. Figure 4 is a flowchart showing an example of the white balance adjustment process for the network camera system 100 shown in Figure 1.

[0044] In step S1 of Figure 4, the control unit 10 of the camera 1 estimates the light source color based on the image data acquired by the imaging unit 11 (step S1).

[0045] Next, in step S2, the control unit 10 refers to the estimated light source color stored in the memory unit 12 and determines whether the light source color has changed beyond a predetermined range from the light source color previously stored in the memory unit 12 (step S2).

[0046] If the estimated light source color has not changed by more than a predetermined range since the previous measurement (i.e., the determination in step S2 is No), in step S8, the control unit 10 instructs the image display device 2 to transmit image data to the communication unit 14 and terminates the process (step S8).

[0047] On the other hand, if the estimated light source color has changed by more than a predetermined range from the previous time (if the determination in step S2 is Yes), in step S3, the control unit 10 determines whether or not the correction result sharing mode is set (step S3).

[0048] If the correction result sharing mode is set (if the determination in step S3 is Yes), in step S4, the control unit 10 determines whether or not it is set on the receiving side (step S4).

[0049] If the receiving side is selected (the determination in step S4 is that the receiving side is selected), in step S5, the control unit 10 receives and refers to the correction result from the transmitting camera 1 (step S5).

[0050] On the other hand, if the setting is for the transmitting side (if the determination in step S4 is that it is for the transmitting side), in step S6, the control unit 10 shares the correction result with the receiving side camera 1 (step S6).

[0051] Next, after the processing in steps S5 and S6, in step S7, the control unit 10 corrects the white balance (step S7).

[0052] On the other hand, if the correction result sharing mode is not set (if the determination in step S3 is No), the control unit 10 performs the process in step S7.

[0053] Subsequently, in step S8, the control unit 10 instructs the image display device 2 to transmit image data to the communication unit 14 and terminates the process (step S8).

[0054] Figure 5 is an explanatory diagram illustrating the problems with the white balance adjustment process when the sharing mode for correction results is not set in the network camera system 100 shown in Figure 1. Figure 5(A) is an example of a photograph taken before white balance adjustment, and Figure 5(B) is an example of the same photograph taken in Figure 5(A) after white balance adjustment.

[0055] Figure 5(A) is the original image of a vacant lot covered with weeds. In the example of Figure 5(A), the majority of the captured image is occupied by green weeds. When the majority of an image is occupied by green objects like this, the average color becomes biased towards green, which can lead to incorrect corrections because the light source color cannot be estimated correctly. Figure 5(B) is an example of an image where the white balance adjustment incorrectly corrected the entire image to an unnatural reddish-brown color due to a miscalculation that the image was illuminated by a green light source.

[0056] Figure 6 is an explanatory diagram showing an example of the results of the white balance adjustment process when the correction result sharing mode is set in the network camera system 100 of Figure 1. Figure 6(A) is an image of a residential area taken by camera 1A, which is set up on the transmitting side. As shown in Figure 6(A), camera 1A can properly adjust the white balance because the captured image contains objects of a relatively wide range of colors. Therefore, the correction result of the white balance adjustment on camera 1A is sent to camera 1B, which is set as the receiving device.

[0057] Figure 6(B) shows an image of a residential area taken by camera 1B, which is set up on the receiving side. Camera 1B adjusts the white balance by referring to the correction results received from Camera 1A. In this way, even if the majority of the images captured by Camera 1B are composed of green objects, the white balance can be adjusted appropriately.

[0058] In this way, the system accepts the setting of the image correction value sharing mode, and when this camera 1 is set as the image correction value receiver, it performs white balance adjustment based on the image correction value shared with other cameras 1 set as the transmitter, thereby realizing a network camera system 100 that can perform appropriate white balance adjustment regardless of the shooting environment.

[0059] (Variation 1) A modified example 1 of the network camera system 100 according to Embodiment 1 of this disclosure will be described. In step S2 of Figure 4, a key feature is that in step S3, the control unit 10 determines whether or not the correction result sharing mode is set and performs white balance adjustment only if the estimated light source color has changed by more than a predetermined range from the previously saved light source color.

[0060] On the other hand, as a modification 1, the control unit 10 may perform the determination in step S3 each time a predetermined check time (for example, 5 minutes) has elapsed, regardless of any change in the estimated light source color.

[0061] In this way, a network camera system 100 can be realized that can properly adjust the white balance regardless of the shooting environment by performing white balance adjustments periodically after a predetermined check time has elapsed, regardless of changes in the estimated light source color.

[0062] (Modification 2) As another modification, the control unit 10 may perform the determination in step S3 each time a predetermined check time (e.g., 5 minutes) has elapsed, provided that the estimated light source color has not changed by more than a predetermined range from the previously saved light source color.

[0063] In this way, by performing white balance adjustment periodically after a predetermined check time has elapsed, provided that the estimated light source color has not changed, a network camera system 100 can be realized that can properly adjust the white balance regardless of the shooting environment.

[0064] (Variation 3) In the determination in step S4 of Figure 4, if the receiver is set, in step S5 the control unit 10 receives and refers to the correction result from the camera 1 on the transmitting side. On the other hand, if the receiver is set to the transmitting side, in step S6 the control unit 10 shares the correction result with the camera 1 on the receiving side.

[0065] On the other hand, as a third variation, instead of referring to / sharing the correction result, the light source color estimated in step S1 may be referred to / shared, and the white balance adjustment may be performed based on that light source color.

[0066] In this way, a network camera system 100 can be realized that accepts the setting of shared mode, and when this camera 1 is set as the receiving side, performs white balance adjustment based on the light source color shared with other cameras 1 set as the transmitting side, thereby enabling appropriate white balance adjustment regardless of the shooting environment.

[0067] A key feature of Figure 2 is that each camera 1 is equipped with a shared mode setting unit 15. On the other hand, as a modified example 4, the system may further include an external management server (not shown), which accepts and manages the shared mode settings for each camera 1.

[0068] In this way, instead of having to configure the shared mode on each individual camera 1, the shared mode settings for each camera 1 can be centrally managed on the management server, enabling a network camera system 100 that can properly adjust the white balance regardless of the shooting environment.

[0069] (Variation 5) Next, a modified example 5 of the network camera system 100 according to Embodiment 1 of this disclosure will be described based on Figure 7. Figure 7 is an explanatory diagram showing an example of white balance adjustment of image data received from multiple cameras 1A to 1I in a modified example 1 of the network camera system in Figure 1.

[0070] Figure 7(A) shows that the screen of the display unit 24 of the image display device 2 is divided into nine sections, with images received from multiple cameras 1A to 1I displayed on each of the smaller screens. Note that the text within each small screen is for explanatory purposes only; in reality, only images received from cameras 1A to 1I are displayed.

[0071] In the example in Figure 7(A), the images from cameras 1A to 1G show the indoors, but the image from camera 1H shows both the outdoors and indoors equally, and the image from camera 1I shows the outdoors. Therefore, the colors displayed are different from those of the images from cameras 1A to 1G.

[0072] Therefore, as shown in Figure 7(B), cameras 1A to 1C are set as the transmitting side, and cameras 1H and 1I are set as the receiving side.

[0073] If multiple cameras 1 are set as the transmitting side, the correction amount obtained by taking the average of the correction results of the multiple cameras 1 may be shared with the receiving camera 1. In the example shown in Figure 7(B), the correction amount obtained by taking the average of the correction results from the transmitting cameras 1A to 1C is shared with the receiving cameras 1H and 1I. On the other hand, the remaining cameras 1D to 1G, for which the correction result sharing mode is not set, perform white balance adjustment based on image data obtained by shooting individually.

[0074] Thus, it is not necessary to set the correction result sharing mode for all cameras 1 that make up the network camera system 100, and any number of cameras 1 can be set as both the transmitting and receiving sides.

[0075] In this way, the system accepts the setting of a shared mode for image correction values, and when this camera 1 is set as the receiver of image correction values, it corrects the image data based on the image correction values ​​shared with other cameras 1 set as the transmitters, thereby realizing a network camera system 100 that can properly adjust the white balance regardless of the shooting environment.

[0076] Preferred embodiments of this disclosure include combinations of any of the embodiments described above. In addition to the embodiments described above, various modifications of this disclosure are possible. These modifications should not be construed as being outside the scope of this disclosure. This disclosure should include the meaning of equivalents to the claims and all variations within that scope. [Explanation of Symbols]

[0077] 1,1A,1B: Camera, 2: Image display device, 3: Network, 10,20: Control unit, 11: Imaging unit, 12,23: Storage unit, 13: Image correction unit, 14,21: Communication unit, 15: Shared mode setting unit, 22: Image processing unit, 24: Display unit, 25: Operation unit

Claims

1. A network camera system consisting of multiple imaging devices, The imaging device is An imaging unit that captures images and acquires image data, An image correction unit calculates predetermined image correction values ​​necessary for image adjustment based on the aforementioned image data, The image correction value is stored in a storage unit, A sharing mode setting unit that accepts the setting of the sharing mode for the aforementioned image correction value, A communication unit that transmits and receives data with other imaging devices, The system comprises an imaging unit, an image correction unit, a storage unit, a shared mode setting unit, and a control unit that controls the communication unit, If the shared mode setting unit has not received a setting for the shared mode of the image correction value, the control unit causes the image data to be corrected by the image correction unit based on the image correction value acquired by the imaging unit. Furthermore, if the shared mode setting unit accepts the setting of the shared mode for the image correction value, and this imaging device is set as the transmitting side for the image correction value, the image correction value acquired by the imaging unit is shared with the other imaging device set as the receiving side, and the image data is corrected by the image correction unit based on the image correction value. On the other hand, the network camera system is characterized in that, when the shared mode setting unit accepts the setting of the shared mode for the image correction value and the imaging device is set as the receiving side for the image correction value, the image data is corrected by the image correction unit based on the image correction value shared with the other imaging device set as the transmitting side.

2. The control unit stores the light source color estimated based on the image correction value acquired by the imaging unit in the storage unit. The network camera system according to claim 1, wherein if the color of the light source changes beyond a predetermined range from the previous time, the image correction unit corrects the image data.

3. The network camera system according to claim 1, wherein the control unit receives the setting of the sharing mode for the image correction value from the sharing mode setting unit, and when the imaging device and one or more other imaging devices are set as the transmitting side for the image correction value, the control unit causes the other imaging devices set as the receiving side to share the average value of the image correction value acquired by the imaging unit and the image correction value acquired by the other imaging devices set as the transmitting side.

4. The network camera system according to claim 1, wherein the image adjustment is white balance adjustment.

5. The system further includes an image display device that displays an image based on image data received from the aforementioned plurality of imaging devices, The network camera system according to claim 1, wherein the control unit causes the image display device to transmit the corrected image data to the communication unit.

Images