Lighting control device and program

Abstract

To control a luminaire in accordance with camera work.SOLUTION: A reflection determining unit 12 of a lighting control device 1-1 generates no reflection (same area) or no reflection (different area) as reflection information on the basis of, position information of a camera 2 and a luminaire 3. Further, the reflection determination unit 12 calculates an image size and image area of the luminaire 3 from an image I, and determines an estimated size and an estimated area of the entire luminaire 3 reflected in the image I on the basis of, the position information of the camera 2 and the luminaire 3 and the like. The reflection determination unit 12 compares both the sizes and both the areas, and determines reflection-presence (perfect match), reflection-presence (partial match / end of field angle), or reflection-presence (partial match / overlap) as reflection information. The lighting control unit 13 generates a control signal or the like for following an orientation of the camera 2 according to the reflection information, and the luminaire 3 operates according to the control signal.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a lighting control device and a program for controlling lighting fixtures installed on a filming location for video production. [Background technology]

[0002] In 2D video production, lighting is an important element in creating a three-dimensional effect for the subject, and various lighting techniques are used on set. However, conventional lighting techniques require an expert to set and adjust lighting equipment according to the subject's predetermined position. As an example of a lighting technique, a lighting system has been proposed that enables desired lighting to match the shooting scene (see, for example, Patent Document 1).

[0003] If the subject's position changes, the lighting equipment must be adjusted each time, and once the lighting equipment is set up, the subject's position is limited and the subject cannot move around freely.

[0004] For example, if a large number of lighting fixtures can be placed densely, the subject can be constantly illuminated, but the cost of equipment increases with the number of lighting fixtures.Even in studios where lighting fixtures cannot be placed densely, if the studio has a catwalk, it is possible to assign personnel to each lighting fixture and adjust the lighting according to the subject's movements, but this requires a large number of human resources.

[0005] Moving lights are known as products that can be controlled externally without directly touching the lighting fixture. Moving lights are lighting fixtures that can be controlled externally to pan, tilt, zoom, dim, turn on and off. Moving lights are also often used on film sets for video productions to add variety to the performance of music programs and the like, and the direction of the light source, color scheme, etc. can be controlled by remote control. As an example of a method using moving lights, a system that simplifies the operation of moving lights has been proposed (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2019 / 54416 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-208222 Summary of the Invention [Problem to be solved by the invention]

[0007] The lighting system described in the aforementioned Patent Document 1 requires that a lighting method appropriate for the shooting scene be determined in advance, but does not propose a method for changing the direction of lighting to match the position of the camera during shooting.

[0008] Furthermore, the aforementioned moving lights are generally used to add accents to a production, and when lighting a subject, precise position adjustments by panning and tilting using a remote control device are required.

[0009] Furthermore, the system described in the aforementioned Patent Document 2 does not propose a lighting method that matches the camerawork.

[0010] Therefore, the present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a lighting control device and program that can control lighting fixtures in accordance with camerawork. [Means for solving the problem]

[0011] In order to solve the above problem, a lighting control device according to claim 1 controls lighting fixtures that illuminate a subject in accordance with camerawork of a camera that photographs the subject, and comprises: a camera information storage unit that stores position information, attitude information, and camera parameters of the camera; a lighting fixture information storage unit that stores position information, attitude information, shape, and size of the lighting fixtures; a reflection determination unit that determines a state of reflection of the lighting fixtures relative to the camera based on the position information, attitude information, and camera parameters of the camera stored in the camera information storage unit, the position information, attitude information, shape, and size of the lighting fixtures stored by the lighting fixture information storage unit, and an image photographed by the camera; and a lighting control unit that generates control signals to control the lighting fixtures in accordance with the state of reflection determined by the reflection determination unit, wherein the reflection determination unit divides a location where the subject is photographed into two areas and determines whether the camera and the lighting fixtures are in the same area or in different areas based on the position information of the camera and the position information of the lighting fixtures. If it is determined that the camera and the lighting fixture are in the same area, reflection information indicating that there is no reflection and that the camera and the lighting fixture are in the same area is generated, and the number of whiteout pixels caused by the illumination of the lighting fixture is calculated from the image as a whiteout pixel count WC. If it is determined that the camera and the lighting fixture are in different areas and the whiteout pixel count WC is smaller than a preset threshold, it is determined that there is no reflection and that the camera and the lighting fixture are in different areas. and when it is determined that the camera and the lighting device are in different areas and that the number of whiteout pixels WC is not smaller than the threshold, generates reflection information with reflection indicating that the reflection is present, and when the reflection determination unit has generated the reflection information with no reflection (same area), the lighting control unit causes the lighting device to follow the attitude of the camera by generating the control signal for panning or tilting to follow the attitude of the camera,When the reflection information without reflection (different area) is generated, the control signal is not generated and the state of the lighting device is maintained, and when the reflection information with reflection is generated, the control signal for reducing the illuminance of the lighting device or the control signal for pan, tilt or roll is generated so that the lighting device is not reflected in the camera.

[0012] Further, in a lighting control device according to claim 2, when the reflection determination unit determines that the camera and the lighting fixture are present in different areas and determines that the number of blown-out pixels WC is not smaller than the threshold, the lighting control device determines, based on the position information, the attitude information and the camera parameters of the camera, the position information, the attitude information, the shape and the size of the lighting fixture, and the image, reflection information indicating that there is reflection (perfect match) indicating that the lighting fixture included in the image and the lighting fixture estimated to be captured in the image completely match, reflection information indicating that there is reflection (partial match / edge of view) indicating that the lighting fixture included in the image and the lighting fixture estimated to be captured in the image partially match and that the lighting fixture is present at an edge of the angle of view of the image, or reflection information indicating that the lighting fixture included in the image and the lighting fixture estimated to be captured in the image partially match and that the lighting fixture is present at an edge of the angle of view of the image. The lighting control unit generates reflection information indicating that the estimated lighting fixture partially matches the target lighting fixture and that the lighting fixture overlaps the subject included in the image (partial match / overlap), and when the reflection information indicating that there is reflection (complete match) is generated by the reflection determination unit, the lighting control unit generates the control signal to reduce the illuminance of the lighting fixture to prevent the lighting fixture from being reflected in the camera, when the reflection information indicating that there is reflection (partial match / edge of field) is generated, the lighting control unit generates the control signal to pan, tilt, or roll to prevent the lighting fixture from being reflected in the camera, and when the reflection information indicating that there is reflection (partial match / overlap) is generated, the lighting control unit generates the control signal to reduce the illuminance of the lighting fixture to prevent the lighting fixture from being reflected in the camera.

[0013] Furthermore, a lighting control device according to claim 3 is the lighting control device according to claim 2, wherein when the reflection determination unit determines that the camera and the lighting fixture are located in different areas and determines that the number of blown-out pixels WC is not smaller than the threshold, the lighting control device calculates the size and area of ​​the lighting fixture from the image as an image size and an image area, and calculates the overall size and area of ​​the lighting fixture shown in the image as an estimated size and an estimated area from the position information, the attitude information, and the camera parameters of the camera, and the position information, the attitude information, the shape, and the size of the lighting fixture. When it is determined that the image size and the estimated size are the same and that the image area and the estimated area are the same, reflection information indicating that reflections are present (perfect match) is generated; when it is determined that the image size and the estimated size are different and that the image area and the estimated area are different, reflection information indicating that reflections are present (partial match / edge of field of view) is generated; and when it is determined that the image size and the estimated size are the same and that the image area and the estimated area are different, reflection information indicating that reflections are present (partial match / overlap).

[0015] Also, claims 4a lighting control device that controls lighting fixtures that illuminate a subject in accordance with camerawork of a plurality of cameras that photograph the subject, the lighting control device comprising: a camera information storage unit that stores position information, attitude information, and camera parameters for each of the plurality of cameras; a lighting fixture information storage unit that stores position information, attitude information, shape, and size of the lighting fixtures; a reflection determination unit that determines a state of reflection of the lighting fixtures relative to a reference camera, using a predetermined one of the plurality of cameras as a reference camera, based on the position information, attitude information, and camera parameters for each of the plurality of cameras stored in the camera information storage unit, the position information, attitude information, shape, and size of the lighting fixtures stored by the lighting fixture information storage unit, and an image photographed by the reference camera; and a lighting control unit that generates a control signal to control the lighting fixtures in accordance with the state of reflection determined by the reflection determination unit, and if it is determined that the plurality of cameras and the lighting fixtures are present in the same area based on the position information of the plurality of cameras and the position information of the lighting fixtures, it sets one of the plurality of cameras as a first reference camera and generates reflection information of no reflection (same area) indicating that there is no reflection and that the first reference camera and the lighting fixtures are present in the same area; if it is determined that at least one of the plurality of cameras is present in a different area from the lighting fixtures and that another camera and the lighting fixture are present in the same area based on the position information of each of the plurality of cameras and the position information of the lighting fixtures, it sets one of the plurality of cameras that is present in a different area from the lighting fixtures as a second reference camera, and calculates the number of blown-out white pixels due to illumination from the lighting fixtures from the image captured by the second reference camera as a blown-out white pixel count WC; and if it is determined that the blown-out white pixel count WC is smaller than a predetermined threshold, it generates reflection information of no reflection (different area) indicating that there is no reflection and that the second reference camera and the lighting fixtures are present in different areas;When it is determined that the number of whiteout pixels WC is not smaller than the threshold, reflection information indicating the presence of reflection is generated, and when the reflection information without reflection (same area) is generated by the reflection determination unit, the lighting control unit generates the pan or tilt control signal for making the lighting device follow the attitude of the first reference camera, so that the lighting device follows the attitude of the first reference camera, when the reflection information without reflection (different area) is generated, the control signal is not generated and the state of the lighting device is maintained, and when the reflection information with reflection is generated, the control signal for reducing the illuminance of the lighting device or the pan, tilt or roll control signal is generated, so that the lighting device is not reflected in the second reference camera.

[0016] Furthermore, claims 5the program causes a computer constituting a lighting control device that controls lighting fixtures that illuminate a subject in accordance with the camerawork of a camera that photographs the subject to function as: a camera information holding unit that holds position information, attitude information, and camera parameters of the camera; a lighting fixture information holding unit that holds position information, attitude information, shape, and size of the lighting fixtures; a reflection determination unit that determines a state of reflection of the lighting fixtures relative to the camera based on the position information, attitude information, and camera parameters of the camera held in the camera information holding unit, the position information, attitude information, shape, and size of the lighting fixtures held by the lighting fixture information holding unit, and an image photographed by the camera; and a lighting control unit that generates a control signal to control the lighting fixtures in accordance with the state of reflection determined by the reflection determination unit, and If it is determined that the camera and the lighting fixture are in the same area, reflection information indicating that there is no reflection and that the camera and the lighting fixture are in the same area is generated, and the number of whiteout pixels caused by the illumination of the lighting fixture is calculated from the image as a whiteout pixel count WC. If it is determined that the camera and the lighting fixture are in different areas and the whiteout pixel count WC is smaller than a preset threshold, it is determined that there is no reflection and that the camera and the lighting fixture are in different areas. generate reflection information without reflection (different area) indicating that the camera and the lighting device are present in the area, and if it is determined that the camera and the lighting device are present in different areas and it is determined that the number of whiteout pixels WC is not smaller than the threshold, generate reflection information with reflection indicating that the reflection is present, and if the reflection information without reflection (same area) is generated by the reflection determination unit, the lighting control unit causes the lighting device to follow the attitude of the camera by generating the control signal for panning or tilting to follow the attitude of the camera,When the reflection information without reflection (different area) is generated, the control signal is not generated and the state of the lighting device is maintained, and when the reflection information with reflection is generated, the control signal for reducing the illuminance of the lighting device or the control signal for pan, tilt or roll is generated so that the lighting device is not reflected in the camera. [Effects of the Invention]

[0017] As described above, according to the present invention, lighting equipment can be controlled in accordance with camerawork, and lighting control suitable for filming can be achieved with a small number of personnel and lighting equipment. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of a lighting system including a lighting control device according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the configuration of a lighting control device according to a first embodiment; [Figure 3] 3 is a block diagram showing an example of the configuration of a reflection determination unit in the first embodiment. FIG. [Figure 4] 10 is a flowchart illustrating an example of processing performed by a reflection determination unit in the first embodiment. [Figure 5] 10 is a flowchart showing an example of a shape match determination process (step S409). [Figure 6] 10 is a flowchart illustrating an example of processing by an illumination control unit in the first embodiment. [Figure 7] (1) is a layout diagram when the camera and lighting equipment are located in studio S2. (2) is a layout diagram when the camera is located in studio S2 and the lighting equipment is located in studio S1. [Figure 8] (1) is a diagram illustrating an image I with reflections (perfect match), (2) is a diagram illustrating an image I with reflections (partial match / edge of view), and (3) is a diagram illustrating an image I with reflections (partial match / overlap). [Figure 9] (1) is a diagram illustrating a safety area SE, and (2) is a diagram illustrating a band area BE. [Figure 10] FIG. 10 is a diagram illustrating an example of the overall configuration of a lighting system including a lighting control device according to a second embodiment. [Figure 11] FIG. 10 is a block diagram showing a configuration example of a lighting control device according to a second embodiment. [Figure 12] FIG. 10 is a block diagram showing an example of the configuration of a reflection determination unit in a second embodiment. [Figure 13] 10 is a flowchart illustrating an example of processing performed by a reflection determination unit in the second embodiment. [Figure 14] 10 is a flowchart illustrating an example of processing by an illumination control unit in the second embodiment. [Figure 15] FIG. 2 is a diagram illustrating a photographable area of ​​a camera. [Figure 16] (1) is a layout diagram when two cameras and lighting equipment are located in studio S2. (2) is a layout diagram when a first camera and lighting equipment are located in studio S2 and a second camera is located in studio S1. DETAILED DESCRIPTION OF THE INVENTION

[0019] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Example 1 First, a description will be given of Example 1. Example 1 is an example in which an area where a camera and lighting fixtures are present is determined, and whether the lighting fixtures are reflected in the camera is determined based on an image captured by the camera, and the lighting fixtures are controlled according to the determination result so that they are not reflected in the camera.

[0020] 1 is a diagram showing an example of the overall configuration of a lighting system including a lighting control device according to Example 1. This lighting system 101 is configured to include a lighting control device 1-1, a camera 2, and lighting fixtures 3. The lighting system 101 is a system that controls lighting fixtures 3 by determining the area in which the camera 2 and lighting fixtures 3 are located, and by determining whether lighting fixtures 3 are reflected in an image of a subject 100 (not shown) captured by camera 2.

[0021] Camera 2 photographs subject 100 (not shown) through camerawork by a cameraman. Camera 2 outputs image I including subject 100, attitude information of camera 2, and camera parameters to lighting control device 1-1. If camera 2 is provided with a pan head such as an electric pan head, attitude information indicating the direction in which camera 2 is facing is output from an encoder mounted on the pan head. The attitude information may also be output from an encoder mounted on the lens. Camera 2 also outputs camera parameters such as focal length, sensor size, resolution, and angle of view.

[0022] Camera 2 may be a camera that is manually operated by a cameraman, or a robot camera that is equipped with an electric camera platform so that panning, tilting, etc. can be controlled externally. Camera 2 may also be a camera whose attitude, angle of view, etc. can be estimated by camera calibration using markers placed in the studio.

[0023] Lighting fixture 3 is a fixture that illuminates subject 100 (not shown) and outputs attitude information of lighting fixture 3 to lighting control device 1-1. Lighting fixture 3 also receives control signals from lighting control device 1-1 to control lighting fixture 3, such as making lighting fixture 3 follow the attitude of camera 2 or preventing lighting fixture 3 from being captured by camera 2. Based on the control signals, lighting fixture 3 performs control such as changing the direction (illumination direction) of lighting fixture 3 or reducing illuminance. Lighting fixture 3 may be a lighting fixture whose color temperature, light distribution, or color scheme can be adjusted based on the control signals.

[0024] If the lighting fixture 3 is provided with a pan head such as an electric pan head, attitude information indicating the direction in which the lighting fixture 3 is facing is output from an encoder mounted on the pan head.

[0025] The lighting control device 1-1 is a device that controls lighting fixtures 3 that illuminate a subject 100 in accordance with the camerawork of a camera 2 performed by a cameraman. The lighting control device 1-1 inputs, from the camera 2, attitude information of the camera 2, as well as camera parameters such as focal length, sensor size, resolution, and angle of view, and inputs an image I captured by the cameraman. The lighting control device 1-1 also inputs, from the lighting fixture 3, attitude information of the lighting fixture 3.

[0026] As described above, the lighting control device 1-1 acquires the attitudes of the camera 2 and the lighting fixtures 3 based on encoder values ​​from encoders mounted on the motorized pan heads provided on the camera 2 and the lighting fixtures 3. Alternatively, the lighting control device 1-1 may estimate the attitudes of the camera 2 and the lighting fixtures 3 by having small sensor cameras attached to the camera 2 and the lighting fixtures 3 capture images of feature point markers placed in the studio. The same applies to the positions of the camera 2 and the lighting fixtures 3, which will be described later.

[0027] Furthermore, the lighting control device 1-1 inputs position information indicating the position of the camera 2 and position information indicating the position of the lighting fixture 3 from a sensor or device not shown, as well as inputs the preset shape (e.g., circular, rectangular, square) and size (e.g., width and height) of the lighting fixture 3.

[0028] The positions of the camera 2 and the lighting fixture 3 may be acquired by a surveying method using, for example, a Global Navigation Satellite System (GNSS) or a beacon, or a method using a surveying instrument such as a laser rangefinder. The shape and size of the lighting fixture 3 may be input by a user operation.

[0029] The lighting control device 1-1 stores position information, etc. input from the camera 2, lighting fixtures 3, etc., and performs determination processes based on the stored position information, etc. of the camera 2 and lighting fixtures 3, such as determining the area in which the camera 2 and lighting fixtures 3 are located, and determining whether the lighting fixtures 3 are reflected when the camera 2 photographs the subject 100.

[0030] Based on the results of the area determination, the reflection determination, etc., the lighting control device 1-1 generates a control signal for following the camera work, a control signal for preventing the lighting fixture 3 from being reflected in the camera 2, etc., and outputs the control signal to the lighting fixture 3. As a result, the lighting fixture 3 operates in accordance with the control signal to prevent itself from being reflected in the camera 2 or to follow the camera work.

[0031] [Lighting control device 1-1 / Example 1] Next, a detailed description will be given of the lighting control device 1-1 shown in Fig. 1. Fig. 2 is a block diagram showing an example of the configuration of the lighting control device 1-1 of Example 1. This lighting control device 1-1 includes a camera information storage unit 10, a lighting fixture information storage unit 11, a reflection determination unit 12, and a lighting control unit 13.

[0032] The camera information storage unit 10 stores the position information of the camera 2 input from a sensor (not shown) or the like, as well as the attitude information and camera parameters of the camera 2 input from the camera 2. As described above, the camera parameters include focal length, sensor size, resolution, angle of view, etc. The various pieces of information stored in the camera information storage unit 10 are updated to the latest information.

[0033] The lighting fixture information storage unit 11 stores the position information of the lighting fixture 3 input from a sensor (not shown) or the like, the attitude information of the lighting fixture 3 input from the lighting fixture 3, and the preset shape and size of the lighting fixture 3. For example, if the lighting fixture 3 is a spotlight, the lighting fixture information storage unit 11 stores a circle as the shape, and if the lighting fixture 3 is a floodlight, the lighting fixture information storage unit 11 stores a rectangle or square as the shape. The various pieces of information stored in the lighting fixture information storage unit 11 are updated to the latest information.

[0034] The reflection determination unit 12 inputs the position information, posture information, and camera parameters of the camera 2 from the camera information storage unit 10, and inputs the position information, posture information, shape, and size of the lighting fixture 3 from the lighting fixture information storage unit 11, and further inputs the image I captured by the camera 2.

[0035] The reflection determination unit 12 determines the state of reflection of the lighting fixtures 3 relative to the camera 2 based on the position information of the camera 2 and the image I. Then, the reflection determination unit 12 generates reflection information indicating the state of reflection based on the determination result, and outputs the reflection information to the lighting control unit 13. Details of the reflection determination unit 12 will be described later.

[0036] The lighting control unit 13 receives the reflection information from the reflection determination unit 12 and the attitude information of the camera 2 from the camera information storage unit 10. Based on the reflection information and the attitude information of the camera 2, the lighting control unit 13 generates control signals for following the camera work, control signals for preventing the lighting fixtures 3 from being reflected in the camera 2, and the like, and outputs the control signals to the lighting fixtures 3. Details of the lighting control unit 13 will be described later.

[0037] (Reflection determination unit 12) Next, a description will be given of the details of the reflection determination unit 12 shown in Fig. 2. Fig. 3 is a block diagram showing an example of the configuration of the reflection determination unit 12 in the first embodiment, and Fig. 4 is a flowchart showing an example of the processing of the reflection determination unit 12 in the first embodiment.

[0038] 3, the reflection determination unit 12 includes a position determination unit 20, a whiteout determination unit 21, and a shape match determination unit 22. The processing of these components will be described below with reference to FIGS.

[0039] The reflection determination unit 12 receives the position information, posture information, and camera parameters of camera 2 from the camera information storage unit 10 (step S401), and also receives the position information, posture information, shape, and size of lighting fixture 3 from the lighting fixture information storage unit 11 (step S402). The reflection determination unit 12 also receives the image I captured by camera 2 (step S403).

[0040] In steps S404 to S409 described below, reflection determination unit 12 determines the state of reflection of lighting fixture 3 relative to camera 2, generates reflection information indicating the state of reflection, and outputs it to lighting control unit 13.

[0041] Specifically, the position determination unit 20 determines whether the camera 2 and the lighting fixture 3 are located in the same area (same area) or in different areas (different areas) based on the input position information of the camera 2 and the lighting fixture 3 (step S404).

[0042] FIG. 7(1) is a layout diagram when camera 2 and lighting fixture 3 are located in studio S2, and FIG. 7(2) is a layout diagram when camera 2 is located in studio S2 and lighting fixture 3 is located in studio S1.

[0043] A cameraman operating camera 2 is to photograph subject 100 illuminated by lighting equipment 3 in a studio. When the location (studio) where subject 100 is to be photographed is divided into two areas, the first area is studio S1 and the second area is studio S2. Specifically, the X and Y axes are set in the horizontal direction of the studio, the Z axis is set in the height direction, and the area on the positive side of the Y axis is studio S1 and the area on the negative side of the Y axis is studio S2. The position coordinates of subject 100 are set to the origin (X,Y,Z)=(0,0,0), which is the center position of the X,Y,Z axes, and the position coordinates of camera 2 are set to (X cam ,Y cam ,Z cam ), and the position coordinate of lighting fixture 3 is (X light ,Y light ,Z light )

[0044] Although not shown, the orientation of the camera 2 is (φ cam ,θ cam ,α cam ) and the position of lighting fixture 3 is (φ light ,θ light ,α light ) φ cam ,φ light is the pan angle, θ cam ,θ light is the tilt angle, α cam ,α light is the roll angle.

[0045] In the layout diagram shown in Figure 7(1), the position coordinates of camera 2 are (X cam ,Y cam ,Z cam )=(0,-Y cam2 ,Z cam2 ), and the position coordinate of lighting fixture 3 is (X light ,Y light ,Z light )=(X light3 ,-Y light3 ,Z light3 ) Y cam2 ,Z cam2 ,X light3 ,Y light3 ,Z light3 is a positive real number.

[0046] In this case, camera 2 and lighting fixture 3 are in studio S2, which is on the negative side of the Y axis, and therefore are in the same area. The same is true if camera 2 and lighting fixture 3 are in studio S1.

[0047] On the other hand, in the layout diagram shown in FIG. 7(2), the position coordinates of camera 2 are (X cam ,Y cam ,Z cam )=(0,-Y cam2 ,Z cam2 ) The position coordinates of lighting fixture 3 are (X light ,Y light ,Z light )=(X light3 ,Y light3 ,Z light3 )

[0048] In this case, camera 2 is located in studio S2 and lighting fixture 3 is located in studio S1, so camera 2 and lighting fixture 3 are located in different areas. The same applies if camera 2 is located in studio S1 and lighting fixture 3 is located in studio S2.

[0049] Studios S1 and S2 are areas obtained by dividing the entire studio by the positive and negative sides of the Y axis, but they may also be areas divided by the positive and negative sides of the X or Y axis, or by a combination of azimuth and elevation angles.

[0050] In the layout diagram of FIG. 7(1), the camera information storage unit 10 shown in FIG. 2 stores (X cam ,Y cam ,Z cam )=(0,-Y cam2 ,Z cam2 ), and the posture information of camera 2 is (φ cam ,θ cam ,α cam )=(0,0,0) as the position information of the lighting fixture 3. light ,Y light ,Z light )=(X light3 ,-Y light3 ,Z light3 ), and the attitude information of lighting fixture 3 is (φ light ,θ light ,α light ) will be held.

[0051] 3 and 4, if the position determination unit 20 determines in step S404 that the camera 2 and the lighting device 3 are in the same area (step S404: same area), it determines that the lighting device 3 is not reflected in the camera 2.

[0052] Then, position determination unit 20 generates reflection information for the area without reflection and outputs it to lighting control unit 13 (step S405), thereby ending the process. In this case, lighting device 3 functions as a key light or fill light, and there is no reflection of lighting device 3 in image I captured by camera 2.

[0053] This reflection information of no reflection (same area) indicates that the lighting fixture 3 is not reflected in the camera 2, and the camera 2 and the lighting fixture 3 are in the same area.

[0054] On the other hand, if the position determination unit 20 determines in step S404 that the camera 2 and the lighting device 3 are in a different area (step S404: different area), it generates a result indicating that the camera 2 and the lighting device 3 are in a different area. Then, the position determination unit 20 outputs the result to the whiteout determination unit 21, and the process proceeds to step S406.

[0055] When the result indicating that the camera 2 and the lighting fixture 3 are in different regions is input from the position determination unit 20, the whiteout determination unit 21 calculates the number of whiteout pixels WC from the image I (step S406).

[0056] Specifically, for each pixel constituting image I, if the pixel value (R, G, B) is equal to or greater than a preset threshold (R1, G1, B1) (R≧R1, G≧G1, and B≧B1), then whiteout determination unit 21 determines that pixel is a pixel in which whiteout appears due to illumination from lighting device 3. For example, (R1, G1, B1)=(255, 255, 255). Then, whiteout determination unit 21 counts the number of whiteout pixels in image I and calculates this as the number of whiteout pixels WC.

[0057] Here, the blown-out highlight determination unit 21 assumes that (illuminated portions of) lighting fixtures 3 are present at the locations of blown-out pixels in image I, calculates the number of blown-out highlight pixels WC, and determines whether or not lighting fixtures 3 are reflected in the image in the processing of step S407, which will be described later. Alternatively, the blown-out highlight determination unit 21 may generate a virtual template of lighting fixtures 3 based on the position information and orientation information of lighting fixtures 3, and determine the position of lighting fixtures 3 by searching for the template by template matching with lighting fixtures 3 reflected in image I. The blown-out highlight determination unit 21 sets the number of pixels at the relevant positions as the blown-out highlight pixel number WC.

[0058] The blown-out highlight determination unit 21 compares the number of blown-out highlight pixels WC with a preset threshold and determines whether the number of blown-out highlight pixels WC is smaller than the threshold (step S407). If the blown-out highlight determination unit 21 determines in step S407 that the number of blown-out highlight pixels WC is smaller than the threshold (step S407: Y), it determines that although the camera 2 and the lighting fixture 3 are in different regions, there is no blown-out highlight, and therefore the lighting fixture 3 is not reflected in the camera 2.

[0059] Then, blown-out highlight determination unit 21 generates reflection information of no reflection (different area) and outputs this to lighting control unit 13 (step S408), thereby ending the process. In this case, lighting device 3 functions as a backlight, and there is no reflection of lighting device 3 in image I captured by camera 2.

[0060] This reflection information of no reflection (different area) indicates that the lighting fixture 3 is not reflected in the camera 2, and the camera 2 and the lighting fixture 3 are located in different areas.

[0061] On the other hand, if it is determined in step S407 that the number of whiteout pixels WC is not smaller than the threshold value (step S407: N), the whiteout determination unit 21 determines that the camera 2 and the lighting device 3 are in different regions and there is whiteout, and therefore determines that the lighting device 3 is reflected in the camera 2. Then, the whiteout determination unit 21 generates a result indicating that there is whiteout, outputs the result to the shape matching determination unit 22, and proceeds to step S409.

[0062] Shape matching determination unit 22 receives a result indicating the presence of whiteout from whiteout determination unit 21. Then, shape matching determination unit 22 performs a shape matching determination process based on the input position information, posture information, and camera parameters of camera 2, position information, posture information, shape, and size of lighting device 3, and image I (step S409), generates reflection information with reflection (described later), outputs this information to lighting control unit 13, and terminates the process.

[0063] This shape matching determination process is a process that determines whether there is a match or partial match by comparing the shape shown in the blown-out image of lighting fixture 3 calculated from image I (the shape of lighting fixture 3 shown in image I) with the overall shape of lighting fixture 3 that will be shown in image I, which is estimated from the position information of camera 2, etc. and the position information of lighting fixture 3, etc. (the overall shape of lighting fixture 3 that is estimated to be shown in image I).

[0064] This reflection information indicating that the lighting fixture 3 is reflected in the camera 2 and that the camera 2 and the lighting fixture 3 are located in different areas.

[0065] (Shape matching determination process (step S409)) Fig. 5 is a flowchart showing an example of the shape matching determination process (step S409) shown in Fig. 4. With reference to Figs. 3 and 5, shape matching determination unit 22 extracts from image I an image of blown-out highlights, which are lighting fixtures 3 included in the image I, calculates the size (width and height) and area representing the shape of the blown-out highlight image, and sets these as the image size and image area of ​​lighting fixtures 3 (step S501). The process of extracting the blown-out highlight image from image I is performed by extracting pixels where blown-out highlights appear in the process of step S406 in Fig. 4.

[0066] The shape matching determiner 22 estimates the size (width and height) and area representing the overall shape of the lighting fixture 3 shown in image I from the position information, orientation information, and camera parameters of the camera 2, and the position information, orientation information, shape, and size of the lighting fixture 3 (step S502).The shape matching determiner 22 then determines this as the estimated size and estimated area of ​​the entire lighting fixture 3.

[0067] This allows for the estimated size and area of ​​the entire lighting fixture 3 corresponding to image I, when the entire lighting fixture 3 is included in image I captured by camera 2. The size of image I is determined by the camera parameters, such as the focal length and sensor size of camera 2. Therefore, the estimated size and area of ​​the entire lighting fixture 3 corresponding to image I can be obtained from the position information, orientation information, and camera parameters of camera 2, as well as the position information, orientation information, shape, and size of lighting fixture 3.

[0068] For example, the shape matching determination unit 22 determines the overall size (width) l of the lighting fixture 3 shown in the image I using the following formula: w may be estimated. [Number 1] l w =(Focal×Light w ×Image w ) / (Distance×Sensor w ) ···(1) The focal length of camera 2 included in the camera parameters is set to Focal, and the size (width) of lighting fixture 3 is set to Light. w The resolution (width) included in the camera parameters is used as the Image w The distance between camera 2 and lighting fixture 3 is Distance, and the sensor size (width) of camera 2 included in the camera parameters is Sensor w The distance Distance between camera 2 and lighting fixture 3 is calculated from the position information of camera 2 and the position information of lighting fixture 3.

[0069] Then, the shape matching determination unit 22 determines the overall size (height) l of the lighting fixture 3 shown in the image I using a height formula similar to the formula (1) above.h The area is estimated by estimating the size (width) and the size (height) of the object.

[0070] In addition, the shape matching determination unit 22 may use a table that stores the overall size (width and height) and area of ​​the lighting fixture 3 shown in image I, corresponding to the position information, posture information and camera parameters of the camera 2, and the position information, posture information, shape and size of the lighting fixture 3.

[0071] The shape matching determination unit 22 compares the image size and image area of ​​the lighting fixture 3 calculated from image I in step S501 with the estimated size and estimated area of ​​the entire lighting fixture 3 estimated from the camera parameters of camera 2, etc. in step S502.

[0072] Specifically, the shape matching determination unit 22 determines whether the image size of the lighting fixture 3 is the same as the estimated size of the entire lighting fixture 3, which is the width and height, and whether the image area of ​​the lighting fixture 3 is the same as the estimated area of ​​the entire lighting fixture 3 (step S503).

[0073] Regarding whether or not the two sizes are the same, if the difference between the two sizes is within a preset threshold, it is determined that they are the same, and if the difference between the sizes exceeds the threshold, it is determined that they are not the same. The same applies to the area.

[0074] If it is determined in step S503 that the image size and estimated size of lighting fixture 3 are the same and that the image area and estimated area are the same (step S503: Y), shape matching determination unit 22 determines that the entire lighting fixture 3 is reflected in camera 2. In other words, shape matching determination unit 22 determines that camera 2 and lighting fixture 3 are in different regions, there is whiteout, the entire lighting fixture 3 is reflected in camera 2, and the lighting fixture 3 (size and area) obtained from image I and the lighting fixture 3 (size and area) obtained from the camera parameters of camera 2 and the like completely match.

[0075] Then, shape match determiner 22 generates reflection information indicating that reflection is present (perfect match), outputs this to lighting controller 13 (step S504), and ends the process. In this case, lighting device 3 functions as a backlight, but lighting device 3 is reflected in image I captured by camera 2.

[0076] This reflection information of reflection (exact match) indicates that the lighting fixture 3 is reflected in the camera 2, and that the lighting fixture 3 (size and area) included in image I matches the lighting fixture 3 (size and area) estimated to be reflected in image I exactly.

[0077] Fig. 8(1) is a diagram illustrating an image I with a reflection (perfect match). As shown in Fig. 8(1), there is no subject 100 that could be an obstacle between camera 2 and lighting fixture 3, and the entire lighting fixture 3 is reflected in camera 2 together with subject 100, with the lighting fixture 3 being blown out.

[0078] In this case, the shape matching determination unit 22 determines that the image size of the lighting fixture 3 and the estimated size of the entire lighting fixture 3 are the same, and that the image area of ​​the lighting fixture 3 and the estimated area of ​​the entire lighting fixture 3 are the same, and reflection information with reflection (perfect match) is generated.

[0079] Returning to Figures 3 and 5, if the shape matching determination unit 22 determines in step S503 that the image size and estimated size of the lighting device 3 are not the same or that the image area and estimated area are not the same (step S503: N), it proceeds to step S505.

[0080] Moving on from step S503(N), shape match determination unit 22 determines whether the image size of lighting fixture 3 differs from the estimated size of the entire lighting fixture 3 and whether the image area of ​​lighting fixture 3 differs from the estimated area of ​​the entire lighting fixture 3 (step S505). The difference between the image size of lighting fixture 3 and the estimated size of the entire lighting fixture 3 refers to a case where either one or both of the width and height differ.

[0081] If it is determined in step S505 that the image size and estimated size of lighting fixture 3 differ and that the image area and estimated area differ (step S505: Y), shape matching determination unit 22 determines that part of lighting fixture 3 is reflected in camera 2. In other words, shape matching determination unit 22 determines that camera 2 and lighting fixture 3 are in different regions, there is whiteout, part of lighting fixture 3 is reflected at the edge of the field of view of camera 2, and the lighting fixture 3 obtained from image I partially matches the lighting fixture 3 obtained from the camera parameters of camera 2, etc.

[0082] Then, shape match determiner 22 generates reflection information indicating that reflection is present (partial match / edge of field), outputs this to lighting controller 13 (step S506), and ends the process. In this case, lighting device 3 functions as a backlight, but part of lighting device 3 is reflected at the edge of the field of image I captured by camera 2.

[0083] This reflection information of "reflection present (partial match / edge of view)" indicates that the lighting fixture 3 is reflected in the camera 2, the lighting fixture 3 (size and area) included in image I partially matches the lighting fixture 3 (size and area) estimated to be reflected in image I, and the lighting fixture 3 is present at the edge of the view of image I.

[0084] Fig. 8(2) is a diagram illustrating image I with reflections (partial match / edge of field). As shown in Fig. 8(2), there is no subject 100 that could be an obstacle between camera 2 and lighting fixture 3, and subject 100 is reflected in camera 2. In addition, part of lighting fixture 3 (portion α of lighting fixture 3 other than the hatched portion) is reflected in the edge of the field of camera 2, resulting in blown-out highlights of lighting fixture 3.

[0085] In this case, the shape matching determination unit 22 determines that the image size of the lighting fixture 3 and the estimated size of the entire lighting fixture 3 are different, and that the image area of ​​the lighting fixture 3 and the estimated area of ​​the entire lighting fixture 3 are different, and generates reflection information indicating that reflections are present (partial match / edge of field of view).

[0086] Note that, based on the determination in step S505(Y), shape matching determination unit 22 determines that part of lighting fixture 3 is reflected in the edge of the field of view of camera 2, i.e., that lighting fixture 3 is cut off at the edge of the field of view of image I. However, shape matching determination unit 22 may determine that lighting fixture 3 is cut off at the edge of the field of view of image I using another determination method.

[0087] FIG. 9(1) is a diagram explaining the safety area SE. The area where lighting fixture 3 is completely within the angle of view of image I is defined as the safety area SE. If lighting fixture 3 is reflected in a location outside the safety area SE in the width direction of lighting fixture 3, it is determined that lighting fixture 3 is cut off at the edge of the angle of view, as shown in the following formula. [Number 2] lw w <safetyarea w <Image w -lw w ···(2) The size (width) of the image of the blown-out light fixture 3 is lw w The width of the preset safety area SE is defined as safetyarea w The resolution (width) included in the camera parameters is used as the Image w Let's say.

[0088] For example, a case will be described in which the size (width) of the blown-out image of the lighting fixture 3 is reflected on the left side of the safety area SE. First, the shape matching determination unit 22 determines the size (width) lw of the blown-out image. w and the width outside the safety area SE (Image w -safetyarea w The shape matching determination unit 22 compares the result of division obtained by dividing the size (width) lw of the blown-out image by 2 (here, the width outside the safety area SE is assumed to be equal on both sides). w If is smaller than the division result, the size (width) lw of the whiteout image is set in the left and right areas outside the safety area SE. w It is considered that there is a possibility that [Number 3] lw w <(Image w -safetyarea w ) / 2 ···(3)

[0089] Next, taking the width direction (right direction in FIG. 9) of the image I as the x-axis and the height direction (upward direction in FIG. 9) as the y-axis, the x-axis coordinate value of the leftmost end of the safety area SE is s1, and the width lw of the size (width) of the image of the white overshoot w Let the x-axis coordinate value of the right end in be lw1. The shape matching determination unit 22 compares the coordinate values s1 and lw1, and when lw1 is larger than s1 (s1 < lw1), the size (width) lw of the image of the white overshoot is within the left region outside the safety area SE w fits, and it is determined that the lighting fixture 3 is cut off at the corner of the viewing angle.

[0090] On the other hand, regarding the height direction of the lighting fixture 3, when the lighting fixture 3 is reflected in a location outside the safety area SE, it is determined that the lighting fixture 3 is cut off at the corner of the viewing angle. Let the size (height) of the image of the white overshoot, which is the lighting fixture 3, be lw h and the preset height of the safety area SE be safetyarea h and the resolution (height) included in the camera parameters be Image h be.

[0091] For example, the case where the size (height) of the image of the white overshoot of the lighting fixture 3 is reflected above the safety area SE will be described as an example. First, the shape matching determination unit 22 determines the size (height) lw of the image of the white overshoot h and the division result obtained by dividing the width outside the safety area SE (Image<000009३] -safetyarea h ) by 2 (here, it is assumed that the height outside the safety area SE is evenly distributed above and below). The shape matching determination unit 22 compares the size (height) lw of the image of the white overshoot with the division result as follows. When the size (height) lw of the image of the white overshoot h [[ID=३7]]is smaller than the division result, it is determined that the size (height) lw of the image of the white overshoot may fit within the upper and lower regions outside the safety area SE h . [Number 4] lw h <(Image h -safetyarea h ) / twenty four)

[0092] Next, the y-axis coordinate value of the top edge of the safety area SE is set to s2, and the size (width) of the whiteout image is set to lw. w The shape matching determination unit 22 compares the y-axis coordinate value s2 with lw2, and if lw2 is smaller than s2 (s2>lw2), it determines that the size (width) of the whiteout image is lw in the upper region outside the safety area SE. h is within the viewing angle, and it is determined that lighting fixture 3 is cut off at the edge of the viewing angle.

[0093] Returning to Figures 3 and 5, if the shape matching determination unit 22 determines in step S505 that the image size and estimated size of the lighting device 3 do not differ or that the image area and estimated area do not differ (step S505: N), it proceeds to step S507.

[0094] The shape matching determination unit 22 proceeds from step S505(N) and determines whether the image size of the lighting fixture 3 and the estimated size of the entire lighting fixture 3 are the same, and whether the image area of ​​the lighting fixture 3 and the estimated area of ​​the entire lighting fixture 3 are different (step S507).

[0095] If it is determined in step S507 that the image size and estimated size of lighting fixture 3 are the same but that the image area and estimated area are different (step S507: Y), shape matching determination unit 22 determines that a portion of lighting fixture 3 is reflected in camera 2 and that lighting fixture 3 overlaps subject 100 (lighting fixture 3 and subject 100 overlap on the optical axis of camera 2). In other words, shape matching determination unit 22 determines that camera 2 and lighting fixture 3 are in different regions, there is whiteout, lighting fixture 3 is reflected in camera 2 overlapping subject 100, and lighting fixture 3 obtained from image I partially matches lighting fixture 3 obtained from the camera parameters of camera 2, etc.

[0096] Then, shape match determiner 22 generates reflection information indicating that reflection is present (partial match / overlap), outputs this to lighting controller 13 (step S508), and ends the process. In this case, lighting device 3 functions as a backlight, but image I captured by camera 2 includes a reflection of part of lighting device 3 overlapping subject 100.

[0097] This reflection information of reflection (partial match / overlap) indicates that the lighting fixture 3 is reflected in the camera 2, the lighting fixture 3 (size and area) included in image I partially matches the lighting fixture 3 (size and area) estimated to be reflected in image I, and the lighting fixture 3 overlaps with the subject 100 included in image I (behind the subject 100).

[0098] Fig. 8(3) is a diagram illustrating image I with reflections (partial match / overlap). As shown in Fig. 8(3), subject 100 exists as an obstacle between camera 2 and lighting fixture 3, and part of lighting fixture 3 is reflected in the center of camera 2 while overlapping subject 100, resulting in blown-out highlights in the area of ​​lighting fixture 3.

[0099] In this case, the shape matching determination unit 22 determines that the image size of the lighting fixture 3 and the estimated size of the entire lighting fixture 3 are the same, and that the image area of ​​the lighting fixture 3 and the estimated area of ​​the entire lighting fixture 3 are different, and generates reflection information indicating that there is reflection (partial match / overlap).

[0100] Returning to Figures 3 and 5, if in step S507 the shape matching determination unit 22 determines that the image size and estimated size of the lighting device 3 are not the same or that the image area and estimated area are not different (step S507: N), it displays an error and terminates the processing.

[0101] (Lighting control unit 13) Next, the lighting control unit 13 shown in Fig. 2 will be described in detail. Fig. 6 is a flowchart showing an example of processing by the lighting control unit 13 in the first embodiment. The lighting control unit 13 receives reflection information from the reflection determination unit 12 and also receives posture information of the camera 2 from the camera information storage unit 10 (step S601). The posture information of the camera 2 is the latest information and is updated over time.

[0102] The lighting control unit 13 determines whether the reflection information indicates no reflection (same area) (step S602). If the lighting control unit 13 determines in step S602 that the reflection information indicates no reflection (same area) (step S602: Y), it generates a control signal for following the attitude of the camera 2 and outputs it to the lighting device 3 (step S603), and ends the process. This control signal is, for example, a signal for panning or tilting.

[0103] For example, when the lighting control unit 13 determines that only the pan angle of the camera 2 has changed by 10° in the positive direction (φ cam =10), a control signal is generated to change the pan angle of lighting fixture 3 by +10°.

[0104] As a result, the lighting fixture 3 operates based on the control signal and follows the attitude of the camera 2.

[0105] On the other hand, if the illumination control unit 13 determines in step S602 that the reflection information does not indicate no reflection (same area) (step S602: N), the illumination control unit 13 proceeds to step S604.

[0106] Moving on from step S602(N), lighting control unit 13 determines whether the reflection information indicates no reflection (different area) (step S604). If lighting control unit 13 determines in step S604 that the reflection information indicates no reflection (different area) (step S604:Y), lighting control unit 13 does not control lighting device 3 and causes lighting device 3 to maintain its current state. In other words, lighting control unit 13 does not generate or output a control signal (step S605) and ends the process.

[0107] As a result, the lighting fixture 3 maintains its current state without performing any operation such as following the attitude of the camera 2.

[0108] On the other hand, if the illumination control unit 13 determines in step S604 that the reflection information does not indicate no reflection (different area) (step S604: N), the process proceeds to step S606.

[0109] The lighting control unit 13 proceeds from step S604(N) and determines whether the reflection information indicates that a reflection is present (complete match) (step S606). If the lighting control unit 13 determines in step S606 that the reflection information indicates that a reflection is present (complete match) (step S606:Y), the lighting control unit 13 generates a control signal to adjust the illuminance of the lighting device 3 so that the lighting device 3 is not reflected in the camera 2, and outputs the control signal to the lighting device 3 (step S607), and then ends the process.

[0110] In this case, even if lighting control unit 13 generates and outputs a control signal to change the attitude of lighting device 3, it is difficult to prevent lighting device 3 from appearing in camera 2. That is, even if the attitude of lighting device 3 is changed, it is difficult to determine in step S407 of FIG. 4 that the number of whiteout pixels WC is smaller than the threshold, that there is no whiteout in image I, and that lighting device 3 is not appearing in camera 2. For this reason, lighting control unit 13 generates a control signal to adjust the illuminance of lighting device 3 to lower its illuminance. The control signal may also be a signal to turn off the illuminance.

[0111] Furthermore, if the lighting fixture 3 is equipped with a lifting mechanism (for example, a telescopic pole or crane mechanism) that changes the height of the lighting fixture 3, the lighting control unit 13 may generate a control signal to raise or lower the lighting fixture 3. For example, when the lighting fixture 3 is attached to a pole installed on the ground, the lifting is controlled by a hydraulic or pneumatic system that uses a telescopic drive, or when the lighting fixture 3 is hung from a studio baton, a winch is used. For example, when image I is divided into two, top and bottom, and the lighting fixture 3 appears in the upper part of image I, a control signal for raising the lighting fixture 3 is generated, and stepwise lifting control is performed. The above-mentioned control is performed until it is determined in step S407 of FIG. 4 that the number of blown-out pixels WC is smaller than the threshold.

[0112] By using such a control signal, lighting fixture 3 is prevented from appearing in camera 2; that is, in step S407 of FIG. 4, it is determined that the number of whiteout pixels WC is smaller than the threshold value, there is no whiteout in image I, and lighting fixture 3 is not reflected in camera 2.

[0113] As a result, lighting fixture 3 operates based on the control signal, and lighting control unit 13 can change lighting fixture 3 from being reflected in camera 2 to not being reflected in camera 2.

[0114] On the other hand, if the illumination control unit 13 determines in step S606 that the reflection information does not indicate the presence of reflection (complete match) (step S606: N), the illumination control unit 13 proceeds to step S608.

[0115] The lighting control unit 13 proceeds from step S606(N) and determines whether the reflection information indicates that reflections are present (partial match / edge of the angle of view) (step S608). If the lighting control unit 13 determines in step S608 that the reflection information indicates that reflections are present (partial match / edge of the angle of view) (step S608:Y), the lighting control unit 13 generates and outputs to the lighting device 3 a pan, tilt, or roll control signal for changing the attitude of the lighting device 3 so that the lighting device 3 is not reflected in the camera 2 (so that the lighting device 3 is cut off from the image I) (step S609), and ends the process.

[0116] For example, when lighting control unit 13 receives information from reflection determination unit 12 indicating that lighting fixture 3 is reflected at the left or right edge of image I, lighting control unit 13 generates a pan control signal to prevent lighting fixture 3 from being reflected in camera 2. Furthermore, when lighting control unit 13 receives information from reflection determination unit 12 indicating that lighting fixture 3 is reflected at the top or bottom edge of image I, lighting control unit 13 generates a tilt control signal to prevent lighting fixture 3 from being reflected in camera 2. In this case, reflection determination unit 12 generates information indicating that reflection is present (partial match / edge of view) and at which edge of image I lighting fixture 3 is reflected, and outputs the information to lighting control unit 13.

[0117] By using such a control signal, lighting fixture 3 is prevented from appearing in camera 2; that is, in step S407 of FIG. 4, it is determined that the number of whiteout pixels WC is smaller than the threshold value, there is no whiteout in image I, and lighting fixture 3 is not reflected in camera 2.

[0118] As a result, lighting fixture 3 operates based on the control signal, and lighting control unit 13 can change lighting fixture 3 from being reflected in camera 2 to not being reflected in camera 2.

[0119] If the lighting fixture 3 is capable of controlling the barn door, the lighting control unit 13 determines whether the lighting fixture 3 is reflected in a preset barn door area BE in the image I. If the lighting control unit 13 determines that the lighting fixture 3 is reflected, it generates a control signal to close the barn door and outputs it to the lighting fixture 3. When the lighting control unit 13 inputs the control signal to close the barn door, the lighting fixture 3 closes the barn door.

[0120] 9(2) is a diagram explaining the band door area BE. The area between the lighting fixture 3 and a point a predetermined distance away from the edge of the image I (a length in the width and height directions that is equal to or less than the distance between the edge of the image I and the edge of the safety area SE) is defined as the band door area BE.

[0121] If all or part of image I of lighting fixture 3 is reflected in the band door area BE, lighting fixture 3 will close the band door. Note that when lighting fixture 3 closes the band door, the shape of lighting fixture 3 changes, so lighting fixture information storage unit 11 needs to store the shape according to the adjusted angle of the band door.

[0122] On the other hand, if the illumination control unit 13 determines in step S608 that the reflection information does not indicate the presence of reflection (partial match / edge of angle of view) (step S608: N), the process proceeds to step S610.

[0123] The lighting control unit 13 proceeds from step S608(N) to determine whether the reflection information indicates that reflections are present (partial match / overlap) (step S610). If the lighting control unit 13 determines in step S610 that the reflection information indicates that reflections are present (partial match / overlap) (step S610:Y), the lighting control unit 13 generates a control signal to adjust the illuminance of the lighting device 3, etc., and outputs the control signal to the lighting device 3 (step S611), similar to the process of step S607, in order to prevent the lighting device 3 from being reflected in the camera 2, and then ends the process.

[0124] In this case, even if lighting control unit 13 generates and outputs a control signal to change the attitude of lighting fixture 3, it is difficult to prevent lighting fixture 3 from appearing in camera 2. For this reason, lighting control unit 13 generates a control signal to adjust the illuminance of lighting fixture 3 to lower the illuminance, as in the process of step S607. Furthermore, if lighting fixture 3 is equipped with an elevation mechanism that changes the height of lighting fixture 3, lighting control unit 13 may generate a control signal to raise or lower lighting fixture 3.

[0125] By using such a control signal, lighting fixture 3 is prevented from appearing in camera 2; that is, in step S407 of FIG. 4, it is determined that the number of whiteout pixels WC is smaller than the threshold value, there is no whiteout in image I, and lighting fixture 3 is not reflected in camera 2.

[0126] As a result, lighting fixture 3 operates based on the control signal, and lighting control unit 13 can change lighting fixture 3 from being reflected in camera 2 to not being reflected in camera 2.

[0127] On the other hand, if the illumination control unit 13 determines in step S610 that the reflection information does not indicate the presence of reflection (partial match / overlap) (step S610: N), it ends the process.

[0128] As described above, according to the lighting control device 1-1 of Example 1, when the reflection determination unit 12 determines that the camera 2 and the lighting fixture 3 are present in the same area based on the position information of the camera 2 and the lighting fixture 3, it generates reflection information without reflection (same area).

[0129] If the reflection determination unit 12 determines that the camera 2 and the lighting fixture 3 are in a different area, it calculates the number of whiteout pixels WC from the image I, and if it determines that the number of whiteout pixels WC is smaller than a threshold, it generates reflection information without reflection (different area).

[0130] The reflection determination unit 12 calculates the image size and image area of ​​the lighting fixture 3 from image I, and also determines the estimated overall size and estimated area of ​​the lighting fixture 3 appearing in image I based on the position information of the camera 2 and the lighting fixture 3, etc.

[0131] If the reflection determination unit 12 determines that both sizes and both areas are the same, it generates reflection information indicating that reflections are present (complete match); if it determines that both sizes and both areas are different, it generates reflection information indicating that reflections are present (partial match / edge of field of view); and if it determines that both sizes are the same and both areas are different, it generates reflection information indicating that reflections are present (partial match / overlap).

[0132] If the reflection information indicates no reflection (same area), the lighting control unit 13 outputs a control signal (pan, tilt) to the lighting device 3 to make it follow the attitude of the camera 2, and if the reflection information indicates no reflection (different area), the lighting control unit 13 does not output a control signal.

[0133] The lighting control unit 13 outputs a control signal for adjusting the illuminance when the reflection information indicates that reflections are present (complete match), and outputs control signals for pan, tilt, and roll when the reflection information indicates that reflections are present (partial match / edge of angle of view).Furthermore, the lighting control unit 13 outputs a control signal for adjusting the illuminance when the reflection information indicates that reflections are present (partial match / overlap).

[0134] As a result, the lighting fixtures 3 that receive the control signals operate to follow the posture of the camera 2 when the reflection information indicates no reflection (same area), but do not operate when the reflection information indicates no reflection (different area) because no control signal is received. Furthermore, when the reflection information indicates that a reflection is present (exact match), the lighting fixtures 3 operate to reduce the illuminance so that the lighting fixtures are not reflected by the camera 2. Furthermore, when the reflection information indicates that a reflection is present (partial match / edge of the angle of view), the lighting fixtures 3 operate in the pan, tilt, and roll directions so that the lighting fixtures are not reflected by the camera 2. Furthermore, when the reflection information indicates that a reflection is present (partial match / edge of the angle of view), the lighting control unit 13 operates to reduce the illuminance so that the lighting fixtures are not reflected by the camera 2.

[0135] This allows lighting fixtures 3 to be controlled in accordance with the camerawork that follows subject 100 in the imaging space. Furthermore, lighting control suitable for imaging can be achieved with a small number of personnel and lighting fixtures 3.

[0136] Example 2 Next, a description will be given of Example 2. Example 2 is an example in which whether lighting fixture 3 is reflected in camera 2 is determined based on information such as the angle of view of camera 2, and lighting fixture 3 is controlled according to the determination result so that lighting fixture 3 is not reflected in camera 2.

[0137] 10 is a diagram showing an example of the overall configuration of a lighting system including a lighting control device of Example 2. This lighting system 102 is configured with a lighting control device 1-2, a camera 2, and a lighting fixture 3. The lighting system 102 is a system that determines whether or not the lighting fixture 3 is reflected based on information such as the angle of view included in the camera parameters of the camera 2, without using an image I captured by the camera 2, and controls the lighting fixture 3.

[0138] Camera 2 outputs the attitude information and camera parameters of camera 2 to lighting control device 1-2. Like camera 2 shown in Fig. 1, camera 2 is provided with, for example, an electric pan head, and an encoder mounted on the pan head or lens outputs attitude information indicating the facing direction of camera 2. Camera 2 also outputs camera parameters such as focal length, sensor size, resolution, and angle of view.

[0139] The lighting fixture 3 is similar to the lighting fixture 3 shown in FIG. 1, and therefore a detailed description thereof will be omitted here.

[0140] 1, lighting control device 1-2 receives, from camera 2, attitude information of camera 2 as well as camera parameters such as focal length, sensor size, resolution, and angle of view. Lighting control device 1-2 also receives, from lighting fixture 3, attitude information of lighting fixture 3. Furthermore, lighting control device 1-2 receives, from a sensor or device (not shown), position information indicating the position of camera 2 and position information indicating the position of lighting fixture 3.

[0141] The lighting control device 1-2 stores position information, posture information, and camera parameters input from the camera 2 and lighting fixture 3, etc., and determines whether the lighting fixture 3 is reflected when the camera 2 photographs the subject 100 based on the angle of view information, etc. contained in the stored camera parameters.

[0142] Based on the determination result, lighting control device 1-2 generates a control signal to prevent lighting device 3 from appearing in camera 2's image, and outputs the control signal to lighting device 3. As a result, lighting device 3 operates in accordance with the control signal so as not to appear in camera 2's image.

[0143] [Lighting control device 1-2 / Example 2] Next, the lighting control device 1-2 shown in Fig. 10 will be described in detail. Fig. 11 is a block diagram showing an example of the configuration of a lighting control device 1-2 according to Example 2. This lighting control device 1-2 includes a camera information storage unit 14, a lighting device information storage unit 15, a reflection determination unit 16, and a lighting control unit 17.

[0144] The camera information storage unit 14 stores the position information of the camera 2 input from a sensor (not shown) or the like, the attitude information of the camera 2 input from the camera 2, and the camera parameters. As described above, the camera parameters include the focal length, sensor size, resolution, angle of view, etc. The various pieces of information stored in the camera information storage unit 14 are updated to the latest information. The camera information storage unit 14 may also calculate the angle of view from the focal length and sensor size included in the camera parameters and store this.

[0145] The lighting fixture information storage unit 15 stores the position information of the lighting fixture 3 input from a sensor (not shown) or the like, and the attitude information of the lighting fixture 3 input from the lighting fixture 3. The various information stored in the lighting fixture information storage unit 15 is updated to the latest information.

[0146] The reflection determination unit 16 receives the position information, attitude information, and camera parameters of the camera 2 from the camera information storage unit 14 , and also receives the position information of the lighting fixture 3 from the lighting fixture information storage unit 15 .

[0147] Reflection determination unit 16 determines whether or not there is a reflection of lighting device 3 based on the position information, attitude information, and angle of view information included in the camera parameters of camera 2, and position information of lighting device 3, generates reflection information based on the determination result, and outputs the reflection information to lighting control unit 17. Details of reflection determination unit 16 will be described later.

[0148] Illumination control unit 17 receives the reflection information from reflection determination unit 16, generates a control signal based on the reflection information to prevent lighting fixture 3 from being reflected in camera 2, and outputs the control signal to lighting fixture 3. Details of illumination control unit 17 will be described later.

[0149] (Reflection determination unit 16) Next, a description will be given of the details of the reflection determination unit 16 shown in Fig. 11. Fig. 12 is a block diagram showing an example of the configuration of the reflection determination unit 16 in the second embodiment, and Fig. 13 is a flowchart showing an example of the processing of the reflection determination unit 16 in the second embodiment.

[0150] 12, the reflection determination unit 16 includes a shooting area calculation unit 23 and a determination unit 24. The processing of these components will be described below with reference to FIGS.

[0151] The reflection determination unit 16 receives the position information, attitude information, and camera parameters of the camera 2 from the camera information storage unit 14 (step S1301), and also receives the position information of the lighting fixture 3 from the lighting fixture information storage unit 15 (step S1302).

[0152] In the processing of steps S1303 to S1309 described below, reflection determination unit 16 determines the state of reflection of lighting fixture 3 relative to camera 2, generates reflection information indicating the state of reflection, and outputs it to lighting control unit 17.

[0153] The photographing area calculation unit 23 of the reflection determination unit 16 calculates the distance d between the camera 2 and the lighting fixture 3 from the position information of the camera 2 and the position information of the lighting fixture 3 (step S1303).

[0154] The shooting area calculation unit 23 calculates the horizontal distance b1 using the distance d, the posture information of camera 2, and the field of view information (horizontal field of view a1) included in the camera parameters using the following formula, thereby obtaining the horizontal range (the distance that defines the horizontal range) ±b1 that can be shot by camera 2 (step S1304). [Number 5] b1=d×tan(a1 / 2) (5)

[0155] Similarly, the shooting area calculation unit 23 calculates the vertical distance b2 using the distance d, the posture information of camera 2, and the field of view information (vertical field of view a2) included in the camera parameters, thereby obtaining the vertical range (the distance defining the vertical range) ±b2 that can be shot by camera 2 (step S1305).

[0156] The photographic area calculation unit 23 calculates the area that can be photographed by the camera 2 (photographable area) based on the horizontal range ±b1 that can be photographed by the camera 2 calculated in step S1304 and the vertical range ±b2 that can be photographed by the camera 2 calculated in step S1305 (step S1306). Then, the photographic area calculation unit 23 outputs information about the photographable area to the determination unit 24.

[0157] The photographing area calculation unit 23 calculates the horizontal distance b1 using the distance d, the attitude information of the camera 2, and the angle of view information (horizontal angle of view a1) included in the camera parameters, according to the above formula (5). Alternatively, the photographing area calculation unit 23 may calculate the horizontal angle of view a1 from the sensor size (width) and focal length included in the camera parameters, and calculate the horizontal distance b1 according to the above formula (5).

[0158] Specifically, the sensor size (width) of camera 2 is Sensor w , and the focal length is Focal, the horizontal angle of view a1 is calculated by the following formula. [Number 6] a1=2×atan(Sensor w / (2×Focal)) ···(6)

[0159] Similarly, the shooting area calculation unit 23 may calculate the vertical angle of view a2 using the sensor size (height) and focal length included in the camera parameters, and calculate the vertical distance b2 using the distance d, the posture information of the camera 2, and the calculated vertical angle of view a2.

[0160] 15 is a diagram illustrating the captureable area of ​​camera 2. The optical axis of camera 2 is set based on the posture information of camera 2, and the point on the optical axis of camera 2 that is a distance d from the position of camera 2 is set as origin O. Then, a rectangle with a width (horizontal) of 2×b1 and a height (vertical) of 2×b2 is assumed to be centered at origin O and perpendicular to the optical axis, and the vertices of this rectangle are set as R1, R2, R3, and R4.

[0161] The area that can be photographed by camera 2 is a quadrangular pyramid area that is surrounded by lines that extend from the position of camera 2 and pass through vertices R1, R2, R3, and R4.

[0162] That is, the photographing area calculation unit 23 sets the optical axis of the camera 2 from the attitude information of the camera 2, and identifies the origin O of the distance d on the optical axis from the position of the camera 2 indicated by the position information of the camera 2. Then, the photographing area calculation unit 23 sets a rectangle having a width (horizontal) of 2×b1 and a height (vertical) of 2×b2, centered on the origin O and perpendicular to the optical axis, sets a quadrangular pyramid surrounded by straight lines passing through the vertices of the rectangle from the position of the camera 2, and sets the inside of the sides of the quadrangular pyramid as the photographable area of ​​the camera 2.

[0163] 12 and 13, the determination unit 24 receives the position information of the lighting fixture 3 from the lighting fixture information storage unit 15 and also receives the information on the captureable area from the imaging area calculation unit 23. Then, based on the position information of the lighting fixture 3 and the captureable area, the determination unit 24 determines whether the lighting fixture 3 is present in the captureable area of ​​the camera 2 (step S1307).

[0164] If the determination unit 24 determines in step S1307 that the lighting fixture 3 is present in the capture area of ​​the camera 2 (step S1307: Y), it generates reflection information with reflection, outputs this to the lighting control unit 17 (step S1308), and terminates the processing.

[0165] On the other hand, if the determination unit 24 determines in step S1307 that the lighting fixture 3 is not present in the capture area of ​​the camera 2 (step S1307: N), it generates reflection information without reflection, outputs this to the lighting control unit 17 (step S1309), and terminates the processing.

[0166] (Lighting control unit 17) Next, the illumination control unit 17 shown in Fig. 11 will be described in detail. Fig. 14 is a flowchart showing an example of processing by the illumination control unit 17 in Example 2. The illumination control unit 17 receives reflection information from the reflection determination unit 16 (step S1401).

[0167] The lighting control unit 17 determines whether the reflection information indicates that a reflection is present (step S1402). If the lighting control unit 17 determines in step S1402 that the reflection information indicates that a reflection is present (step S1402: Y), the lighting control unit 17 generates a control signal to adjust the illuminance of the lighting device 3, etc., and outputs the control signal to the lighting device 3 (step S1403), similar to the process of step S607 in FIG. 6, in order to prevent the lighting device 3 from being reflected in the camera 2, and then ends the process.

[0168] As a result, lighting fixture 3 operates based on the control signal, and lighting control unit 17 can change lighting fixture 3 from being reflected in camera 2 to not being reflected in camera 2.

[0169] On the other hand, if the illumination control unit 17 determines in step S1402 that the reflection information does not indicate the presence of reflection (step S1402: N), the process proceeds to step S1404.

[0170] Moving on from step S1402(N), lighting control unit 17 determines whether the reflection information indicates no reflection (step S1404). If lighting control unit 17 determines in step S1404 that the reflection information indicates no reflection (step S1404:Y), lighting control unit 17 does not control lighting device 3 and causes lighting device 3 to maintain its current state. In other words, lighting control unit 17 does not generate or output a control signal (step S1405) and ends the process.

[0171] As a result, lighting fixture 3 maintains its current state without performing any operation such as changing the illuminance.

[0172] On the other hand, if the illumination control unit 17 determines in step S1404 that the reflection information does not indicate no reflection (step S1404: N), it ends the process.

[0173] As described above, according to the lighting control device 1-2 of Example 2, the reflection determination unit 16 uses the distance d between the camera 2 and the lighting fixture 3 and the field of view information (horizontal field of view a1, vertical field of view a2) included in the camera parameters to calculate the horizontal range ±b1 and vertical range ±b2 that the camera 2 can capture, using equation (5) etc., and calculates the captureable area of ​​the camera 2.

[0174] If the reflection determination unit 16 determines that the lighting fixture 3 is present in the photographable area based on the position information of the lighting fixture 3 and the photographable area, it generates reflection information with reflection, and if it determines that the lighting fixture 3 is not present in the photographable area, it generates reflection information without reflection.

[0175] When the reflection information indicates that reflection is present, the illumination control unit 17 outputs a control signal for adjusting the illuminance, and when the reflection information indicates that reflection is not present, the illumination control unit 17 does not output a control signal.

[0176] As a result, when the reflection information indicates that there is a reflection, the lighting fixture 3 that receives the control signal operates to reduce the illuminance so that it is not reflected in the camera 2. On the other hand, when the reflection information indicates that there is no reflection, the lighting fixture 3 does not receive a control signal and therefore does not operate.

[0177] This allows for easy control of lighting fixtures 3 in accordance with camerawork that follows subject 100 in the imaging space. Furthermore, lighting control suitable for imaging can be achieved with a small number of personnel and lighting fixtures 3.

[0178] Example 3 Next, a description will be given of Example 3. While Examples 1 and 2 are examples in which one camera 2 and one lighting fixture 3 are provided, Example 3 is an example in which two cameras 2 and one lighting fixture 3 are provided.

[0179] Figure 16(1) is a layout diagram when two cameras 2-1, 2-2 and lighting equipment 3 are located in studio S2, and in the layout diagram shown in Figure 7(1), it shows that a second camera 2-2 is also located in studio S2 in the same way as the first camera 2-1.

[0180] In the case of the arrangement shown in Figure 16(1), the lighting control device of Example 3 has a configuration similar to that of the lighting control device 1-1 shown in Figure 2, and the camera information storage unit 10 stores the position information, etc. of the two cameras 2-1 and 2-2.

[0181] The lighting control device of the third embodiment generates reflection information without reflection (same area) for each of the cameras 2-1 and 2-2.

[0182] However, when there is no reflection (same area), a conflict occurs between the control signal for causing lighting device 3 to follow the attitude of camera 2-1 and the control signal for causing lighting device 3 to follow the attitude of camera 2-2. In other words, the lighting control device of Example 3 cannot perform control to cause lighting device 3 to follow the attitude of camera 2-1 and control to cause lighting device 3 to follow the attitude of camera 2-2 at the same time.

[0183] Therefore, the lighting control device of Example 3 presets one of cameras 2-1 and 2-2 as a reference camera and performs control based on the position information, etc. of the reference camera. Specifically, reflection determination unit 12 performs the reflection determination of Example 1 described above based on the position information, etc. of the reference camera and image I input from the reference camera, and lighting control unit 13 outputs a control signal to lighting fixture 3 to make it follow the reference camera, for example.

[0184] For example, assume that camera 2-1 is set as the reference camera, reflection determination unit 12 generates reflection information indicating no reflection (same area), camera 2-1 pans by 10°, and camera 2-2 pans by 5°. In this case, lighting control unit 13 generates a control signal to change the pan angle of lighting device 3 by 10° and outputs it to lighting device 3.

[0185] In the layout diagram shown in FIG. 16(1), even if there are three or more cameras 2 in studio S2, the same processing as described above is carried out.

[0186] Fig. 16(2) is a layout diagram in which the first camera 2-1 and lighting fixture 3 are located in studio S2, and the second camera 2-2 is located in studio S1. This shows that the second camera 2-2 is located in a different studio S1 from the first camera 2-1 in the layout diagram shown in Fig. 7(1).

[0187] In the case of the arrangement shown in Figure 16(2), the lighting control device of Example 3 has a configuration similar to that of the lighting control device 1-1 shown in Figure 2, and the camera information storage unit 10 stores the position information of the two cameras 2-1 and 2-2, etc.

[0188] The lighting control device of Example 3 generates reflection information for camera 2-1 with no reflection (same area), and generates reflection information for camera 2-2 with no reflection (different area) or with reflection (exact match).

[0189] However, a conflict occurs between the control signal generated for camera 2-1 and the control signal generated for camera 2-2 for lighting fixture 3. In other words, a conflict occurs between the control signal for tracking the posture of camera 2-1 and the control signal for preventing lighting fixture 3 from being captured by camera 2-2, and lighting fixture 3 cannot achieve both controls simultaneously.

[0190] Therefore, the lighting control device of Example 3 prioritizes control to prevent lighting fixture 3 from appearing in camera 2-2 over control to cause lighting fixture 3 to follow the orientation of camera 2-1. This is because if control to cause lighting fixture 3 to follow the orientation of camera 2-1 were prioritized over control to prevent lighting fixture 3 from appearing in camera 2-2, the likelihood of lighting fixture 3 appearing in camera 2-2 would increase.

[0191] That is, the lighting control device of Example 3 sets camera 2-2, which is located in studio S1 and not in the same area as lighting fixture 3, as the reference camera, and performs control based on the position information, etc. of camera 2-2. Specifically, reflection determination unit 12 performs the reflection determination of Example 1 based on the position information, etc. of camera 2-2 and image I input from camera 2-2, and lighting control unit 13 outputs a control signal to lighting fixture 3 according to the reflection determination.

[0192] When the reflection of camera 2-2 disappears, the lighting control device of the third embodiment performs control based on the position information of camera 2-1 located in studio S2, which is in the same area as lighting fixture 3.

[0193] 16(2), if there are two or more cameras 2-2 in studio S1, one of the two or more cameras 2-2 is preset as the reference camera. Also, if there are two or more cameras 2-1 in studio S2, control is performed based on the position information of one of the two or more cameras 2-1 that has been preset as a process after the reflection of camera 2-2 disappears.

[0194] In addition, in the layout diagrams shown in FIGS. 16(1) and 16(2), the processing of the third embodiment is applicable not only to the first embodiment but also to the second embodiment.

[0195] As described above, according to the lighting control device of Example 3, when cameras 2-1, 2-2 and lighting fixture 3 are present in studio S2 in the same area, one of cameras 2-1, 2-2 is set in advance as the reference camera, and reflection determination unit 12 performs reflection determination based on the position information of the reference camera and image I input from the reference camera, and lighting control unit 13 outputs a control signal to lighting fixture 3 according to the result of the reflection determination.

[0196] Furthermore, if camera 2-1 and lighting fixture 3 are located in studio S2 in the same area, and camera 2-2 is located in studio S1 in a different area, the lighting control device sets camera 2-2, which is located in studio S1 but is not in the same area as lighting fixture 3, as the reference camera, and reflection determination unit 12 performs reflection determination based on the position information of camera 2-2 and image I input from camera 2-2, and lighting control unit 13 outputs a control signal to lighting fixture 3 according to the result of the reflection determination.

[0197] After that, when the lighting fixture 3 is no longer reflected in camera 2-2, the lighting control device sets camera 2-1, which is located in studio S2, the same area as lighting fixture 3, as the reference camera, and the reflection determination unit 12 performs a reflection determination based on the position information of camera 2-1 and image I input from camera 2-1, and the lighting control unit 13 outputs a control signal to lighting fixture 3 according to the result of the reflection determination.

[0198] This allows the lighting fixtures 3 to be controlled in accordance with the camerawork of the reference camera that follows the subject 100 in the imaging space. Even when multiple cameras 2 are present, lighting control suitable for imaging can be achieved with a small number of personnel and lighting fixtures 3.

[0199] Although the present invention has been described above with reference to Examples 1 to 3, the present invention is not limited to the above Examples 1 to 3 and can be modified in various ways without departing from the technical concept thereof.

[0200] A normal computer can be used as the hardware configuration of the lighting control device 1-1 of Example 1, the lighting control device 1-2 of Example 2, and the lighting control device of Example 3. The lighting control device 1-1, etc., is configured by a computer equipped with a CPU, a volatile storage medium such as RAM, a non-volatile storage medium such as ROM, an interface, etc.

[0201] The functions of the camera information storage unit 10, lighting fixture information storage unit 11, reflection determination unit 12 and lighting control unit 13 provided in the lighting control device 1-1 of Example 1 are each realized by having a CPU execute a program that describes these functions.

[0202] Furthermore, the functions of the camera information storage unit 14, lighting device information storage unit 15, reflection determination unit 16, and lighting control unit 17 provided in the lighting control device 1-2 of Example 2 are also realized by having the CPU execute a program describing these functions. Similarly, in the case of the lighting control device of Example 3, each of the components is realized by having the CPU execute a program describing the function of each component.

[0203] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be stored in a storage medium such as a magnetic disk (e.g., a floppy disk, a hard disk), an optical disk (e.g., a CD-ROM, a DVD), or a semiconductor memory and distributed, or can be transmitted and received via a network. [Explanation of symbols]

[0204] 1-1,1-2 Lighting control device 2,2-1,2-2 Camera 3. Lighting fixtures 10,14 Camera information storage unit 11,15 Lighting equipment information storage section 12,16 Reflection detection unit 13,17 Lighting control unit 20 Position determination section 21 Overexposure detection section 22 Shape matching determination section 23 Shooting area calculation unit 24 Judgment section 100 subjects 101,102 Lighting System I Image WC Number of whiteout pixels S1, S2 Studio SE Safety Area BE Bandaeria

Claims

1. 1. A lighting control device that controls lighting fixtures that illuminate a subject in accordance with camerawork of a camera that photographs the subject, a camera information storage unit that stores position information, attitude information, and camera parameters of the camera; a lighting fixture information storage unit that stores position information, posture information, shape and size of the lighting fixture; a reflection determination unit that determines a state of reflection of the lighting fixture relative to the camera based on the position information, the attitude information, and the camera parameters of the camera stored in the camera information storage unit, the position information, the attitude information, the shape, and the size of the lighting fixture stored in the lighting fixture information storage unit, and an image captured by the camera; a lighting control unit that generates a control signal for controlling the lighting device in accordance with the state of the reflection determined by the reflection determination unit, The reflection determination unit Assuming that a location where the subject is photographed is divided into two areas, determining whether the camera and the lighting fixture are located in the same area or different areas based on the position information of the camera and the position information of the lighting fixture; If it is determined that the camera and the lighting fixture are present in the same area, reflection information indicating that there is no reflection and that the camera and the lighting fixture are present in the same area is generated, From the image, the number of whiteout pixels caused by the illumination of the lighting device is calculated as the whiteout pixel count WC; If it is determined that the camera and the lighting equipment are present in different areas and the number of whiteout pixels WC is smaller than a preset threshold, reflection information indicating that there is no reflection and that the camera and the lighting equipment are present in different areas is generated. If it is determined that the camera and the lighting device are present in different areas and it is determined that the number of whiteout pixels WC is not smaller than the threshold value, reflection information indicating that reflection is present is generated; The lighting control unit When the reflection determination unit generates reflection information indicating no reflection (same area), the control signal for panning or tilting the lighting device to follow the attitude of the camera is generated, thereby causing the lighting device to follow the attitude of the camera; When the reflection information indicating no reflection (different area) is generated, the control signal is not generated and the state of the lighting device is maintained; A lighting control device characterized in that, when the reflection information indicating the presence of reflection is generated, the control signal for reducing the illuminance of the lighting device, or the control signal for panning, tilting, or rolling is generated, thereby preventing the lighting device from being reflected in the camera.

2. 2. The lighting control device according to claim 1, The reflection determination unit If it is determined that the camera and the lighting equipment are present in different areas and it is determined that the number of whiteout pixels WC is not smaller than the threshold value, generating reflection information with reflection (perfect match) indicating that the lighting fixture included in the image and the lighting fixture estimated to be captured in the image are a perfect match, reflection information with reflection (partial match / edge of view) indicating that the lighting fixture included in the image and the lighting fixture estimated to be captured in the image are a partial match and the lighting fixture is present at the edge of the angle of view of the image, or reflection information with reflection (partial match / overlap) indicating that the lighting fixture included in the image and the lighting fixture estimated to be captured in the image are a partial match and the lighting fixture overlaps the subject included in the image, based on the position information, the attitude information, and the camera parameters of the camera, the position information, the attitude information, the shape, and the size of the lighting fixture, and the image; The lighting control unit When the reflection determination unit generates the reflection information indicating that reflection is present (complete match), the control signal is generated to reduce the illuminance of the lighting device, thereby preventing the lighting device from being reflected by the camera, When the reflection information indicating that reflection is present (partial match / edge of angle of view) is generated, the control signal for panning, tilting, or rolling is generated to prevent the lighting fixture from being reflected in the camera, A lighting control device characterized in that, when the reflection information indicating the presence of reflection (partial match / overlap) is generated, the control signal is generated to reduce the illuminance of the lighting device, thereby preventing the lighting device from being reflected in the camera.

3. 3. The lighting control device according to claim 2, The reflection determination unit If it is determined that the camera and the lighting equipment are present in different areas and it is determined that the number of whiteout pixels WC is not smaller than the threshold value, calculating the size and area of ​​the lighting fixture from the image as an image size and an image area, and estimating the overall size and area of ​​the lighting fixture shown in the image as an estimated size and an estimated area from the position information, the attitude information, and the camera parameters of the camera, and the position information, the attitude information, the shape, and the size of the lighting fixture; If it is determined that the image size and the estimated size are the same and that the image area and the estimated area are the same, the reflection information indicating that reflection is present (complete match) is generated; If it is determined that the image size and the estimated size are different and that the image area and the estimated area are different, the reflection information indicating that reflection is present (partial match / edge of angle of view) is generated; A lighting control device characterized in that, when it is determined that the image size and the estimated size are the same and the image area and the estimated area are different, reflection information indicating that reflection exists (partial match / overlap) is generated.

4. 1. A lighting control device that controls lighting fixtures that illuminate a subject in accordance with camerawork of a plurality of cameras that photograph the subject, a camera information storage unit that stores position information, orientation information, and camera parameters for each of the plurality of cameras; a lighting fixture information storage unit that stores position information, posture information, shape and size of the lighting fixture; a reflection determination unit that determines a state in which the lighting fixture is reflected in the reference camera based on the position information, the attitude information, and the camera parameters for each of the plurality of cameras stored in the camera information storage unit, the position information, the attitude information, the shape, and the size of the lighting fixture stored in the lighting fixture information storage unit, and an image captured by the reference camera, using a predetermined one of the plurality of cameras as a reference camera; and a lighting control unit that generates a control signal for controlling the lighting device in accordance with the state of the reflection determined by the reflection determination unit, The reflection determination unit If the location where the subject is photographed is divided into two areas, and it is determined that the plurality of cameras and the lighting fixtures are present in the same area based on the position information for each of the plurality of cameras and the position information for the lighting fixtures, one of the plurality of cameras is set as a first reference camera, and reflection information indicating that there is no reflection and that the first reference camera and the lighting fixtures are present in the same area (same area) is generated; when it is determined based on the position information for each of the plurality of cameras and the position information for the lighting fixtures that at least one of the plurality of cameras is located in the area different from the lighting fixtures and that another camera and the lighting fixtures are located in the same area, setting one of the plurality of cameras that is located in the area different from the lighting fixtures as a second reference camera; From the image captured by the second reference camera, the number of whiteout pixels caused by the illumination of the lighting fixture is calculated as a whiteout pixel count WC, and if it is determined that the whiteout pixel count WC is smaller than a preset threshold, reflection information indicating that there is no reflection (different region) indicating that the second reference camera and the lighting fixture are present in different regions is generated; If it is determined that the number of whiteout pixels WC is not smaller than the threshold value, then generating reflection information indicating that reflection is present, The lighting control unit When the reflection determination unit generates reflection information indicating no reflection (same area), the control signal for panning or tilting the lighting device to follow the attitude of the first reference camera is generated, thereby causing the lighting device to follow the attitude of the first reference camera; When the reflection information indicating no reflection (different area) is generated, the control signal is not generated and the state of the lighting device is maintained; A lighting control device characterized in that, when the reflection information indicating the presence of reflection is generated, the control signal for reducing the illuminance of the lighting device, or the control signal for panning, tilting, or rolling is generated, so that the lighting device is not reflected in the second reference camera.

5. a computer constituting an illumination control device that controls lighting fixtures that illuminate an object in accordance with the camerawork of a camera that photographs the object; a camera information storage unit that stores position information, attitude information, and camera parameters of the camera; a lighting fixture information storage unit that stores position information, posture information, shape and size of the lighting fixture; a reflection determination unit that determines a state in which the lighting fixture is reflected by the camera based on the position information, the attitude information, and the camera parameters of the camera stored in the camera information storage unit, the position information, the attitude information, the shape, and the size of the lighting fixture stored in the lighting fixture information storage unit, and an image captured by the camera; and functioning as a lighting control unit that generates a control signal for controlling the lighting device in accordance with the state of reflection determined by the reflection determination unit; The reflection determination unit Assuming that a location where the subject is photographed is divided into two areas, determining whether the camera and the lighting fixture are located in the same area or different areas based on the position information of the camera and the position information of the lighting fixture; If it is determined that the camera and the lighting fixture are present in the same area, reflection information indicating that there is no reflection and that the camera and the lighting fixture are present in the same area is generated, From the image, the number of whiteout pixels caused by the illumination of the lighting device is calculated as the whiteout pixel count WC; If it is determined that the camera and the lighting equipment are present in different areas and the number of whiteout pixels WC is smaller than a preset threshold, reflection information indicating that there is no reflection and that the camera and the lighting equipment are present in different areas is generated. If it is determined that the camera and the lighting equipment are present in different areas and it is determined that the number of whiteout pixels WC is not smaller than the threshold value, reflection information indicating that reflection is present is generated; The lighting control unit When the reflection determination unit generates reflection information indicating no reflection (same area), the control signal for panning or tilting the lighting device to follow the attitude of the camera is generated, thereby causing the lighting device to follow the attitude of the camera; When the reflection information indicating no reflection (different area) is generated, the control signal is not generated and the state of the lighting device is maintained; When the reflection information indicating the presence of a reflection is generated, the program generates the control signal to reduce the illuminance of the lighting device, or the control signal for panning, tilting, or rolling, thereby preventing the lighting device from being reflected by the camera.

Images