Control device and control method for multi-ranging camera

The control device adjusts zoom focal lengths based on camera distances to ensure consistent point cloud densities across multiple cameras, addressing visual discomfort in volumetric image generation.

JP2024157339APending Publication Date: 2024-11-07JVC KENWOOD CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023071641
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-25
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

When multiple distance-measuring cameras with different distances and zoom settings capture images, the varying angles of view and point cloud densities result in visual discomfort in volumetric image generation.

Method used

A control device adjusts the zoom focal lengths of non-reference cameras based on the distance to a reference camera, ensuring consistent point cloud densities across multiple cameras by calculating and controlling the focal lengths using the distances between cameras and the subject.

Benefits of technology

This approach ensures that images captured by multiple cameras have consistent point cloud densities, reducing visual discomfort and enabling high-quality volumetric image generation without the need for dedicated studios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157339000001_ABST
    Figure 2024157339000001_ABST
Patent Text Reader

Abstract

To provide a control device for a multi-ranging camera that can match amounts of point groups in photographed images by respective ranging cameras even if a plurality of ranging cameras is arranged in a state in which distances to a subject are different from each other.SOLUTION: An arithmetic section 31 calculates a distance from each ranging camera of a plurality of ranging cameras 1A to 1D arranged to surround a subject 2 and including a zoom lens to a center 2c of the subject 2. One of the plurality of ranging cameras 1A to 1D is set as a reference ranging camera. A zoom controller 33 controls a focal length of zoom when the ranging camera other than the reference ranging camera photographs the subject 2 to control the zoom of the ranging cameras other than the reference ranging camera to a focal length by multiplying the focal length of the zoom when the reference ranging camera photographs the subject 2 by a value obtained by dividing the distance from the reference ranging camera to the center 2c of the subject 2 by a distance from the ranging camera other than the reference ranging camera to the center 2c of the subject 2.SELECTED DRAWING: Figure 9
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a control device and a control method for a multi-distance measuring camera. [Background technology]

[0002] A volumetric image generating device uses multiple distance measuring cameras to capture an object from a 360-degree angle and generate a volumetric image, which is a three-dimensional model of the object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 105697 Summary of the Invention [Problem to be solved by the invention]

[0004] If the distance between the subject and each ranging camera is different, the angle of view when each ranging camera captures the subject will not match, and the amount of point cloud in the image captured by each ranging camera will be different. Similarly, if the ranging camera is equipped with a zoom lens and each ranging camera zooms with a different zoom amount, the amount of point cloud in the image captured by each ranging camera will be different. If a volumetric image is generated by synthesizing images captured by multiple ranging cameras with different amounts of point clouds, a visually strange feeling may be generated.

[0005] If a subject is photographed in a dedicated studio called a volumetric studio, where the distance between the subject and each ranging camera is strictly controlled, the amount of points in the images captured by each ranging camera will match. However, renting a dedicated studio is expensive, and it is not easy to photograph the subject.

[0006] The present invention aims to provide a control device and control method for a multi-ranging camera that can match the amount of point clouds in images captured by each ranging camera, even if the multiple ranging cameras are positioned at different distances from the subject. [Means for solving the problem]

[0007] The present invention provides a control device for a multi-ranging camera, the control device including: a calculation unit which calculates the distance from each of a plurality of ranging cameras, each including a zoom lens, arranged to surround a subject, to the center of the subject; and a zoom control unit which sets one of the plurality of ranging cameras as a reference ranging camera and controls the zoom of the ranging cameras other than the reference ranging camera so that the focal length of the zoom when the ranging camera other than the reference ranging camera photographs the subject is the focal length obtained by multiplying the focal length of the zoom when the reference ranging camera photographs the subject by the distance from the reference ranging camera to the center of the subject divided by the distance from the ranging camera other than the reference ranging camera to the center of the subject.

[0008] The present invention provides a method for controlling a multi-ranging camera, in which a computer device controlling a plurality of ranging cameras arranged to surround a subject calculates the distance from each of the plurality of ranging cameras to the center of the subject, sets one of the plurality of ranging cameras as a reference ranging camera, and controls the zoom of ranging cameras other than the reference ranging camera so that the focal length of the zoom when the ranging camera other than the reference ranging camera photographs the subject is a focal length obtained by multiplying the focal length of the zoom when the reference ranging camera photographs the subject by a value obtained by dividing the distance from the reference ranging camera to the center of the subject by the distance from the ranging camera other than the reference ranging camera to the center of the subject. Effect of the Invention

[0009] According to the multi-ranging camera control device and control method of the present invention, even if multiple ranging cameras are positioned at different distances from the subject, the amount of point clouds in the images captured by each ranging camera can be made to be consistent. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a plurality of distance measuring cameras arranged to surround a subject. [Diagram 2] FIG. 2 is a conceptual diagram showing point cloud data generated by the multiple distance measuring cameras shown in FIG. 1 photographing a subject with the same zoom amount. [Diagram 3] FIG. 3 is a diagram showing a state in which the control device for the multi-distance measuring camera according to the first embodiment measures the distances between the facing distance measuring cameras among a plurality of distance measuring cameras. [Figure 4A] FIG. 4A is a flowchart showing a partial process in which the control device of the multiple distance measuring cameras according to the first embodiment measures the distance between the facing distance measuring cameras. [Figure 4B] FIG. 4B is a flowchart showing a partial process subsequent to FIG. 4A in which the control device of the multiple distance measuring cameras according to the first embodiment measures the distance between the facing distance measuring cameras. [Figure 5A] FIG. 5A is a plan view of the lens barrel of the distance measuring camera as viewed from the front. [Figure 5B] FIG. 5B is a plan view showing a state in which a lens cap is attached to the lens of the distance measuring camera. [Figure 5C] FIG. 5C is a plan view showing a state in which a lens cap with a white center and a black periphery is attached to the lens of the distance measuring camera. [Figure 6] FIG. 6 is a partial perspective view showing a state in which a lens cap is attached to the lens barrel of the distance measuring camera. [Figure 7] FIG. 7 is a conceptual diagram showing a near-infrared image generated by one of a plurality of distance measuring cameras photographing an opposing distance measuring camera. [Figure 8]FIG. 8 is a diagram showing the luminance level of near-infrared light in the horizontal direction at the vertical position passing through the lens cap in the near-infrared image shown in FIG. [Figure 9] FIG. 9 is a diagram showing a state in which a plurality of distance measuring cameras are photographing a subject. [Figure 10] FIG. 10 is a diagram showing in detail how the distance to the subject is measured by the distance measuring camera. [Figure 11A] FIG. 11A is a flowchart showing a partial process in which a control device of a multi-distance measuring camera according to the second embodiment measures the distances to two adjacent distance measuring cameras. [Figure 11B] FIG. 11B is a flowchart showing a partial process subsequent to FIG. 11A, in which the control device of the multi-distance measuring camera according to the second embodiment measures the distances to two adjacent distance measuring cameras. [Figure 11C] FIG. 11C is a flowchart showing a partial process subsequent to FIG. 11B in which the control device of the multi-distance measuring camera according to the second embodiment measures the distances to two adjacent distance measuring cameras. [Figure 11D] FIG. 11D is a flowchart showing a partial process subsequent to FIG. 11C in which the control device of the multi-distance measuring camera according to the second embodiment measures the distances to two adjacent distance measuring cameras. [Figure 11E] FIG. 11E is a flowchart showing a partial process subsequent to FIG. 11D in which the control device of the multi-distance measuring camera according to the second embodiment measures the distances to two adjacent distance measuring cameras. [Figure 12] FIG. 12 is a conceptual diagram showing a near-infrared image generated by one of a plurality of distance measurement cameras capturing images of two adjacent distance measurement cameras. [Figure 13A] FIG. 13A is a diagram showing point cloud coordinates at the center of the lens cap extracted by the control device of the multiple distance measuring camera according to the second embodiment in a state in which the distance measuring camera 1A is photographing the distance measuring cameras 1B and 1D. [Figure 13B]FIG. 13B is a diagram showing point cloud coordinates at the center of the lens cap extracted by the control device of the multiple distance measuring camera according to the second embodiment in a state in which the distance measuring camera 1B is photographing the distance measuring cameras 1A and 1C. [Figure 13C] FIG. 13C is a diagram showing point cloud coordinates at the center of the lens cap extracted by the control device of the multiple distance measuring camera according to the second embodiment in a state in which the distance measuring camera 1C is photographing the distance measuring cameras 1B and 1D. [Figure 13D] FIG. 13D is a diagram showing point cloud coordinates at the center of the lens cap extracted by the control device of the multiple distance measuring camera according to the second embodiment in a state in which the distance measuring camera 1D is photographing the distance measuring cameras 1A and 1C. [Figure 14] FIG. 14 is a diagram showing the coordinate system of the distance measuring camera 1B as seen from the distance measuring camera 1A. [Figure 15] FIG. 15 is a diagram showing the relationship between the axial directions of the distance measuring cameras 1A and 1B, modeled on FIG. [Figure 16] FIG. 16 is a diagram showing the field of view when the control device of the multi-distance measuring camera according to the second embodiment determines the center of the subject. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a control device and a control method for a multi-distance measuring camera according to each embodiment will be described with reference to the accompanying drawings.

[0012] First Embodiment The control device and control method for a multi-distance measuring camera according to the first embodiment are suitable for use in cases where a subject photographed by a plurality of distance measuring cameras can be temporarily moved when adjusting each of the distance measuring cameras.

[0013] In FIG. 1, distance measuring cameras 1A to 1D are arranged to surround a subject 2. The distance measuring cameras 1A to 1D have the same performance. The distance measuring cameras 1A to 1D have a zoom function by having a zoom lens 1ZL in a lens barrel 11. Although the subject 2 is a sphere in FIG. 1, the subject 2 does not have to be a sphere. Any distance measuring camera that does not specify which of the distance measuring cameras 1A to 1D is referred to as a distance measuring camera 1. The distance measuring camera 1 calculates the distance to the subject by irradiating light of, for example, a near-infrared wavelength from an infrared light source such as a vertical-resonator surface-emitting laser 13 (see FIG. 5A) and receiving the light reflected from the subject with a sensor.

[0014] The subject 2 may be surrounded by any number of distance measuring cameras 1. The distance measuring cameras 1 may be cameras called TOF (Time Of Flight) cameras or 3D scanners.

[0015] Distance measuring cameras 1A and 1C are arranged so that their lens barrels 11 face each other with the subject 2 being placed between distance measuring cameras 1A and 1C. Distance measuring cameras 1B and 1D are arranged so that their lens barrels 11 face each other with the subject 2 being placed between distance measuring cameras 1B and 1D. The distances from sensors (not shown) to the center 2c of the subject 2 in distance measuring cameras 1A to 1D are respectively Da, Db, Dc, and Dd. It is assumed that Da to Dd have the relationship Da>Dc>Db>Dd.

[0016] Assume that the zoom amounts of the distance measuring cameras 1A to 1D are the same. When the distance data generated by the distance measuring cameras 1A to 1D is converted into point cloud data, the point cloud data PCa to PCd corresponding to the distance measuring cameras 1A to 1D are conceptually as shown in Fig. 2. The size of the subject 2 in the point cloud data PCa to PCd is largest in the point cloud data PCd, followed by PCb, PCc, and PCa in that order. Therefore, the amount of points contained in the subject 2 differs among the point cloud data PCa to PCd.

[0017] When the point cloud data PCa to PCd are synthesized to generate a volumetric image that is a three-dimensional model of the subject 2, a visually unnatural appearance may occur. Therefore, the zoom amounts of the distance measuring cameras 1A to 1D need to be individually adjusted so that the size of the subject 2 is the same for all of them.

[0018] In Fig. 3, a zoom control device 30, which is a control device for a multi-distance measuring camera according to the first embodiment, includes a calculation unit 31, a storage unit 32, and a zoom control unit 33. The zoom control device 30 can be configured by a computer device such as a personal computer. As shown in Fig. 3, the subject 2 shown in Fig. 1 is temporarily moved to an arbitrary position outside the area between the distance measuring cameras 1A and 1C and between the distance measuring cameras 1B and 1D.

[0019] The zoom control device 30 measures the distance between two mutually facing distance measurement cameras 1 of the distance measurement cameras 1A to 1D in accordance with the flow charts shown in FIGS. 4A and 4B.

[0020] In FIG. 4A, the zoom control device 30 sets the distance measurement camera 1A to a mode in which the distance measurement camera 1A measures the distance to the distance measurement camera 1C in step S1. In step S2, the user attaches a white lens cap 12 to the lens barrel 11 of the distance measurement camera 1C. FIG. 5A is a plan view of the lens barrel 11 of the distance measurement camera 1 as seen from the front. At the tip of the lens barrel 11, for example, four vertical-cavity surface-emitting lasers (VCSELs) 13 are provided around the lens 11L. The VCSELs 13 emit near-infrared light. As shown in FIG. 5B, the user attaches the lens cap 12 to cover the lens 11L of the distance measurement camera 1C.

[0021] As shown in Fig. 5C, the user may attach lens cap 12B having a white center portion 12B1 including the optical axis and a black peripheral portion 12B2 to lens barrel 11 of distance measuring camera 1. As shown in Fig. 6, the user may attach white lens cap 12C that covers the entire tip of lens barrel 11.

[0022] The housing of the distance measuring camera 1 including the lens barrel 11 is painted with a dark color such as black or gray, or is made of a material that is dark in color such as black or gray. Since the reflectance of near-infrared light is low on a dark colored subject, the accuracy of distance measurement when the distance measuring camera 1 measures the distance to the opposing distance measuring camera 1 deteriorates and accurate distance measurement is not possible. By attaching the lens cap 12, 12B, or 12C to the lens barrel 11, the distance to the opposing distance measuring camera 1 can be measured accurately.

[0023] The lens cap 12, the center portion 12B1 of the lens cap 12B, and the lens cap 12C may be painted with white paint or may be made of a white material. For example, a composite oxide pigment, which has a high reflectance of near-infrared light, may be used as the white paint for the purpose of blocking heat outdoors. In the following description, it is assumed that the lens cap 12 is attached to the lens barrel 11.

[0024] In step S3, the zoom control device 30 controls the distance measuring camera 1A as follows to obtain a distance measurement value Daccap, which is the distance to the distance measuring camera 1C calculated by the distance measuring camera 1A. The distance measuring camera 1 employs an iToF (Indirect ToF) method and generates a near-infrared image at the same time as a distance measuring image showing distance data as pixels. FIG. 7 conceptually shows a near-infrared image NIa generated by the distance measuring camera 1A photographing the distance measuring camera 1C. FIG. 8 shows the luminance level of near-infrared light in the horizontal direction at a vertical position passing through the lens cap 12 in the near-infrared image NIa.

[0025] The sensor of the distance measuring camera 1A receives near-infrared light emitted from the VCSEL 13 and reflected back by the lens cap 12 attached to the lens barrel 11 of the distance measuring camera 1C. At this time, the zoom control device 30 stops the emission of near-infrared light from the VCSEL 13 in the distance measuring cameras 1B to 1D.

[0026] The zoom control device 30 controls the distance measurement camera 1A to generate a distance measurement value Daccap by averaging only distance measurement values ​​in an area of ​​the near-infrared image NIa that corresponds to the lens cap 12 and exceeds a threshold value Th set by the zoom control device 30. If the zoom control device 30 controls the distance measurement camera 1A in this way, the distance measurement camera 1A can accurately measure the distance to the tip of the lens barrel 11 of the distance measurement camera 1C. The zoom control device 30 acquires the distance measurement value Daccap generated in this way by the distance measurement camera 1A.

[0027] The zoom control device 30 may color the area in the near-infrared image NIa that exceeds the threshold value Th in yellow, for example, and display the near-infrared image NIa on a display unit (not shown). By checking the area colored in yellow in the near-infrared image NIa displayed on the display unit, the user can determine whether the distance measuring camera 1A generates the distance measurement value Daccap based on the distance measurement value of the area corresponding to the lens cap 12.

[0028] In step S4, the calculation unit 31 of the zoom control device 30 adds the distance Ds from the lens cap of the distance measurement camera 1C to the sensor, as shown in Fig. 3, to the distance measurement value Daccap to calculate the distance Dac from the distance measurement camera 1A to the distance measurement camera 1C. The distance Ds from the lens cap of the distance measurement camera 1C to the sensor is known, and the zoom control device 30 stores the distance Ds in the memory unit 32 in advance.

[0029] In step S5, the zoom control device 30 sets the distance measurement camera 1B to a mode in which the distance measurement camera 1B measures the distance to the distance measurement camera 1D. In step S6, the user attaches the white lens cap 12 to the lens barrel 11 of the distance measurement camera 1D. In step S7, the zoom control device 30 controls the distance measurement camera 1B in the same manner as controlling the distance measurement camera 1A, and obtains a distance measurement value Dbdcap, which is the distance to the distance measurement camera 1D calculated by the distance measurement camera 1B. At this time, the zoom control device 30 stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1A, 1C, and 1D.

[0030] In step S8, the calculation unit 31 of the zoom control device 30 adds the distance Ds from the lens cap 12 of the distance measurement camera 1D to the sensor to the distance measurement value Dbdcap to calculate the distance Dbd from the distance measurement camera 1B to the distance measurement camera 1D.

[0031] 4B, in step S9, the zoom control device 30 sets the distance measurement camera 1C to a mode in which the distance measurement camera 1C measures the distance to the distance measurement camera 1A. In step S10, the user attaches a white lens cap 12 to the lens barrel 11 of the distance measurement camera 1A. In step S11, the zoom control device 30 controls the distance measurement camera 1C in the same manner as it controls the distance measurement camera 1A, and obtains a distance measurement value Dcacap, which is the distance to the distance measurement camera 1A calculated by the distance measurement camera 1C. At this time, the zoom control device 30 stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1A, 1B, and 1D.

[0032] In step S12, the calculation unit 31 of the zoom control device 30 adds the distance Ds from the lens cap 12 of the distance measurement camera 1A to the sensor to the distance measurement value Dcacap to calculate the distance Dca from the distance measurement camera 1C to the distance measurement camera 1A.

[0033] In step S13, the zoom control device 30 sets the distance measurement camera 1D to a mode in which the distance measurement camera 1D measures the distance to the distance measurement camera 1B. In step S14, the user attaches a white lens cap 12 to the lens barrel 11 of the distance measurement camera 1B. In step S15, the zoom control device 30 controls the distance measurement camera 1D in the same manner as controlling the distance measurement camera 1A, and obtains a distance measurement value Ddbcap, which is the distance to the distance measurement camera 1B calculated by the distance measurement camera 1D. At this time, the zoom control device 30 stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1A, 1B, and 1C.

[0034] In step S16, the calculation unit 31 of the zoom control device 30 adds the distance Ds from the lens cap 12 of the distance measurement camera 1B to the sensor to the distance measurement value Ddbcap to calculate the distance Ddb from the distance measurement camera 1D to the distance measurement camera 1B.

[0035] In step S17, the zoom control device 30 sets the zoom control device 30 to a distance preservation mode. In step S18, the calculation unit 31 of the zoom control device 30 calculates an average value Dacave of the distances Dac and Dca, and calculates an average value Dbdave of the distances Dbd and Ddb. In step S19, the calculation unit 31 stores the average values ​​Dacave and Dbdave in the storage unit 32 and ends the process.

[0036] Theoretically, the distances Dac and Dca are the same value, and the distances Dbd and Ddb are the same value. In reality, the distance measured by the distance measuring camera 1 includes an error. By calculating the average value Dacave of the distances Dac and Dca and the average value Dbdave of the distances Dbd and Ddb, it is possible to obtain the accurate distance between the distance measuring cameras 1A and 1C and the accurate distance between the distance measuring cameras 1B and 1D.

[0037] In this way, it is preferable to calculate the average value Dacave of the distances Dac and Dca, and calculate the average value Dbdave of the distances Dbd and Ddb. The process of measuring the distance between the distance measurement cameras 1A and 1C may be simplified by calculating only the distance Dac or only the distance Dca. The process of measuring the distance between the distance measurement cameras 1B and 1D may be simplified by calculating only the distance Dbd or only the distance Ddb.

[0038] In this manner, the zoom control device 30 completes the measurement of the distance between the two mutually facing distance measurement cameras 1 of the distance measurement cameras 1A to 1D, and ends the distance measurement mode.

[0039] Next, the zoom control device 30 transitions to a mode for photographing the subject 2. As shown in Fig. 9, the subject 2 shown in Fig. 1 is returned to a position between the distance measurement camera 1A and the distance measurement camera 1C and between the distance measurement camera 1B and the distance measurement camera 1D. In Fig. 9, the zoom control device 30 acquires a distance D2a from the sensor of the distance measurement camera 1A to the surface of the subject 2 from the distance measurement camera 1A, and acquires a distance D2c from the sensor of the distance measurement camera 1C to the surface of the subject 2 from the distance measurement camera 1C. In addition, the zoom control device 30 acquires a distance D2b from the sensor of the distance measurement camera 1B to the surface of the subject 2 from the distance measurement camera 1B, and acquires a distance D2d from the sensor of the distance measurement camera 1D to the surface of the subject 2 from the distance measurement camera 1D.

[0040] The calculation unit 31 of the zoom control device 30 calculates the width Wac of the subject 2 in the direction connecting the ranging cameras 1A and 1C according to equation (1), and calculates the width Wbd of the subject 2 in the direction connecting the ranging cameras 1B and 1D according to equation (2). Wac=Dacave-D2a-D2c …(1) Wbd = Dbdave - D2b - D2d ... (2)

[0041] The calculation unit 31 calculates the distances Da to Dd from the distance measuring cameras 1A to 1D to the center 2c of the subject 2, respectively, in accordance with equations (3) to (6). The center 2c of the subject 2 is located at Wac / 2 and Wbd / 2. Da = D2a + Wac / 2 … (3) Db = D2b + Wbd / 2 … (4) Dc=D2c+Wac / 2 …(5) Dd = D2d + Wbd / 2 … (6)

[0042] The calculation unit 31 calculates the distances Da to Dd for each frame of the distance measurement images generated by the distance measurement cameras 1A to 1D. When the subject 2 is a moving object rather than a stationary object, the calculation unit 31 may calculate the distances Da to Dd when the subject 2 changes or moves.

[0043] Next, the user designates one of the distance measurement cameras 1A to 1D as the reference distance measurement camera using the zoom control device 30. It is assumed that the user has designated the distance measurement camera 1A as the reference distance measurement camera. The user arbitrarily sets the zoom amount of the distance measurement camera 1A using the zoom control unit 33 of the zoom control device 30. The focal length of the zoom of the distance measurement camera 1A is set to FA.

[0044] The zoom control unit 33 controls the zoom focal lengths FB to FD of the distance measurement cameras 1B to 1D based on the equations (7) to (9), respectively. The zoom control unit 33 supplies zoom control values ​​to the distance measurement cameras 1B to 1D so that the zoom of the distance measurement cameras 1B to 1D becomes the focal lengths FB to FD shown in the equations (7) to (9). FB = FA × Da / Db … (7) FC = FA × Da / Dc … (8) FD = FA × Da / Dd … (9)

[0045] In this way, when the user sets the zoom focal length of the distance measurement camera 1A, which is the reference distance measurement camera, to FA, the zoom control unit 33 controls the zoom focal lengths FB to FD of the distance measurement cameras 1B to 1D based on the equations (7) to (9), respectively. Since the zoom amounts of the distance measurement cameras 1A to 1D are controlled according to the distance between the subject 2 and each distance measurement camera 1, the subject 2 is photographed at approximately the same size by the distance measurement cameras 1A to 1D.

[0046] As a result, the angles of view when the distance measuring cameras 1A to 1D capture the subject 2 are almost the same, and therefore the amount of point clouds in the images captured by each distance measuring camera 1 is almost the same. Therefore, when the images captured by the distance measuring cameras 1A to 1D are synthesized to generate a volumetric image, almost no visual discomfort occurs.

[0047] When the user changes the focal length of the zoom of the distance measuring camera 1A, the zoom control unit 33 immediately controls the focal length of the zoom of the distance measuring cameras 1B to 1D so as to correspond to the change in the focal length of the distance measuring camera 1A. The amount of points in the images captured by the distance measuring cameras 1A to 1D is always almost the same, and a volumetric image with almost no visual discomfort can be generated.

[0048] However, since there is a limit to the range in which the zoom lens 1ZL in the range finding camera 1 can move, the zoom control device 30 (zoom control section 33) needs to control the focal length of the zoom in the range finding cameras 1B-1D within the range in which the zoom lens 1ZL can move. The zoom lenses 1ZL in the range finding cameras 1A-1D have the same performance, and the zoom lens 1ZL is variable from the wide end to the telephoto end, from the focal length W (mm) to T (mm).

[0049] When distance measuring camera 1A is designated as the main camera, the maximum distance Dmax is the largest distance among distances Db to Dd that is greater than distance Da. When none of distances Db to Dd are greater than distance Da, distance Da is the maximum distance Dmax. When none of distances Db to Dd are greater than distance Da, distance Da is the maximum distance Dmax. When none of distances Db to Dd are less than distance Da, distance Da is the minimum distance Dmin. When none of distances Db to Dd are less than distance Da, distance Da is the minimum distance Dmin. If the focal length at maximum distance Dmax is Fmax and the focal length at minimum distance Dmin is Fmin, the focal lengths Fmax and Fmin are expressed by equations (10) and (11).

[0050] Fmax = FA × Dmax / Da … (10) Fmin = FA × Dmin / Da … (11)

[0051] When the first condition that the focal length Fmin is smaller than the focal length W and the focal length Fmax is smaller than the focal length T is satisfied, the zoom control device 30 replaces the focal length FA in the expressions (10) and (11) with the focal length Fa shown in the expression (12). Fa = W × W / Fmin … (12)

[0052] When the second condition is satisfied, that is, the focal length Fmin is greater than the focal length W and the focal length Fmax is greater than the focal length T, the zoom control device 30 replaces the focal length FA in the expressions (10) and (11) with the focal length Fa shown in the expression (13). Fa = T × T / Fmax … (13)

[0053] When a third condition is satisfied, that is, the focal length Fmin is smaller than the focal length W, the focal length Fmax is greater than the focal length T, and the focal length FA is smaller than (TW) / 2+W, the zoom control device 30 replaces the focal length FA in equations (10) and (11) with the focal length Fa shown in equation (14). Fa = W × W / Fmin … (14)

[0054] When a fourth condition is satisfied, that is, the focal length Fmin is smaller than the focal length W, the focal length Fmax is greater than the focal length T, and the focal length FA is greater than (TW) / 2+W, the zoom control device 30 replaces the focal length FA in equations (10) and (11) with the focal length Fa shown in equation (15). Fa = T × T / Fmax … (15)

[0055] If none of the first to fourth conditions is satisfied, the zoom control device 30 sets the focal length FA in equations (10) and (11) as it is as the focal length Fa, as shown in equation (16). Fa = FA … (16)

[0056] The zoom control device 30 sets the focal length FA of the zoom of the distance measurement camera 1A to the focal length Fa shown in equations (12) to (16). The zoom control device 30 replaces the focal length FA in equations (7) to (9) with the focal length Fa shown in equations (12) to (16) and then calculates the focal lengths FB to FD of the distance measurement cameras 1B to 1D.

[0057] As described above, the calculation unit 31 of the zoom control device 30 is disposed to surround the subject 2, and calculates the distance from each of the multiple distance measuring cameras 1 including the zoom lens 1ZL to the center 2c of the subject 2.

[0058] The zoom control unit 33 of the zoom control device 30 sets one of the multiple distance measurement cameras 1 as the reference distance measurement camera 1. The zoom control unit 33 controls the focal length of the zoom when the distance measurement cameras 1 other than the reference distance measurement camera 1 capture the subject 2 as follows. The zoom control unit 33 obtains a focal length by multiplying the focal length of the zoom when the reference distance measurement camera 1 captures the subject 2 by a value obtained by dividing the distance from the reference distance measurement camera 1 to the center 2c of the subject 2 by the distance from the distance measurement camera 1 other than the reference distance measurement camera 1 to the center 2c of the subject 2. The zoom control unit 33 controls the zoom of the distance measurement cameras 1 other than the reference distance measurement camera 1 to achieve the focal length obtained in this manner.

[0059] By controlling the zoom focal length of the ranging cameras 1A-1D within the movable range of the zoom lens 1ZL, the zoom control device 30 can roughly match the amount of point clouds in the images captured by the ranging cameras 1A-1D, thereby generating a volumetric image that is almost visually natural.

[0060] <Second embodiment> The control device and control method for a multi-distance measuring camera according to the second embodiment are suitable for use in cases where it is not possible to temporarily move a subject photographed by multiple distance measuring cameras when adjusting each distance measuring camera. If it is not possible to temporarily move the subject, it is not possible to calculate the distance between two opposing distance measuring cameras 1, such as the distances Dac, Dca, Dbd, and Ddb in the first embodiment. The control device and control method for a multi-distance measuring camera according to the second embodiment are suitable for use in cases where it is not possible to calculate the distance between two opposing distance measuring cameras 1.

[0061] The control device for the multi-distance measuring camera according to the second embodiment can be configured with a zoom control device 30 shown in Fig. 3 or Fig. 9. In the second embodiment, explanations of matters common to the first embodiment may be omitted. The control device for the multi-distance measuring camera according to the second embodiment finds the center 2c of the subject 2 in the following manner, with the subject 2 placed between the distance measuring cameras 1A to 1D as shown in Fig. 9.

[0062] FIG. 10 shows in detail how the distance measuring camera 1 measures the distance to the subject 2. FIG. 10 shows only the lens barrel 11 and the sensor 14 of the distance measuring camera 1, which are simplified into a rectangular shape. The distance measured by the distance measuring camera 1 is expressed as the sum of the distance HRa from the surface of the subject 2 to the entrance pupil position, the distance HRb from the entrance pupil position to the image-side principal plane, and the distance HRc from the image-side principal plane to the sensor 14, which are shown by a dashed line. FIG. 10 shows an example of near-infrared light incident on the left end of the sensor 14. When converting distance data into point cloud data, the distance measuring camera 1 generates a point cloud represented by (x, y, z) coordinates with the focal length position from the surface of the sensor 14 to the focal point Fp as the origin.

[0063] The zoom control device 30, which is a control device for a multi-distance measurement camera according to the second embodiment, measures the distance to two adjacent distance measurement cameras 1 in global space represented by (x, y, z) coordinates, rather than to opposing distance measurement cameras 1, in accordance with the flowcharts shown in Figures 11A to 11E.

[0064] 11A, in step S21, the zoom control device 30 sets the distance measurement camera 1A to a mode in which the distance measurement camera 1A measures the distances to the distance measurement cameras 1B and 1D. The zoom control device 30 causes the VCSEL 13 in the distance measurement camera 1A to emit near-infrared light and stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1B to 1D.

[0065] In step S22, the user attaches the white lens caps 12 to the lens barrels 11 of the distance measuring cameras 1B and 1D. In step S23, the zoom control device 30 displays a near-infrared image on the display unit. At this time, the zoom control device 30 may further display a distance measuring image and a point cloud image on the display unit.

[0066] In step S24, the zoom control device 30 acquires a distance measurement value Dabcap to the distance measurement camera 1B and a distance measurement value Dadcap to the distance measurement camera 1D. In step S25, the zoom control device 30 converts the distance measurement values ​​Dabcap and Dadcap into point cloud coordinates PA[x,y,z] corresponding to the distance measurement value Dabcap and point cloud coordinates PA[x,y,z] corresponding to the distance measurement value Dadcap, respectively. In step S26, the user specifies the center of the lens cap 12 with the cursor.

[0067] Fig. 12 shows the near-infrared image NIa2 displayed on the display unit in step S23. As shown in Fig. 12, if the front of the lens cap 12 is visible in the near-infrared image NIa2, the user may specify the center of the lens cap 12 with the cursor in step S26. If the front of the lens cap 12 is not visible in the near-infrared image NIa2 due to the orientation of the distance measuring camera 1B or 1D, the user may attach the lens cap 12C shown in Fig. 6 to the lens barrel 11 of the distance measuring camera 1B or 1D in step S22. The user may specify the position of the lens cap 12C corresponding to the center of the lens cap 12 with the cursor in step S26.

[0068] In step S27, the zoom control device 30 extracts the point cloud coordinates PA[xb,yb,zb] and PA[xd,yd,zd] at the center of the lens cap 12 from each point cloud coordinate PA[x,y,z] and stores them in the memory unit 32.

[0069] 13A shows the point cloud coordinates PA[xb, yb, zb] and PA[xd, yd, zd] extracted in step S27 by the zoom control device 30. If the point cloud coordinates of the focal point Fp in the distance measuring camera 1A are the origin coordinates PA[0, 0, 0], the point cloud coordinates PA[xb, yb, zb] indicate the distance from the origin coordinates PA[0, 0, 0] to the lens cap 12 of the distance measuring camera 1B, and the point cloud coordinates PA[xd, yd, zd] indicate the distance from the origin coordinates PA[0, 0, 0] to the lens cap 12 of the distance measuring camera 1D.

[0070] 11B, in step S28, the zoom control device 30 sets the distance measurement camera 1B to a mode in which the distance measurement camera 1B measures the distances to the distance measurement cameras 1A and 1C. The zoom control device 30 causes the VCSEL 13 in the distance measurement camera 1B to emit near-infrared light, and stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1A, 1C, and 1D.

[0071] In step S29, the user attaches white lens caps 12 to the lens barrel 11 of the distance measuring cameras 1A and 1C. In step S30, the zoom control device 30 displays a near-infrared image on the display unit. At this time, the zoom control device 30 may also display a distance measuring image and a point cloud image on the display unit. Also in step S29, depending on the orientation of the distance measuring camera 1A or 1C, the lens cap 12C may be attached to the lens barrel 11 instead of the lens cap 12.

[0072] In step S31, the zoom control device 30 acquires a distance measurement value Dbacap to the distance measurement camera 1A and a distance measurement value Dbccap to the distance measurement camera 1C. In step S32, the zoom control device 30 converts the distance measurement values ​​Dbacap and Dbccap into point cloud coordinates PB[x,y,z] corresponding to the distance measurement value Dbacap and point cloud coordinates PB[x,y,z] corresponding to the distance measurement value Dbccap, respectively. In step S33, the user specifies the center of the lens cap 12 with the cursor.

[0073] In step S34, the zoom control device 30 extracts point cloud coordinates PB[xa,ya,za] and PB[xc,yc,zc] at the center of the lens cap 12 from each point cloud coordinate PB[x,y,z] and stores them in the memory unit 32.

[0074] 13B shows the point cloud coordinates PB[xa,ya,za] and PB[xc,yc,zc] extracted in step S34 by the zoom control device 30. If the point cloud coordinates of the focal point Fp in the distance measurement camera 1B are the origin coordinates PB[0,0,0], the point cloud coordinates PB[xa,ya,za] indicate the distance from the origin coordinates PB[0,0,0] to the lens cap 12 of the distance measurement camera 1A, and the point cloud coordinates PB[xc,yc,zc] indicate the distance from the origin coordinates PB[0,0,0] to the lens cap 12 of the distance measurement camera 1C.

[0075] 11C, in step S35, the zoom control device 30 sets the distance measurement camera 1C to a mode in which the distance measurement camera 1C measures the distances to the distance measurement cameras 1B and 1D. The zoom control device 30 causes the VCSEL 13 in the distance measurement camera 1C to emit near-infrared light, and stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1A, 1B, and 1D.

[0076] In step S36, the user attaches white lens caps 12 to the lens barrels 11 of the distance measuring cameras 1B and 1D. In step S37, the zoom control device 30 displays a near-infrared image on the display unit. At this time, the zoom control device 30 may also display a distance measuring image and a point cloud image on the display unit. Also in step S36, depending on the orientation of the distance measuring camera 1B or 1D, the lens cap 12C may be attached to the lens barrel 11 instead of the lens cap 12.

[0077] In step S38, the zoom control device 30 acquires a distance measurement value Dcbcap to the distance measurement camera 1B and a distance measurement value Dcdcap to the distance measurement camera 1D. In step S39, the zoom control device 30 converts the distance measurement values ​​Dcbcap and Dcdcap into point cloud coordinates PC[x,y,z] corresponding to the distance measurement value Dcbcap and point cloud coordinates PC[x,y,z] corresponding to the distance measurement value Dcdcap, respectively. In step S40, the user specifies the center of the lens cap 12 with the cursor.

[0078] In step S41, the zoom control device 30 extracts point cloud coordinates PC[xb,yb,zb] and PC[xd,yd,zd] at the center of the lens cap 12 from each point cloud coordinate PC[x,y,z] and stores them in the memory unit 32.

[0079] 13C shows the point cloud coordinates PC[xb, yb, zb] and PC[xd, yd, zd] extracted in step S41 by the zoom control device 30. If the point cloud coordinates of the focal point Fp in the distance measuring camera 1C are the origin coordinates PC[0, 0, 0], the point cloud coordinates PC[xb, yb, zb] indicate the distance from the origin coordinates PC[0, 0, 0] to the lens cap 12 of the distance measuring camera 1B, and the point cloud coordinates PC[xd, yd, zd] indicate the distance from the origin coordinates PC[0, 0, 0] to the lens cap 12 of the distance measuring camera 1D.

[0080] 11D, in step S42, the zoom control device 30 sets the distance measurement camera 1D to a mode in which the distance measurement camera 1D measures the distances to the distance measurement cameras 1A and 1C. The zoom control device 30 causes the VCSEL 13 in the distance measurement camera 1D to emit near-infrared light and stops the emission of near-infrared light from the VCSEL 13 in the distance measurement cameras 1A to 1C.

[0081] In step S43, the user attaches white lens caps 12 to the lens barrels 11 of the distance measuring cameras 1A and 1C. In step S44, the zoom control device 30 displays a near-infrared image on the display unit. At this time, the zoom control device 30 may also display a distance measuring image and a point cloud image on the display unit. Also in step S42, depending on the orientation of the distance measuring camera 1A or 1C, the lens cap 12C may be attached to the lens barrel 11 instead of the lens cap 12.

[0082] In step S45, the zoom control device 30 acquires a distance measurement value Ddacap to the distance measurement camera 1A and a distance measurement value Ddccap to the distance measurement camera 1C. In step S46, the zoom control device 30 converts the distance measurement values ​​Ddacap and Ddccap into point cloud coordinates PD[x,y,z] corresponding to the distance measurement value Ddacap and point cloud coordinates PD[x,y,z] corresponding to the distance measurement value Ddccap, respectively. In step S47, the user specifies the center of the lens cap 12 with the cursor.

[0083] In step S48, the zoom control device 30 extracts the point cloud coordinates PD[xa,ya,za] and PD[xc,yc,zc] at the center of the lens cap 12 from each point cloud coordinate PD[x,y,z] and stores them in the memory unit 32.

[0084] 13D shows the point cloud coordinates PD[xa, ya, za] and PD[xc, yc, zc] extracted in step S48 by the zoom control device 30. If the point cloud coordinates of the focal point Fp in the distance measurement camera 1D are set to the origin coordinates PD[0,0,0], the point cloud coordinates PD[xa, ya, za] indicate the distance from the origin coordinates PD[0,0,0] to the lens cap 12 of the distance measurement camera 1A, and the point cloud coordinates PD[xc, yc, zc] indicate the distance from the origin coordinates PD[0,0,0] to the lens cap 12 of the distance measurement camera 1C.

[0085] 11E, in step S49, the calculation unit 31 of the zoom control device 30 calculates a rotation angle θbrt and an origin coordinate PA[xcamb, ycamb, zcamb], which will be described later, and stores them in the storage unit 32. In step S50, the calculation unit 31 calculates rotation angles θdrt and PA[xcamd, ycamd, zcamd], which will be described later, and stores them in the storage unit 32. In step S51, the calculation unit 31 calculates a rotation angle θcrt and an origin coordinate PA[xcamc, ycamc, zcamc], which will be described later, and stores them in the storage unit 32, and ends the process.

[0086] Any one of the distance measuring cameras 1A to 1D can be used as the reference, and the distance measuring camera 1 is not limited to being used as the reference.

[0087] The processing of steps S49 to S51 will be specifically described. The point cloud coordinates PA[xb,yb,zb] and PA[xd,yd,zd], PB[xa,ya,za] and PB[xc,yc,zc], PC[xb,yb,zb] and PC[xd,yd,zd], and PD[xa,ya,za] and PD[xc,yc,zc] extracted by the zoom control device 30 are local point cloud coordinates of each ranging camera 1. Therefore, in steps S49 to S51, the calculation unit 31 converts the point cloud coordinates when ranging is performed with a camera other than the reference ranging camera 1 into global coordinates.

[0088] The conversion from point cloud coordinates to global coordinates will be specifically described with reference to Figures 14 and 15. As shown in Figure 14, if the distance measurement camera 1B is installed at an angle to the distance measurement camera 1A rather than at a right angle, the coordinates of the distance measurement camera 1B will be oblique coordinates as viewed from the distance measurement camera 1A. Therefore, the coordinate system of the distance measurement camera 1B does not match the coordinate system of the distance measurement camera 1A.

[0089] Fig. 15 is a diagram modeling Fig. 14. Using Fig. 15, a method of converting the orientations of the point cloud coordinates PB[xa,ya,za] and PB[xc,yc,zc] by the distance measuring camera 1B into the orientations of the coordinate system of the point cloud coordinates PA[xb,yb,zb] and PA[xd,yd,zd] by the distance measuring camera 1A will be described.

[0090] In FIG. 15, the direction of the optical axis of the distance measuring camera 1A from the origin coordinate PA[0,0,0] of the distance measuring camera 1A is the axis ZA, and the horizontal direction of the distance measuring camera 1A that is perpendicular to the axis ZA is the axis XA. The distance from the origin coordinate PA[0,0,0] of the distance measuring camera 1A to the lens cap 12 is the axis LC. The distance LC is known. The direction of the optical axis of the distance measuring camera 1B from the origin coordinate PB[0,0,0] of the distance measuring camera 1B is the axis ZB, and the horizontal direction of the distance measuring camera 1B that is perpendicular to the axis ZB is the axis XB. The distance from the origin coordinate PB[0,0,0] of the distance measuring camera 1B to the lens cap 12 is the same as the distance LC from the origin coordinate PA[0,0,0] of the distance measuring camera 1A to the lens cap 12.

[0091] The local point cloud coordinates of the lens cap 12 of the ranging camera 1A are PA[0,0,LC], and the point cloud coordinates of the lens cap 12 of the ranging camera 1A based on the local point cloud coordinates of the ranging camera 1B are PB[xa,ya,za]. The local point cloud coordinates of the lens cap 12 of the ranging camera 1B are PB[0,0,LC], and the point cloud coordinates of the lens cap 12 of the ranging camera 1B based on the local point cloud coordinates of the ranging camera 1A are PA[xb,yb,zb].

[0092] Here, the point cloud coordinates of the range finder camera 1A are used as a reference, and the point cloud coordinates on the axes XA, ZA, and the axis YA that is perpendicular to the axes XA and ZA (not shown in Fig. 15) are defined as global coordinates. The axis passing through the origin coordinate PB[0,0,0] parallel to the axis XA is defined as the axis XA', and the axis passing through the origin coordinate PB[0,0,0] parallel to the axis ZA is defined as the axis ZA'. The axis XA' is the axis obtained by rotating the axis XB by an angle θrt, and the axis ZA' is the axis obtained by rotating the axis ZB by an angle θrt.

[0093] By rotating the point cloud coordinates taken by the ranging camera 1B by an angle θrt on both axes XB and ZB and changing the origin coordinate PB[0,0,0] to the origin coordinate PA[0,0,0], the point cloud coordinates PB[xa,ya,za] and PB[xc,yc,zc] based on the near-infrared image NIa2 taken by the ranging camera 1B can be expressed in global coordinates based on the point cloud coordinates of the ranging camera 1A.

[0094] The zoom control device 30 calculates the angle θrt as follows. The angle between the axis ZA and a straight line from the origin coordinate PA[0,0,0] of the distance measurement camera 1A to the lens cap 12 of the distance measurement camera 1B is defined as θabcap. The angle between the axis ZB and a straight line from the origin coordinate PB[0,0,0] of the distance measurement camera 1B to the lens cap 12 of the distance measurement camera 1A is defined as θbacap. The angles θabcap and θbacap can be calculated by the following equations (17) and (18).

[0095] θabcap=arctan(-PA[xb] / PA[zb]) …(17) θbacap=arctan(PB[xa] / PB[za]) …(18)

[0096] If the angle at the intersection between a straight line from the origin coordinate PA[0,0,0] of the ranging camera 1A to the lens cap 12 of the ranging camera 1B and a straight line from the origin coordinate PB[0,0,0] of the ranging camera 1B to the lens cap 12 of the ranging camera 1A is θo, then the angle θrt is expressed in terms of angles θabcap and θbacap and angle θo by the following equation (19).

[0097] θrt=180-θabcap-(180-(180-(θbacap+θo))) =180-θabcap-θbacap-θo…(19)

[0098] The angle θo can be obtained as follows. A point PLa is a position that intersects perpendicularly from the position of the lens cap 12 of the range finder camera 1A to the straight line from the origin coordinates PA[0,0,0] of the range finder camera 1A to the lens cap 12 of the range finder camera 1B. Ap is the distance from the point PLa to the position of the lens cap 12 of the range finder camera 1A, and La is the distance from the point PLa to the lens cap 12 of the range finder camera 1B. A point PLb is a position that intersects perpendicularly from the position of the lens cap 12 of the range finder camera 1B to the straight line from the origin coordinates PB[0,0,0] of the range finder camera 1B to the lens cap 12 of the range finder camera 1A. Bp is the distance from the point PLb to the lens cap 12 of the range finder camera 1B, and Lb is the distance from the point PLb to the lens cap 12 of the range finder camera 1A.

[0099] The distances Ap and Bp are calculated by the following equations (20) and (21). Ap = LC(sinθabcap) … (20) Bp = LC(sinθbacap) … (21)

[0100] The distances La and Lb are calculated by the following equations (22) and (23).

[0101]

number

[0102]

number

[0103] The angle θo can be calculated using the following equation (24) or (25). θo=arctan(Bp / (Lb×Bp / (AP+Bp))) …(24) θo=arctan(AP / (La×Ap / (Ap+Bp))) …(25)

[0104] The distances La and Lb are calculated using the X-axis and Z-axis values ​​of the point cloud coordinates PA[xb,yb,zb] and PB[xa,ya,za], respectively, and may contain measurement errors due to noise, etc. Therefore, it is preferable to set the angle θo to the average value of the angle θo calculated by equation (24) and the angle θo calculated by equation (25). The angle θrt can be found by inputting the average angle θo into equation (19).

[0105] Equation (26) is an arithmetic equation for rotating the point cloud coordinates taken by the ranging camera 1B by the angle θrt on both axes XB and ZB to convert the point cloud coordinates taken by the ranging camera 1B into global coordinates based on the point cloud coordinates of the ranging camera 1A.

[0106]

number

[0107] The x', y', and z' obtained by inputting the point cloud coordinates PB[0,0,LC] of the lens cap 12 of the distance measuring camera 1B into x, y, and z in the formula (26) are defined as the point cloud coordinates BCAP[xb',yb',zb']. The point cloud coordinates BCAP[xb',yb',zb'] are global coordinates based on the distance measuring camera 1A. When the origin coordinates PB[0,0,0] of the distance measuring camera 1B are viewed from the origin coordinates PA[0,0,0] of the distance measuring camera 1A, the origin coordinates PA[xcamb,ycamb,zcamb] of the distance measuring camera 1B are vectorially subtracted from the point cloud coordinates PA[xb',yb',zb'] as shown in formula (27). The origin coordinates PA[xcamb,ycamb,zcamb] are global coordinates based on the distance measuring camera 1A.

[0108] PA[xcamb,ycamb,zcamb]=PA[xb,yb,zb]-BCAP[xb',yb',zb'] …(27)

[0109] 15 is the rotation angle of the local point cloud coordinates of the ranging camera 1B relative to the local point cloud coordinates of the ranging camera 1A, and is referred to as angle θbrt. The rotation angle of the local point cloud coordinates of the ranging camera 1D relative to the local point cloud coordinates of the ranging camera 1A is referred to as angle θdrt. Similarly, for the ranging camera 1D, the calculation unit 31 calculates the angle θdrt, and calculates the point cloud coordinates DCAP[xd', yd', zd'] and the origin coordinate PA[xcamd, ycamd, zcamd] of the global coordinates based on the ranging camera 1A.

[0110] The rotation angle of the local point cloud coordinates of the distance measuring camera 1C relative to the local point cloud coordinates of the distance measuring camera 1A is referred to as angle θcrt. For the distance measuring camera 1C, the zoom control device 30 obtains point cloud coordinates CCAP[xc',yc',zc'] and origin coordinates PA[xcamc,ycamc,zcamc] in the global coordinates based on the distance measuring camera 1A, based on the point cloud coordinates BCAP[xb',yb',zb'] and origin coordinates PA[xcamb,ycamb,zcamb] of the distance measuring camera 1B in the global coordinates.

[0111] First, the calculation unit 31 converts the point cloud coordinates PB[xc,yc,zc] of the lens cap 12 of the distance measuring camera 1C captured by the distance measuring camera 1B into global coordinates PA[xc,yc,zc] based on the distance measuring camera 1A. By obtaining the global coordinates PA[xc,yc,zc] obtained by inputting the angle θbrt and the point cloud coordinates PB[xc,yc,zc] into the formula (26), the point cloud coordinates PB[xc,yc,zc] of the lens cap 12 of the distance measuring camera 1C can be converted into the global coordinates PA[xc,yc,zc] based on the distance measuring camera 1A.

[0112] Next, instead of PA in the formulas (17) to (25), the origin coordinates PA[xcamb, ycamb, zcamb] are temporarily set as PA[0,0,0], and the position PA[xb'-xcamb, yb'-ycamb, zb'-zcamb] of the lens cap 12 of the distance measuring camera 1B and the global coordinates PA[xc, yc, zc] of the distance measuring camera 1C captured by the distance measuring camera 1B are input. Furthermore, instead of PB in the formulas (17) to (25), the point cloud coordinates PC[xb, yb, zb] of the lens cap 12 of the distance measuring camera 1B relative to the local origin coordinates PC[0,0,0] of the distance measuring camera 1C are input, thereby obtaining the angle θcrt.

[0113] In formula (26), the angle θcrt is input to θrt, and the point cloud coordinate PC[0,0,LC] of the lens cap 12 of the distance measuring camera 1C is input to x, y, and z, and the obtained x', y', and z' are defined as point cloud coordinates CCAP[xc', yc', zc']. As shown in formula (28), the origin coordinate PA[xcamc - xcamb, ycamc - ycamb, zcamc - zcamb] is obtained by subtracting the global coordinate CCAP[xc', yc', zc'] of the lens cap 12 of the distance measuring camera 1C from the origin of the distance measuring camera 1C from the global coordinate PA[xc, yc, zc] of the lens cap 12 of the distance measuring camera 1C seen from the distance measuring camera 1B.

[0114] PA[xcamc-xcamb, ycamc-ycamb, zcamc-zcamb]=PA[xc,yc,zc]-CCAP[xc',yc',zc'] …(28)

[0115] Up to equation (28), the origin of the ranging camera 1B is temporarily set to [0,0,0]. As shown in equation (29), in order to return the origin to its original position, PA[xcamb,ycamb,zcamb] is added to equation (28). The origin coordinates PA[xcamc,ycamc,zcamc] of the ranging camera 1C are obtained by adding PA[xcamc,ycamc,zcamc] to equation (28). PA[xcamc, ycamc, zcamc]=PA[xc,yc,zc]-CCAP[xc',yc',zc']+ PA[xcamb,ycamb,zcamb] …(29)

[0116] To improve the accuracy of the position of the distance measuring camera 1C, the calculation unit 31 may perform the following calculation. The calculation unit 31 calculates the angle θcrt based on the point cloud coordinates DCAP[xb',yb',zb'] and origin coordinates PA[xcamd,ycamd,zcamd] of the global coordinates. The calculation unit 31 calculates the point cloud coordinates CCAP[x,y,z] and origin coordinates PA[xcamc,ycamc,zcamc] of the global coordinates based on the distance measuring camera 1A, based on the coordinates of the distance measuring camera 1D.

[0117] The calculation unit 31 calculates the average of the point cloud coordinates CCAP[x,y,z] and origin coordinates PA[xcamc,ycamc,zcamc] based on the coordinates of the distance measuring camera 1D and the point cloud coordinates CCAP[x,y,z] and origin coordinates PA[xcamc,ycamc,zcamc] calculated based on the coordinates of the distance measuring camera 1B as described above. The calculation unit 31 sets the calculated average point cloud coordinates CCAP[x,y,z] and origin coordinates PA[xcamc,ycamc,zcamc] as the final point cloud coordinates CCAP[x,y,z] and origin coordinates PA[xcamc,ycamc,zcamc].

[0118] In this way, the zoom control device 30 can obtain the rotation angles θbrt, θcrt, θdrt, the origin coordinates PA[xcamb, ycamb, zcamb] of the distance measuring camera 1B, the origin coordinates PA[xcamc, ycamc, zcamc] of the distance measuring camera 1C, and the origin coordinates PA[xcamd, ycamd, zcamd] of the distance measuring camera 1D, using the distance measuring camera 1A as a reference.

[0119] Using these values, the zoom control device 30 (calculation unit 31) converts the local point cloud coordinates PB[x,y,z], PC[x,y,z], and PD[x,y,z] of the ranging cameras 1B to 1D into point cloud coordinates PA[x,y,z] in global coordinates based on the ranging camera 1A.

[0120] In Fig. 13B, the local point cloud coordinates of the distance measuring camera 1B obtained by measuring the distance to the subject 2 by the distance measuring camera 1B are PB[xhb, yhb, zhb]. The calculation unit 31 substitutes the angle θbrt for the angle θrt in the formula (26), substitutes the point cloud coordinates PB[xhb, yhb, zhb] for x, y, and z, and obtains PA[x'hb, y'hb, z'hb] as x', y', and z'. The calculation unit 31 vectorially adds the origin coordinates PA[xcamb, ycamb, zcamb] of the distance measuring camera 1B to PA[x'hb, y'hb, z'hb] to obtain the point cloud coordinates PA[x, y, z] of the global coordinates based on the distance measuring camera 1A.

[0121] 13C, the local point cloud coordinates of the distance measuring camera 1C obtained by measuring the distance to the subject 2 are PC[xhc, yhc, zhc]. The calculation unit 31 substitutes the angle θcrt for the angle θrt in the formula (26), substitutes the point cloud coordinates PC[xhc, yhc, zhc] for x, y, and z, and obtains PA[x'hc, y'hc, z'hc] as x', y', and z'. The calculation unit 31 vectorially adds the origin coordinates PA[xcamc, ycamc, zcamc] of the distance measuring camera 1C to PA[x'hc, y'hc, z'hc] to obtain the point cloud coordinates PA[x, y, z] of the global coordinates based on the distance measuring camera 1A.

[0122] In Fig. 13D, the local point cloud coordinates of the ranging camera 1D obtained by measuring the distance to the subject 2 by the ranging camera 1D are set as PD[xhd,yhd,zhd]. The calculation unit 31 substitutes the angle θdrt for the angle θrt in the formula (26), substitutes the point cloud coordinates PD[xhd,yhd,zhd] for x, y, and z, and obtains PA[x'hd,y'hd,z'hd] as x', y', and z'. The calculation unit 31 vectorially adds the origin coordinates PA[xcamd,ycamd,zcamd] of the ranging camera 1D to PA[x'hd,y'hd,z'hd] to obtain the point cloud coordinates PA[x,y,z] of the global coordinates based on the ranging camera 1A.

[0123] A method for determining the center 2c of the subject 2 will be described with reference to Fig. 16. The calculation unit 31 determines the average value of point cloud coordinates PA[x, y, z] in global coordinates based on the distance measuring camera 1A, which are obtained by measuring the distances of the subject 2 and converting them into a point cloud using the distance measuring cameras 1A to 1D, as the coordinates PA[Hcx, Hcy, Hcz] indicating the center 2c of the subject 2.

[0124] In Fig. 16, the range surrounded by AFOV (Angle Field Of View), which is the viewing angle of the distance measuring cameras 1A to 1D, is defined as the field of view range FV0. The range obtained by offsetting the field of view range FV0 inward by a distance FVoffset in the directions of the axes XA, ZA, and the axis YA (not shown) perpendicular to the axes XA and ZA in Fig. 15 is defined as the offset field of view range FV1. It is preferable that the calculation unit 31 determines the average value of only the point cloud coordinates PA[x, y, z] of the distance measuring cameras 1A to 1D within the offset field of view range FV1 as the coordinates PA[Hcx, Hcy, Hcz] of the center 2c of the subject 2.

[0125] The calculation unit 31 calculates the distances Da to Dd from the coordinates PA[Hcx, Hcy, Hcz] of the center 2c of the subject 2 to the origin coordinates PA[xcamb, ycamb, zcamb], PA[xcamc, ycamc, zcamc], and PA[xcamd, ycamd, zcamd] of the distance measuring cameras 1A to 1D based on equations (30) to (33).

[0126]

number

[0127] In this manner, the zoom control device 30 (calculation unit 31) can obtain the distances Da to Dd from the distance measurement cameras 1A to 1D to the center 2c of the subject 2 even if the subject 2 remains surrounded by the distance measurement cameras 1A to 1D. The calculation unit 31 calculates the distances Da to Dd for each frame of the distance measurement images generated by the distance measurement cameras 1A to 1D. The calculation unit 31 may calculate the distances Da to Dd when the subject 2 changes or moves.

[0128] As in the first embodiment, when a user sets the zoom focal length of the distance measurement camera 1A, which is a reference distance measurement camera, to FA, the zoom control device 30 (zoom control unit 33) controls the zoom focal lengths FB to FD of the distance measurement cameras 1B to 1D based on equations (7) to (16), respectively. The zoom control unit 33 controls the zoom focal lengths of the distance measurement cameras 1A to 1D within the movable range of the zoom lens 1ZL.

[0129] In the second embodiment as well, the zoom control unit 33 controls the zoom amount in the distance measurement cameras 1A-1D according to the distance between the subject 2 and each distance measurement camera 1, so that the subject 2 is photographed at approximately the same size by the distance measurement cameras 1A-1D. As a result, the angles of view when the distance measurement cameras 1A-1D photograph the subject 2 are approximately the same, so the amount of point clouds in the images photographed by each distance measurement camera 1 is approximately the same. Therefore, when the images photographed by the distance measurement cameras 1A-1D are synthesized to generate a volumetric image, almost no visual discomfort occurs.

[0130] The present invention is not limited to the first or second embodiment described above, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0131] 1A~1D Range finding camera 2. Subject 30 Zoom control device (control device for multi-range measuring camera) 31 Arithmetic section 32 Storage section 33 Zoom control section

Claims

1. a calculation unit that calculates a distance from each of a plurality of distance measuring cameras including a zoom lens, the distance measuring cameras being arranged to surround the subject, to the center of the subject; a zoom control unit that sets one of the plurality of distance measurement cameras as a reference distance measurement camera, and controls the zoom of the distance measurement cameras other than the reference distance measurement camera so that a focal length of the zoom of the distance measurement cameras other than the reference distance measurement camera when photographing the subject is a focal length obtained by multiplying a focal length of the zoom of the reference distance measurement camera when photographing the subject by a value obtained by dividing a distance from the reference distance measurement camera to a center of the subject by a distance from the distance measurement camera other than the reference distance measurement camera to the center of the subject; A control device for a multi-ranging camera comprising:

2. The calculation unit is calculating a width of the subject in a direction of the two opposing distance measurement cameras based on a distance between two opposing distance measurement cameras among the plurality of distance measurement cameras and a distance from each of the distance measurement cameras to a surface of the subject; Calculating the distance from each of the distance measuring cameras to the center of the subject based on the distance from each of the distance measuring cameras to the surface of the subject and the width of the subject The control device for a multi-distance measuring camera according to claim 1 .

3. The calculation unit is converting the local origin coordinates of each of the distance measuring cameras and the local point cloud coordinates in which each of the distance measuring cameras measures the distance to the subject into origin coordinates of a global coordinate system based on the reference distance measuring camera and point cloud coordinates indicating the distance to the subject; determining a center of the object based on an average value of point cloud coordinates of global coordinates indicating distances to the object obtained by the plurality of distance measuring cameras; Calculate the distance from the center of the object to the origin coordinate of the global coordinate of each of the distance measuring cameras The control device for a multi-distance measuring camera according to claim 1 .

4. The control device for a multi-ranging camera according to claim 3, wherein the calculation unit converts the local origin coordinates of each ranging camera and the local point cloud coordinates in which each ranging camera measures the distance to the subject into origin coordinates of a global coordinate system and point cloud coordinates indicating the distance to the subject, based on the distance between each ranging camera and an adjacent ranging camera to each ranging camera.

5. A computer device that controls multiple distance measuring cameras arranged around the subject, Calculating a distance from each of the plurality of distance measuring cameras to a center of the subject; One of the plurality of distance measuring cameras is set as a reference distance measuring camera, and the zoom of the distance measuring cameras other than the reference distance measuring camera is controlled so that the focal length of the zoom when the reference distance measuring camera photographs the subject is a focal length obtained by multiplying the focal length of the zoom when the reference distance measuring camera photographs the subject by a value obtained by dividing the distance from the reference distance measuring camera to the center of the subject by the distance from the distance measuring camera other than the reference distance measuring camera to the center of the subject. A method for controlling a multi-range camera.

Citation Information

Patent Citations

  • Motion capture camera system and calibration method

    WO2020105697A1