Filming systems, video server systems

Abstract

To achieve high functionalization and high performance of a photographing system on the basis of an obtained video signal.SOLUTION: A control section 42 can obtain map information in the surroundings by a network connection section 43 via a network when position information of an imaging device 10 is recognized from obtained imaging information. The control section 42 can display a video based on a video signal on a display section 45 being a display. The control section 42 analyzes the video on the basis of the map information the imaging information, recognizes buildings and the like (a building, a specific terrain, bridges, and the like) as subjects in the video, for example, associates them with the map information by using the imaging information, and can identify the buildings and the like specifically.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a photography system for taking pictures and a video server system for receiving the video signals from the photography system and performing various processes. [Background technology]

[0002] A radio relay device for television broadcasting (Field Pickup Unit: FPU) relays digitized signals of video captured outside the broadcasting station when transmitting them to the broadcasting station. By improving the functionality of the FPU device, stable and high-quality broadcasting can be achieved. For example, Patent Document 1 describes an FPU device that automatically optimizes the transmission rate of digitized signals based on the video signal. This allows the video signal to be transmitted to the broadcasting station in good transmission conditions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-104579 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, the transmission conditions of the video signal are optimized by examining the video signal itself to be transmitted. In this way, there is a demand for a technology that not only optimizes the transmission conditions based on the obtained video signal, but also improves the functionality and performance of the imaging system.

[0005] The present invention has been made in view of the above circumstances, and has as its object to solve the above problems. [Means for solving the problem]

[0006] The present invention relates to a photography system using an imaging device that captures images and outputs video signals, wherein the imaging device includes position information of the imaging device at the time of capturing images. fruit The imaging device includes an imaging information output unit that outputs imaging information that indicates the state of the imaging device when capturing an image, a communication unit that obtains the video signal and the imaging information from the imaging device, and an analysis unit that recognizes a subject in an image captured by the video signal, recognizes map information about the surroundings of the imaging device including information about structures surrounding the imaging device based on the position information, and recognizes a correspondence between the subject and the structures by comparing the video with the map information, and the analysis unit When a part of information other than the position information in the imaging information is missing, the missing information is used as a parameter, an imaging range for each parameter in the map information captured by the imaging device is calculated, and a correspondence relationship between the subject and the structure in the imaging range is recognized, and the value of the parameter when the subject and the structure most closely match is determined to correspond to the value of the missing information, and the structure that is determined not to be recognized in the imaging range due to information other than the missing information is excluded from the recognition of the correspondence relationship. . The image capturing apparatus may further include a display unit that displays the image while indicating the correspondence between the subject and the structure. In addition, the analysis unit may recognize in advance the object to be imaged by the imaging device, acquire the map information of the surroundings of the object, and by comparing the map information with the image based on the video signal obtained from the imaging device, issue an instruction to the imaging device to control the imaging device so that the object is imaged. The imaging information also includes the altitude of the imaging device, the direction of the center of the field of view when imaging, tilt angle around the optical axis, This may include either the field of view range or the distance to the imaged object. The analysis unit may also register identification information of the structure recognized as the subject. The system may also include a video server system having the imaging device, the communication unit, and the analysis unit, and a transmission device that obtains the video signal and the imaging information from the imaging device and transmits them to the video server system by wireless communication via a directional antenna, wherein the imaging device is fixed to the transmission device and an adjustment display unit is provided that displays the orientation of the antenna in the video that includes the subject whose correspondence with the structure has been identified. The present invention provides A video server system that receives and processes a video signal obtained by an imaging device, and receives from the imaging device a video signal including position information of the imaging device at the time of imaging, together with the video signal. fruitThe imaging device includes a communication unit that receives imaging information that indicates the state of the imaging device when capturing an image, and an analysis unit that recognizes a subject in an image from the video signal, recognizes map information around the imaging device including information about structures around the imaging device from the position information, and recognizes a correspondence between the subject and the structures by comparing the image with the map information, and the analysis unit When a part of information other than the position information in the imaging information is missing, the missing information is used as a parameter, an imaging range for each parameter in the map information captured by the imaging device is calculated, and a correspondence relationship between the subject and the structure in the imaging range is recognized, and the value of the parameter when the subject and the structure most closely match is determined to correspond to the value of the missing information, and the structure that is determined not to be recognized in the imaging range due to information other than the missing information is excluded from the recognition of the correspondence relationship. . [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the functionality and performance of the imaging system based on the obtained video signal. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a configuration of an imaging system according to an embodiment. [Figure 2] FIG. 1 is a diagram showing a configuration of an imaging device used in an imaging system according to an embodiment. [Figure 3] FIG. 1 is a diagram showing the configuration of a video server system used in a photography system according to an embodiment. [Figure 4] 1 is an example of an image obtained by the imaging system according to the embodiment. [Figure 5] 10 is an example of map information used in the imaging system according to the embodiment. [Figure 6] 10 is an example in which a virtual image obtained by appropriately setting parameters is displayed together with an actual image in the imaging system according to the embodiment. [Figure 7] FIG. 10 is a diagram showing a case where the field of view range in the horizontal direction is determined in map information. [Figure 8] FIG. 10 is a diagram showing a case where the field of view range in the vertical direction is determined in map information. [Figure 9] FIG. 10 is a diagram showing ranges corresponding to depth of field in map information. [Figure 10] 10 shows an example of an image in which the focus conditions for each subject in the image are different. [Figure 11] This is an example in which the distance to each subject in the video is recognized. [Figure 12] This is an example in which the distance to each structure in the map information is recognized. [Figure 13] (a) shows an example of an image with optical distortion, and (b) shows an example of the image after correction. [Figure 14] 10A and 10B are diagrams illustrating operations when issuing instructions to bring a desired object into the field of view. [Figure 15] The configuration of the imaging device when adjusting the antenna (a) and the image display (b) are shown.

[0009] Next, a photographing system according to an embodiment of the present invention will be described in detail with reference to the drawings. As shown in Fig. 1, this photographing system 100 includes an imaging device 10 that captures images of a subject to obtain video data (generates a video signal), an FPU device (transmission device) 30 that digitizes the video signal and transmits (relays) it to a broadcasting station, and a video server system 40 that receives the video signal from the broadcasting station, processes the video, and stores it. The imaging device 10 and the FPU device 30 are mounted on, for example, a filming vehicle or helicopter, and the video server system 40 is connected to the FPU device 30 via wireless communication and is fixed at a fixed location on the ground (for example, a broadcasting station).

[0010] In this photography system 100, the image capture device 10 recognizes its own position information and imaging conditions (posture, zoom magnification, etc.), and the imaged subject is recognized using AI technology, etc., thereby recognizing the subject's position information. This information is transmitted to the video server system 40 along with the imaging data. The video server system 40 can use this information to perform various processes.

[0011] 2 is a diagram showing the configuration of this imaging device 10. In this imaging device 10, an image is captured by an imaging element 12 via an optical system 11, and an analog video signal is generated. This video signal undergoes various processes before transmission in a video signal processing unit 13, and is then output to the outside from a video signal output unit 14 as a digital video signal conforming to HD-SDI standards or the like, which is received by the FPU unit 30. Also provided is a control unit (imaging device control unit) 15 that controls the entire imaging device 10. The control unit 15 controls the optical system 11 and the imaging element 12, and controls the field of view (zoom magnification, etc.), focal length, aperture, gain, etc. during imaging.

[0012] Furthermore, various components are provided in this imaging device 10 to recognize the state of the imaging device 10 itself at the time of imaging. First, in Fig. 2, a position information recognition unit 16 that recognizes the position (position information) of the imaging device 10 by receiving a GPS signal, and an altitude sensor 17 that recognizes the altitude (elevation) of the imaging device 10 are provided. These enable the control unit 15 to recognize the position of the imaging device 10 in three-dimensional space at the time of imaging.

[0013] 2, there are provided an orientation sensor 18 that recognizes the orientation (azimuth angle) of the optical axis (center of field of view) of the imaging device 10, and an inclination sensor 19 that recognizes the elevation angle from the horizontal direction of the optical axis of the imaging device 10 and the inclination angle around the optical axis. These sensors enable the control unit 15 to recognize the direction in three-dimensional space of the center of field of view of the imaging device 10 and the inclination angle around the optical axis.

[0014] Furthermore, the control unit 15 can recognize the field of view range around the field of view center in the horizontal and vertical directions from the control information of the optical system 11 and the image sensor 12. Note that the field of view range recognized here in the horizontal and vertical directions is the field of view range when the tilt angle around the optical axis recognized by the tilt sensor 19 is zero as described above, and when this tilt angle is not zero, this field of view range will have a shape tilted according to the tilt angle.

[0015] 2, an illuminance sensor 20 that recognizes the illuminance of the environment of the imaging device 10, and a time recognition unit 21 that recognizes the current date and time are also provided.

[0016] With the above configuration, when imaging device 10 captures an image, control unit 15 can recognize position information (horizontal position and height) and field of view information (direction of the field of view center and extent of the field of view in three-dimensional space) of imaging device 10. Control unit 15 can also recognize the time and illuminance at which the image was captured, and can output all of this recognized information to the outside together with the video data from imaging information output unit 22 as imaging information attached to the video data.

[0017] In the above configuration, the altitude sensor 17, illuminance sensor 20, and time recognition unit 21 may be provided on the helicopter or the like carrying the imaging device, rather than in the imaging device itself. Even in this case, the information obtained from these sensors can be included in the imaging information as described above.

[0018] However, as described above, in the imaging device 10, each item in the imaging information is recognized by a different component, and there are cases where information on only some of the items is obtained with high accuracy. In the example described below, a case where information on only some of the items is obtained will be described.

[0019] 1, the imaging device 10 outputs a video signal and imaging information as described above, and the FPU device 30 receives these signals via a wired connection from the imaging device 10, which is located nearby, and transmits them via wireless communication to the video server system 40, which is located in a remote location. At this time, the FPU device 30 modulates these signals in accordance with an appropriately selected transmission method, and the video server system 40 demodulates them, but the configurations related to this modulation and demodulation are well known and are unrelated to the present invention, so they will not be described below.

[0020] 3 is a diagram showing the configuration of video server system 40. This system is provided with a wireless communication unit (communication unit) 41 that obtains the video signal and imaging information from FPU device 30 via wireless communication. Instructions for various controls on the FPU device 30 side are sent from video server system 40 to FPU device 30 via wireless communication or other communication means by wireless communication unit 41. Also provided is a control unit (analysis unit) 42 that controls the entire video server system 40.

[0021] When the control unit 42 recognizes the location information of the imaging device 10 from the acquired imaging information, it can obtain map information of the surrounding area (the area that may be captured by the imaging device 10) via the network using the network connection unit 43. The obtained video signal, imaging information, and map information are stored in a storage unit 44, which is a large-capacity hard disk or non-volatile memory. Note that if the imaging device 10 (FPU device 30) captures images while moving, map information can be obtained and stored, for example, for each area in which the imaging device 10 is located. Alternatively, if the target to be captured is predetermined, map information including this target may be stored in the storage unit 44 in advance, rather than obtaining map information via the network connection unit 43 after recognizing the location information.

[0022] Furthermore, the control unit 42 can display an image based on the video signal on the display unit 45. The control unit 42 analyzes the image based on the map information and the imaging information, and can, for example, recognize structures or the like (buildings, specific topography, bridges, etc.) as subjects in the image, and can specifically identify the structures or the like by correlating them with the map information using the imaging information. This operation will be specifically explained below. This operation can be performed periodically (for example, every second) when performed in real time when the video signal (video) is acquired, but can also be performed on the video signal stored in the storage unit 44, as will be described later.

[0023] FIG. 4 shows a schematic example of an image obtained from the imaging device 10 (video signal) in this case. The objects are large structures such as buildings that can be recognized by well-known pattern recognition techniques, and three objects a1 to a3 are recognized here. The grid-like lines in FIG. 4 are virtual grid-like lines corresponding to the coordinates shown in FIG. 5, which will be described later. The control unit 42 can specifically identify the objects a1 to a3 in this image by comparing the image with the map information described above. As mentioned above, not all items of information are necessarily available in the imaging information, and therefore, the following describes analysis based on the information for each item.

[0024] First, a case will be described in which the position information of the imaging device 10, particularly the horizontal position information, has been determined from the imaging information. In this case, the control unit 42 obtains map information about the surrounding area of ​​this position. FIG. 5 shows the obtained map information and the position P of the imaging device 10 within it. Here, the horizontal axis is divided into three sections, A to C, and the vertical axis is divided into nine sections, 1 to 9, and the imaging device 10 is positioned at the center of "B8." The control unit 42 can store this map information in the memory unit 44.

[0025] Here, candidates for the above-mentioned subjects a1 to a3 that can be recognized in the image captured by the imaging device 10 (Figure 4) using well-known pattern recognition technology include buildings (structures) such as buildings, and here six structures G, H, I, X, Y, and Z are located at the locations shown in the figure.

[0026] In this case, it is possible to assume that the imaging device 10 is at this position, and various conditions at the time of imaging are used as parameters to simulate an image of the terrain of this map information. The simulated image is compared with the actually obtained image (FIG. 4), and the parameters that produce the closest result can be estimated to be the actual ones. These parameters include the altitude of the imaging device 10, the direction of the field of view center, the field of view range, and the tilt angle around the optical axis. In this case, it is possible to recognize the correspondence between the subjects a1 to a3 recognized in FIG. 4 and the structures G, H, I, X, Y, and Z in FIG. 5.

[0027] 6(a) and 6(b) show examples of the results of comparing each structure to be recognized in a virtual image when these parameters are appropriately set with the actual image (FIG. 4). This display is performed, for example, on the display unit 45. Here, the actual image is depicted with solid lines, and the virtual image is depicted with dotted and dashed lines. Here, structures G, I, and H are considered as candidates for subjects a1 to a3. In FIG. 6(a), the angle in the vertical direction of the center of the field of view differs from that of the actual image, and in FIG. 6(b), the tilt angle around the optical axis differs from that of the actual image. The virtual image to be compared with the actual image is recognized from map information, and the type of information that can be used may be a topographical map, aerial photograph, or any other suitable form that facilitates comparison.

[0028] Here, the number of structures that are candidates for the subject can be reduced by excluding as candidates those structures that are clearly outside the field of view based on the position information of the image capture device 10. For example, based on the horizontal position of the image capture device 10 (FIG. 5), structure X is in the shadow of structure I, so structure X can be excluded from consideration.

[0029] The control unit 42 performs such a comparison and can estimate that the parameters (center of the field of view, tilt angle around the optical axis) that best match the results of Figure 4 are actual, and can recognize, for example, that subjects a1, a2, and a3 in Figure 4 correspond to structures G, I, and H in Figure 5, respectively.

[0030] Next, a case will be described in which the horizontal field of view range can be recognized in addition to the horizontal position of the imaging device 10. FIG. 7 shows an example in which the horizontal field of view range R1 in this case is added to the state shown in FIG. 5. In this case, the range R1 is specified as the field of view range. In this case, altitude (vertical position information) and vertical field of view information (direction of the field of view center, field of view range) are used as parameters, and an image of the terrain of this map information is simulated in the same manner as in FIG. 6. The simulated image is compared with the actually obtained image (FIG. 4). The altitude, etc., at the time of obtaining the closest result is estimated to be the actual one, and the correspondence between the objects recognized in FIG. 4 and the structures G, H, I, X, Y, and Z in FIG. 5 can be recognized. In this case, as described above, structure X, which is in the shadow of structure I, and structure Z, which is clearly outside the field of view range R1 in FIG. 7, can be excluded as candidates for the objects recognized in FIG. 4. Only the structures G, H, I, and Y that were not excluded are selected as candidates for objects a1 to a3, and the correspondence between the objects and the structures can be examined in the same manner as in FIG. 6.

[0031] Next, a case will be described in which the altitude (vertical position information) and vertical field of view information of the imaging device 10 can be recognized in addition to the horizontal position information and horizontal field of view information of the imaging device 10. FIG. 8 is a side view of the vertical situation in this case, seen from the right side in FIG. 7. The imaging device 10 is located at a predetermined height h from the ground at "B8" ("8" in FIG. 8), and the control unit 42 can recognize this height from the imaging information. The field of view range of the imaging device 10 in the vertical direction is R2 in FIG. 8. The height (total height) of each structure from the ground in FIG. 8 is assumed to be as shown, and this can be recognized from map information.

[0032] In this case, similarly to the case described above, structures X and Z in Fig. 8 can be excluded as candidates. Furthermore, structure Y is outside the vertical field of view range R2 in Fig. 8, so it can also be excluded as a candidate that is close to the image capture device 10. Therefore, only structures G, H, and I that were not excluded can be considered as candidates for subjects a1 to a3 recognized in Fig. 4, and the correspondence between the subjects and structures can be investigated in the same manner as in Fig. 6.

[0033] In the above example, the field of view range in the horizontal direction is R1 and the field of view range in the vertical direction is R2, in which case a rectangular image as shown in Figure 4 is obtained. However, the shape of the field of view is not limited to this rectangular shape, and even if it is rectangular, it may be tilted around the optical axis as in the virtual image of Figure 6(b). Even in such cases, the control unit 42 can similarly determine whether each structure is within the field of view range.

[0034] 4 corresponds to one of the structures G, H, I, X, Y, and Z. In this case, the more accurately the number of items in the imaging information is recognized, the easier and more accurately the correspondence between each of the objects and each structure can be estimated.

[0035] Furthermore, the control unit 42 can estimate the distance between each recognized subject and the image capture device 10 based on the focus state of each subject in the recognized video (FIG. 4). In this case, when estimating the correspondence between each subject and each structure, this distance can be used in addition to the position information of the image capture device 10 as described above.

[0036] If the control information for optical system 11 is included in the imaging information, control unit 42 can recognize the distance range in which the image is in focus in the image shown in FIG. 4 from this control information. FIG. 9 shows an example in which the distance range from imaging device 10 is S1. Distance range S1 corresponds to the depth of field of optical system 11. Note that while FIG. 9 is a plan view corresponding to FIG. 5, in reality, the distance range corresponding to the depth of field can also be recognized in the vertical direction shown in FIG. 8.

[0037] FIG. 10 illustrates a case where the focus conditions for each subject are different from those in FIG. 4 . In this case, subject a2 is in focus, resulting in high contrast at its edge (outline), while subjects a1 and a3 are out of focus, resulting in blurred edges. The control unit 42 can recognize this condition using well-known edge analysis techniques in the image (video) profile. Therefore, if the position of the image capture device 10 is determined as shown in FIG. 9 , the control unit 42 can estimate that subject a2 is structure H because structure H is the only object within the distance range S1 from the image capture device 10. Once the position of the image capture device 10 is determined and subject a2 is estimated to be structure H, the control unit 42 can estimate the correspondence between the other subjects a1 and a3 and other structures, in addition to the posture, field of view, etc., of the image capture device 10, even if the posture and field of view of the image capture device 10 are unknown. If the depth of field is greater than S1 in Figure 9, then in addition to structure H, structures G and I could also be candidates for the in-focus subject. However, if the range of distances to the subject is recognized in this way, the structures that are candidates for the subject are more limited, allowing the above estimation to be performed more accurately.

[0038] Furthermore, the distance to the subject can be determined more accurately by using a distance measuring means using, for example, laser light, rather than by using the depth of field as described above. In this case, the control unit 15 of the imaging device 10 can include information on the distance to the subject in the video in the imaging information, and the control unit 42 of the video server system 40 can recognize this distance. This corresponds to the case in Figure 9 where S1 is very small (when the depth of field is small).

[0039] In this case, as shown in Fig. 11, it is possible to recognize the distances (d1, d2, d3) between each subject in the video of Fig. 4 and the imaging device 10. On the other hand, the control unit 42 can recognize the distance between each structure in Fig. 5 and the imaging device 10 from the position information of the imaging device 10 and the map information, as shown in Fig. 12, and can recognize that the distances to structures G, H, and I are D1, D2, and D3, respectively. Under these conditions, it is possible to check the correspondence between each subject and each structure, and to estimate this correspondence more easily and with higher accuracy.

[0040] In this way, if the distance to the subject can be recognized, the control unit 42 can estimate the correspondence between each subject and each structure particularly easily and with high accuracy. If the imaging device 10 is provided with a distance measuring means, this distance can be included as one item of the imaging information, and the video server system 40 can obtain this information. However, as mentioned above, even if this distance is not included in the imaging information, it can be estimated from the focus status of the subject in the video.

[0041] LiDAR (laser radar) is known as a device that oscillates pulsed laser light to obtain an optical image (takes an image) and also obtains distance information corresponding to the image. For this reason, LiDAR can be particularly preferably used as the imaging device 10. In this case, the distance to every location in the image can be recognized, making it possible to particularly easily and accurately estimate the correspondence between each subject and each structure.

[0042] As described above, the control unit 42 recognizes the correspondence between the objects and structures in the image by analyzing the image based on the received video signal. If the map information includes identification information about each structure (for example, the name and address of the building), the control unit 42 can associate this with the object and store it in the storage unit 44. This makes it particularly easy to manage the image when it is used later.

[0043] Furthermore, when analyzing the image as described above, if there is distortion in the image due to, for example, the characteristics of the optical system 11, this can be corrected before performing the above analysis, thereby enabling a more accurate analysis.

[0044] FIG. 13(a) shows an image corresponding to FIG. 4 in the presence of such distortion. In this case, if the video server system 40 recognizes the characteristics of the imaging device 10 in advance, it can perform filtering on the video signal to obtain an image in which the distortion has been compensated. If the parameters for such filtering are stored in the storage unit 44 in advance, the control unit 42 can compensate for the distortion in the actual image shown in FIG. 13(a) and obtain the original image shown in FIG. 13(b). This compensated image can then be used to properly analyze the correspondence between the subject and the structure shown in FIG. 6. Note that instead of compensating for the distortion in the actual image, a correction can be made to add the distortion to a virtual image obtained from map information, and the analysis shown in FIG. 6 can be performed.

[0045] As described above, various types of virtual images, such as topographical maps and aerial photographs, are used as appropriate as images to be compared with actual images. Images that are actually the subject of analysis may be daytime images or nighttime images. Therefore, when displaying images on the display unit 45 as shown in FIG. 6, it is preferable to adjust the brightness, contrast, white balance, and the like of the images to be analyzed. In this case, such corrections can be made using the time included in the imaging information and the illuminance recognized by the illuminance sensor 20.

[0046] By using the imaging system 100 or the video server system 40 that recognizes the correspondence between the subject and the structure as described above, the subject in the video can be recognized in real time as a known structure. This enables the following operations, for example.

[0047] Generally, the video obtained from the imaging device 10 is not used as is for public broadcast (distribution), but rather the video is often processed and stored in the storage unit 44. This processing includes, for example, rendering (blurring, etc.) of a specific subject. Video server systems that perform such processing are described, for example, in Japanese Patent Application Laid-Open Nos. 2012-34218 and 2019-62381. As described above, if the subject in the video is recognized as a known structure and its identification information is recognized, it becomes particularly easy to automatically perform such processing in the video server system 40. In this case, the imaging device 10 can perform the above operation simply by transmitting the imaging information along with the video signal. The imaging information is recognized solely by the imaging device 10, as described above.

[0048] Furthermore, for example, when the imaging device 10 (and FPU device 30) is mounted on a helicopter or the like and photographed by a photographer, in many cases only the minimum necessary equipment is mounted on the helicopter, etc. In such cases, when photographing is performed while the helicopter, etc. is moving, the subject changes from moment to moment, and the photographer is often unable to directly obtain map information such as that shown in Figure 5, and is often unable to recognize in real time the specific identity of the subject in the photographed video.

[0049] In this way, if the photographer cannot properly identify the subject in real time, the photographer may not be able to properly shoot. In contrast, in the above-described photography system 100, the video server system 40 can recognize the specific identity of the subject in real time, so the control unit 42 can issue appropriate instructions to the FPU device 30 located near the photographer.

[0050] Figure 14 shows an example of an image (a) and map information (b) in such a case. Here, it is assumed that the imaging device 10 is located at position "C5" in Figure 5, and its horizontal field of view is R3 in Figure 14(b). In this image (Figure 14(a)), the control unit 42 can recognize structures H, I, and X as subjects, as described above.

[0051] In this case, if the object to be actually photographed is structure G, structure G will not be included in the image (FIG. 14(a)), and control unit 42 can recognize this by recognizing structures H, I, and X as subjects. In this case, in order to photograph structure G, control unit 42 can issue an instruction to FPU device 30 (imaging device 10) to change the field of view range to R4 in FIG. 14(b), for example. Alternatively, the control unit 42 may instruct imaging device 10 to move downward in FIG. 14(b) without changing the field of view.

[0052] This instruction can be displayed as text information on the monitor of the imaging device 10 or the FPU device 30, for example. Alternatively, if the azimuth angle or elevation angle of the imaging device 10 is controlled by controlling a stage, a control signal for this purpose may be transmitted from the video server system 40 to the FPU device 30 (imaging device 10). Furthermore, if the imaging device 10 is mounted on a device whose flight is controlled, such as a drone, a control signal for this device may be transmitted.

[0053] When the subject to be photographed is set in advance in this way, map information about the surroundings of the subject is stored in the memory unit 44 before the actual photographing begins, and the settings of the optical system 11 in the imaging device 10 (such as the lens to be used and the optical system parameters included in the imaging information) are recognized in advance, and the above analysis is performed in real time at the same time as the photographing begins, thereby enabling the subject to be photographed promptly.

[0054] Furthermore, the FPU device 30 must communicate wirelessly with an opposing FPU device that is installed in a building, a radio tower, or the like and connected to the video server system 40, while facing each other with high accuracy. If the building, radio tower, or other structure can be recognized as a subject as described above, instructions to adjust the antenna direction of the FPU device 30 can be given to the cameraman, similar to the instructions given to the cameraman regarding filming. In other words, the above-described filming system 100 can ensure smooth communication between the FPU device 30 and the video server system 40.

[0055] FIG. 15(a) is a diagram showing the configuration of the imaging device 10 and the FPU device 30 in such a case. Here, the FPU device 30 is provided with a parabolic antenna 31 and a viewfinder 32. The antenna 31 has directivity as indicated by the arrow in the figure. The field of view of the imaging device 10 has a field of view range that generally follows the directivity of the antenna 31. In this state, the FPU device 30 is fixed to, for example, a camera platform, so that the orientation of the FPU device 30 (antenna 31) and the imaging device 10 can be adjusted.

[0056] In this state, the image captured by the imaging device 10 before the transmission of the video signal is assumed to be as shown in Figure 15(b). In this case, the FPU device 30 can display the image based on the video signal obtained from the imaging device 10 on the viewfinder (adjustment display unit) 32, similar to the display unit 45 in the video server system 40. Here, as in the above example, it is assumed that structures G, I, and H are recognized in this image. It is also assumed that the opposing FPU device is installed above structure I.

[0057] Here, since the imaging device 10 is fixed to the FPU device 30, the FPU device 30 can recognize the imaging information and display the orientation of the antenna 31 in the image as a point in the image. In Figure 15(b), this orientation is indicated by a cross mark M. This point needs to be aligned with the top of the structure I.

[0058] In this case, the control unit 42 can issue an instruction to change the orientation of the FPU device 30 (antenna 31) so that the cross mark M overlaps the top of the structure I in the viewfinder 32. This instruction can be issued by the FPU device 30 (image capture device 10), as in the case described above. Alternatively, this adjustment can be made without issuing such an instruction by the photographer changing the orientation of the image capture device 10 while looking at the display in Figure 15(b) and recognizing the positional relationship between the cross mark M and the structure. In other words, by performing the above analysis in real time, such antenna adjustment can be made efficiently.

[0059] In the above example, the above analysis is performed in real time on the video server system 40 side to optimize the shooting conditions and communication conditions. However, instead of performing this analysis in real time, the same analysis may be performed at a desired time after the video signal is stored in the storage unit 44. In this case, after this analysis process, for example, rendering processing of the video can be easily performed. If the imaging information includes the time of imaging, the correspondence between the video signal and the imaging information can be easily recognized, and such processing can be easily performed.

[0060] In the above example, the video server system 40 (control unit 42) performs the above-described analysis of the subject in the video. However, this analysis can be performed in a device other than the video server system, such as an imaging device or an FPU device, as long as map information is available. Alternatively, it can be performed in other components not shown in Figure 1. In other words, an analysis unit that performs the above-described analysis and a display unit that displays the video shown in Figure 6 and the like can be provided in any location of the components that make up this shooting system.

[0061] The present invention has been described above based on an embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components, and that such modifications are also within the scope of the present invention. [Explanation of symbols]

[0062] 10. Imaging device 11 Optical system 12 Image sensor 13 Video signal processing section 14 Video signal output section 15 Control section (imaging device control section) 16 Location information recognition unit 17 Altitude Sensor 18 Compass Sensor 19 Inclination sensor 20 Illuminance sensor 21 Time recognition section 30 FPU device (transmission device) 31 Antenna 32 Viewfinder (adjustment display) 40 Video Server System 41 Wireless Communication Unit (Communication Unit) 42 Control unit (analysis unit) 43 Network Connection 44 Storage section 45 Display section 100 Shooting System a1~a3 Subject G, H, I, X, Y, Z structures M cross mark

Claims

1. An imaging system using an imaging device that captures images and outputs video signals, The imaging device is an imaging information output unit that outputs imaging information that includes position information of the imaging device at the time of imaging and indicates a state of the imaging device at the time of imaging, a communication unit that receives the video signal and the imaging information from the imaging device; an analysis unit that recognizes a subject in an image obtained by the video signal, recognizes map information about the surroundings of the imaging device, including information about structures surrounding the imaging device, based on the position information, and recognizes a correspondence between the subject and the structures by comparing the image with the map information; Equipped with when a part of information other than the position information in the imaging information is missing, the analysis unit uses the missing information as a parameter, calculates an imaging range for each of the parameters in the map information imaged by the imaging device, recognizes a correspondence between the subject and the structure in the imaging range, and determines that the value of the parameter when the subject and the structure most closely match corresponds to the value of the missing information; An imaging system characterized in that the structure determined not to be recognized in the imaging range based on information other than the missing information is excluded in recognizing the correspondence relationship.

2. 2. The photographing system according to claim 1, further comprising a display unit that displays the image while indicating the correspondence between the subject and the structure.

3. The photographing system according to claim 1 or claim 2, characterized in that the analysis unit recognizes in advance an object to be photographed by the photographing device, acquires the map information of the surroundings of the object, and issues an instruction to the photographing device to control the photographing device so as to photograph the object by comparing the map information with an image based on the video signal obtained from the photographing device.

4. 3. The photographing system according to claim 1, wherein the photographing information includes any one of the altitude of the photographing device, the direction of the center of the field of view at the time of photographing, the tilt angle around the optical axis, the field of view range, and the distance to the photographed object.

5. 3. The photographing system according to claim 1, wherein the analysis unit registers identification information of the structure recognized as the subject.

6. the imaging device; a video server system having the communication unit and the analysis unit; a transmission device that receives the video signal and the imaging information from the imaging device and transmits them to the video server system by wireless communication via a directional antenna; Equipped with the imaging device is fixed to the transmission device; The photographing system according to claim 1 or claim 2, further comprising an adjustment display unit that displays the orientation of the antenna in the image including the subject for which a correspondence relationship with the structure has been identified.

7. A video server system that receives and processes video signals obtained by an imaging device, a communication unit that receives, from the imaging device, imaging information that includes position information of the imaging device at the time of imaging and indicates a state of the imaging device at the time of imaging, together with the video signal; an analysis unit that recognizes a subject in an image obtained by the video signal, recognizes map information about the surroundings of the imaging device, including information about structures surrounding the imaging device, from the position information, and recognizes a correspondence between the subject and the structures by comparing the image with the map information; Equipped with when a part of information other than the position information in the imaging information is missing, the analysis unit uses the missing information as a parameter, calculates an imaging range for each of the parameters in the map information imaged by the imaging device, recognizes a correspondence between the subject and the structure in the imaging range, and determines that the value of the parameter when the subject and the structure most closely match corresponds to the value of the missing information; A video server system characterized in that the structure determined not to be recognized in the imaging range based on information other than the missing information is excluded in recognizing the correspondence.

Images