Information processing device, control method for information processing device, and program

Abstract

To enable more accurate identification of the subject being photographed. [Solution] A video signal is acquired from a camera-equipped tripod head device, and the aircraft and its vertical stabilizer are detected from the acquired video signal. Then, the distance between the aircraft and the tripod head device is calculated based on the actual dimensions of the detected vertical stabilizer, and it is determined whether or not that distance is within the effective range of the shooting distance. If it is determined that the distance between the aircraft and the tripod head device is within the effective range of the shooting distance, a control signal is sent to the tripod head device to track the aircraft.

Description

[Technical Field]

[0001] This invention relates to an information processing device, a control method for the information processing device, and a program. [Background technology]

[0002] Traditionally, shooting systems that allow users to remotely control a camera and pan / tilt mount from a control unit to acquire desired images are widely known. For example, images of aircraft such as passenger planes are captured by remotely controlling a pan / tilt mount permanently installed on the roof of an airport from a broadcasting station. Furthermore, an automatic tracking shooting system has been proposed that incorporates image recognition technology into this shooting system to recognize subjects in the image and automatically pan, tilt, and zoom to track the subject's movement.

[0003] To realize such an automatic tracking and shooting system, it is important to accurately detect the subject being photographed. Therefore, Patent Document 1 discloses a technique for estimating the location of a moving aircraft using multiple cameras and detecting the subject's position. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2018-195965 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] In recent years, examples of using artificial intelligence (hereinafter abbreviated as AI) and machine learning as image recognition technologies have become known, and in particular, systems using AI and machine learning are known to be able to detect objects with high accuracy. Although AI-based object detection can achieve high accuracy, there are cases where it cannot distinguish between multiple objects that have the same characteristics, such as aircraft. The technology described in Patent Document 1 cannot uniquely determine and detect the subject to be tracked when multiple objects are present in the direction of movement.

[0006] In view of the aforementioned problems, the present invention aims to enable more accurate identification of the subject being photographed. [Means for solving the problem]

[0007] The information processing device according to the present invention is characterized by comprising: an acquisition means for acquiring an image signal from an imaging means; a detection means for detecting a subject and a predetermined part of the subject from the image signal acquired by the acquisition means; a calculation means for calculating the distance between the subject detected by the detection means and the imaging means based on the predetermined part detected by the detection means; and a identification means for identifying the subject as a target subject when the distance calculated by the calculation means is within a predetermined range. [Effects of the Invention]

[0008] According to the present invention, the subject being photographed can be identified more accurately. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of the imaging system according to the embodiment. [Figure 2] This block diagram shows an example of the hardware configuration of each device that makes up the imaging system. [Figure 3] This block diagram shows an example of the functional configuration of each device that makes up the imaging system. [Figure 4]It is a diagram showing an example of the dimensions of a derivative aircraft and the shape of a vertical tail. [Figure 5] It is a conceptual diagram showing the structure of input and output using a learning model. [Figure 6] It is a diagram showing an example of a captured image targeting an aircraft during landing. [Figure 7] It is a flowchart showing an example of the overall processing procedure performed by the information processing apparatus. [Figure 8] It is a diagram showing an example of learning data in the learning mode. [Figure 9] It is a flowchart showing an example of a detailed processing procedure in the automatic tracking mode. [Figure 10] It is a diagram for explaining a method of calculating the height position of the vertical tail in an image and the distance between the pan-tilt device and the aircraft.

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments described below, and various forms within the scope not departing from the gist of this invention are also included in the present invention.

[0011] FIG. 1 is a diagram showing a configuration example of a photographing system 1 according to the present embodiment. The photographing system 1 according to the present embodiment is composed of an information processing apparatus 100, a pan-tilt device 200, an operation device 300, and a network 131.

[0012] When a user operates the control device 300, a command corresponding to the operation is transmitted to the camera-equipped pan / tilt head device 200 via the network 131 and the information processing device 100. The pan / tilt head device 200 controls the camera according to the content of the command, thereby enabling remote operation of the pan / tilt head device 200. The video signal captured by the pan / tilt head device 200 is input to the information processing device 100, where various calculations necessary for automatic tracking and recording of the video signal are performed. The network 131 is a communication line such as a public telephone line or the internet. In this embodiment, the pan / tilt head device 200 and the information processing device 100 are installed in locations such as airports, transmission towers, and television station rooftops, and the control device 300 is installed inside a television station, etc. In the following description of this embodiment, the case in which the information processing device 100 and the pan / tilt head device 200 are installed at an airport and the subject to be tracked for automatic tracking is an aircraft will be described as an example.

[0013] Figure 2 is a block diagram showing an example of the hardware configuration of each device that makes up the shooting system 1 in Figure 1. The hardware components of the information processing device 100, the pan / tilt head device 200, and the operating device 300 will be described below.

[0014] First, the hardware configuration of the information processing device 100 will be described. The CPU 103 uses the RAM 101 as work memory to read and execute programs stored in, for example, the storage unit 105, and controls each part of the information processing device 100. In addition, if a large amount of parallel processing of data is required, processing may be performed using the GPU 102. The RAM 101 is a volatile memory and is used as the main memory, work area, and other temporary storage area of ​​the CPU 103. The input unit 104 is an interface for inputting video signals to the information processing device 100, and is a video transmission interface such as SDI or HDMI (registered trademark).

[0015] The memory unit 105 is a non-volatile memory, and image data, other data, and various programs for the operation of the CPU 103 are stored in predetermined areas. The memory unit 105 is composed of a recording medium such as an HDD or flash memory. The serial communication unit 106 is a communication interface for the CPU 103 to control the pan / tilt head device 200. The network communication unit 107 is a communication interface for the CPU 103 to communicate with the operating device 300. The UI unit 108 is a user interface for receiving operation input from a user operating the information processing device 100 and displaying the status of the information processing device 100 to the user.

[0016] Next, the hardware configuration of the pan / tilt head device 200 will be described. The camera 201 is an imaging means that photographs the area around where the pan / tilt head device 200 is installed and captures moving images including the subject to be tracked. The camera 201 is a camera unit consisting of an image sensor and an optical zoom lens with a changeable magnification, and captures moving images by changing the zoom magnification of the lens under the control of the CPU 204, which will be described later. Furthermore, the camera 201 is equipped with a digital zoom function that locally enlarges a part of the captured image. Digital zoom is performed when the desired magnification cannot be achieved with optical zoom alone, that is, when it is desired to enlarge the image to be captured. The camera 201 is also connected to the input unit 104 of the information processing device 100 by a video transmission interface, and the captured video signal is output to the information processing device 100.

[0017] The drive unit 202 consists of an actuator and its drive circuit, peripheral circuits, etc., for rotating the pan and tilt mount 200 in the pan and tilt directions. The drive unit 202 tracks the subject and captures moving images by rotating the pan and tilt mount 200 in the pan and tilt directions. The serial communication unit 203 is connected to the serial communication unit 106 of the information processing device 100 and is a communication interface for the CPU 204 to communicate with the information processing device 100. The CPU 204 is responsible for controlling each part of the pan and tilt mount 200 according to the program stored in the memory unit 205. The memory unit 205 is a non-volatile memory, and setting data for the pan and tilt mount 200, other data, and various programs for the operation of the CPU 204 are stored in predetermined areas.

[0018] Next, the hardware configuration of the operating device 300 will be described. The network communication unit 301 is a communication interface for the CPU 304 to communicate with the information processing device 100. The operation unit 302 consists of a joystick, operating lever, and various switches, and by operating the operation unit 302, the user can control the rotation, zoom, gain, etc., of the pan / tilt head device 200. The memory unit 303 is a non-volatile memory, and setting data for the operating device 300, other data, and various programs for the operation of the CPU 304 are stored in predetermined areas. The CPU 304 is responsible for controlling each part of the operating device 300 according to the programs stored in the memory unit 303. The display unit 305 consists of LEDs and a display touch panel, and notifies the user of the status and warnings of the pan / tilt head device 200. Note that CPU 103, CPU 204, and CPU 304 are each composed of one or more processors.

[0019] Figure 3 is a block diagram showing an example of the functional configuration of each device that makes up the shooting system 1 in Figure 1. First, an example of the functional configuration of the information processing device 100 will be described. The information processing device 100 includes a tracking processing unit 110, a learning unit 150, a data storage unit 151, a mode management unit 152, an image processing unit 153, and a pan / tilt head control unit 157.

[0020] The tracking processing unit 110 consists of a target detection unit 154, a distance calculation unit 155, and a tracking target determination unit 156, and detects and determines the subject to be tracked from the video signal processed by the image processing unit 153, which will be described later. The processing in the target detection unit 154, the distance calculation unit 155, and the tracking target determination unit 156 will be described later.

[0021] The learning unit 150 performs learning processing to enable the target detection unit 154, which will be described later, to detect the target to be tracked. The details of the learning processing will be described later. The data storage unit 151 performs recording processing of the tracked and captured video, recording processing of learning data, recording processing of the detection results of the target to be tracked, and recording processing of the target distance of the target to be tracked, the actual dimensions of characteristic parts of the target to be tracked, which will be described later, and control command values.

[0022] The mode management unit 152 manages the modes of the information processing device 100, and manages three modes: learning mode, automatic tracking mode, and manual mode. The details of each mode will be described later. The image processing unit 153 processes the video signal from the pan / tilt head device 200 based on the information recorded by the data storage unit 151. Specifically, this involves resizing the image and adjusting the brightness.

[0023] The target detection unit 154 inputs the video signal output from the image processing unit 153 as input data to a trained model generated by the learning unit 150, and detects the subject to be tracked and its characteristic parts. The distance calculation unit 155 calculates the distance between the subject to be tracked detected by the target detection unit 154 and the pan / tilt / grip device 200. The distance calculation unit 155 calculates the distance using the dimensions of the characteristic parts of the subject to be tracked detected in the video. In this embodiment, an aircraft is used as the target to be tracked, and in order to calculate the distance between the aircraft and the pan / tilt / grip device 200, dimensional information of the characteristic parts of the aircraft and pan, tilt, and zoom information of the pan / tilt / grip device 200 are used.

[0024] Here, characteristic parts of an aircraft refer to the iconic components of the aircraft, such as the main wings and vertical stabilizers. While the length of the aircraft body varies greatly depending on the aircraft model, characteristic parts are those whose dimensions are constant or have only small differences between different aircraft models or derivative models. In this embodiment, we will explain using the case where the vertical stabilizer is the characteristic part. The vertical stabilizer is a wing section attached vertically to the tail of the aircraft, and it is a characteristic part that has a height that is almost constant regardless of the fuselage length of each aircraft model.

[0025] Figure 4(a) shows an example of the relationship between the fuselage length and the height of the vertical stabilizer of passenger aircraft X. In the example in Figure 4(a), passenger aircraft X is divided into three variants: X-1, X-2, and X-3, with fuselage lengths of 56.7m, 60.2m, and 68.3m respectively, showing a difference of more than 15% between passenger aircraft X-1 and X-3. However, even if the aircraft type can be identified as passenger aircraft X through AI analysis of the image, it is generally difficult to identify the variant. For example, if passenger aircraft X-1 is located in front of passenger aircraft X-3 from the camera's perspective, their fuselage lengths may appear to be the same. Therefore, fuselage length is not a characteristic feature for determining the distance from the pan / tilt mount (the position of the subject).

[0026] On the other hand, as shown in Figure 4(a), the vertical stabilizer has the same height across the three variants. Therefore, the distance between the aircraft and the pan / tilt head 200 can be estimated based on the ratio of the coordinate dimensions of the vertical stabilizer in the image data to the actual dimensions of the vertical stabilizer stored in the data storage unit 151, and the zoom distance. The specific method for calculating the distance between the aircraft being tracked and the pan / tilt head 200 using the vertical stabilizer, a distinctive feature, will be described later. In this way, it is possible to determine the position of the aircraft based on the dimensions of a distinctive feature, regardless of the dimensions of the aircraft itself (such as the fuselage length). Furthermore, as shown in Figure 4(b), since the vertical stabilizer of an aircraft has a distinctive shape for each aircraft type, it is also possible to estimate the aircraft type from the vertical stabilizer shape information, which differs for each aircraft type. It is assumed that the data storage unit 151 also stores vertical stabilizer shape information, which differs for each aircraft type.

[0027] In this embodiment, the vertical stabilizer is used as a distinctive feature, but if there are few differences between aircraft types and variants, and if it is possible to determine the aircraft's position based on the dimensions of the aircraft's components, then one or more other distinctive features may be used.

[0028] The tracking target determination unit 156 compares the distance obtained by the distance calculation unit 155 with the effective range of the tracking target's shooting distance and determines whether the subject detected by the target detection unit 154 is a tracking target. If the distance obtained by the distance calculation unit 155 is within the effective range of the tracking target's shooting distance, the subject is identified as a tracking target. Here, the effective range of the tracking target's shooting distance is predetermined according to the pan and tilt direction of the drive unit 202 of the pan / tilt head device 200 and the zoom distance of the camera 201, and this information is stored in the data storage unit 151. The effective range of the shooting distance may be determined, for example, based on objects such as signs in a virtual space or photographed in advance, and the means are not limited.

[0029] The pan / tilt head control unit 157 calculates a control signal to control the pan / tilt head device 200 according to the mode described above. If the current mode is automatic tracking mode, it calculates a control signal to control the pan / tilt head device 200 so that the subject that has been determined to be the tracking target by the tracking target determination unit 156 is centered in the image, and outputs it to the pan / tilt head device 200. If the current mode is manual mode, it outputs a control signal from the operating device 300 to the pan / tilt head device 200.

[0030] Next, the functional configuration of the pan / tilt unit 200 will be described. The pan / tilt unit 200 includes a pan / tilt control unit 250, a camera control unit 251, a setting management unit 252, and a communication unit 253.

[0031] The pan-tilt control unit 250 outputs signals to the drive unit 202 for driving pan and tilt based on the control signals received by the communication unit 253. The camera control unit 251 outputs signals to the camera 201 for controlling the zoom of the camera 201, etc., based on the control signals received by the communication unit 253. The setting management unit 252 manages the setting items set by the operating device 300. Specific setting items include the maximum speed of pan and tilt and the driveable range. The communication unit 253 exchanges control signals and status information with the pan-tilt control unit 157 of the information processing device 100 in accordance with a predetermined communication protocol.

[0032] The following describes the functional configuration of the operating device 300. The operating device 300 consists of a communication unit 350 and a display unit 351. The communication unit 350 exchanges control signals and status information with the pan / tilt head control unit 157 of the information processing device 100 in accordance with a predetermined communication protocol. The display unit 351 displays information to be presented to the user on the display unit 305.

[0033] Next, the sequence of events in automatic tracking mode in the information processing device 100, the pan / tilt head device 200, and the control device 300 will be described. First, the user operates the control device 300 and sends a signal to the information processing device 100 to switch to automatic tracking mode. Then, the pan / tilt head device 200 takes a picture, and the video signal is sent to the information processing device 100. The information processing device 100 detects the subject to be tracked from the video signal and controls the pan / tilt head device 200 so that the subject to be tracked is in the center of the image. The detailed processing procedure for automatic tracking mode will be described later.

[0034] Next, we will explain the sequence of events in the learning mode of the information processing device 100. Figure 5(a) is a conceptual diagram showing the input / output structure using a learning model for detecting an object. The input data 500 is image data captured by the pan / tilt head device, and corresponds to one frame image in a video captured by the pan / tilt head device. The learning model 501 is composed of a neural network, and its internal parameters are generated by the learning unit.

[0035] Output data 502-504 contains data on the tags, coordinates, and likelihood of objects present in input data 500. The tags to be output are selected from the tags included in the training data input during training. Two coordinate points are output; as shown in the image region 505, the coordinate information of the top-left and bottom-right points of the circumscribing frame of the estimated object is output. The likelihood is a value between 0 and 1, and a higher value indicates a higher confidence in the detection result for the output tag. By setting a threshold for the likelihood, it is possible to obtain output only for confidence levels above a certain level. Output data 502 represents data on aircraft in flight, output data 503 represents data on parked aircraft, and output data 504 represents data on aircraft landing. The coordinates in output data 502-504 are coordinates within the captured image data.

[0036] On the other hand, in the example shown in Figure 5(a), multiple output data 502-504 are output. In this case, it may not be possible to identify and track only one target aircraft, and the target aircraft may switch to another aircraft detected with a similar likelihood. This situation will be explained in more detail with reference to Figure 6.

[0037] Figure 6 shows an example of a captured image where aircraft 601 is being tracked during landing. In Figure 6, aircraft 601 is the subject being tracked as it lands and moves down runway 600, and its output data likelihood 602 is 0.8. In this image, in addition to aircraft 601 during landing, aircraft 604 parked at apron 603 and aircraft 607 moving along taxiway 606 are also included, which are not being tracked. The likelihood 605 for aircraft 604 is 0.8, and the likelihood 608 for aircraft 607 is also 0.8. The solid line frame 609 indicates that the subject is being tracked, while the dashed lines 610 and 611 indicate that the subject is not being tracked. As in the example in Figure 6, when likelihoods 602, 605, and 608 are all the same, it is difficult to determine the subject being tracked, and the subject may switch unexpectedly.

[0038] In this embodiment, the distance calculation unit 155 calculates the dimensions of characteristic parts of the aircraft by estimating the coordinates that determine the dimensions of those parts within the video, and then calculates the distance to the pan / tilt head device 200 by comparing it with the actual dimensions of those parts. The tracking target determination unit 156 then determines whether the calculated distance is within the effective range of a predetermined shooting distance, thereby identifying the aircraft to be tracked. In this way, the position of the aircraft is calculated from the dimensions of characteristic parts, and the tracking target is determined based on the condition that the aircraft is located within the effective range of the shooting distance.

[0039] Figure 5(b) is a conceptual diagram showing the input / output structure using a learning model for detecting an object in this embodiment. As shown in Figure 5(b), in this embodiment, the learning model is trained to detect not only the aircraft but also the vertical stabilizer, which is a characteristic part. Therefore, by inputting the input data 500 into the learning model 501, not only output data 502 to 504 but also output data 506 to 508 are obtained. Output data 506 to 508 are data related to the tag, coordinate information, and likelihood of the vertical stabilizer within the object present in the input data 500. In this embodiment, the object detection unit 154 acquires data related to the tag, coordinate information, and likelihood of the vertical stabilizer, as well as the aircraft body, as information about the object.

[0040] Next, the detailed processing flow in the information processing device 100 will be described. Figure 7 is a flowchart showing an example of the overall processing procedure performed in the information processing device 100 according to this embodiment. First, in step S700, the mode management unit 152 determines whether the current operating mode is learning mode. If the mode management unit 152 determines that it is learning mode, the process proceeds to step S701; otherwise, the process proceeds to step S706. In step S701, the learning unit 150 accepts input of learning data. Here, the learning data will be explained with reference to Figure 8.

[0041] Figure 8 shows an example of training data in learning mode. The training data is composed of images (input data) and tags of objects contained in the images (training data), and these data are combined into one training data set. Preferably, the input image contains only one object, and the image size is preferably the same across multiple training data sets. In this embodiment, an image of an aircraft 801 is prepared as input data for automatic tracking and photography of an aircraft at an airport. In this embodiment, an image 802 of a part of the aircraft is also prepared and trained as a separate object.

[0042] Next, in step S702, the learning unit 150 determines whether the training data received in step S702 is legitimate data. As mentioned above, training data is either an image of an aircraft linked as input data with the tag "aircraft" as training data, or an image of a vertical stabilizer linked as input data with the tag "vertical stabilizer" as training data. Therefore, if the training data falls into either of these categories, it is determined to be legitimate data. If the learning unit 150 determines that the data is legitimate, it proceeds to step S703; otherwise, it returns to step S701.

[0043] In step S703, the learning unit 150 inputs the training data received in step S701 into the learning model. Then, in step S704, the learning unit 150 trains the learning model. The training method of the learning model is described below.

[0044] The learning unit 150 comprises an error detection unit and an update unit. The error detection unit obtains the error between the output data output from the output layer of the neural network and the training data, in accordance with the input data input to the input layer. The update unit updates the connection weight coefficients between nodes of the neural network, etc., based on the error obtained by the error detection unit, so as to reduce the error. The learning model is trained through the above procedure.

[0045] Next, in step S705, the learning unit 150 determines whether or not the learning of the learning model has been completed. If the learning unit 150 determines that the learning of the learning model has been completed, it terminates the process; otherwise, it returns to step S701. In learning mode, the internal parameters of the learning model are determined by the above processes, and a trained model is generated.

[0046] On the other hand, in step S706, the mode management unit 152 determines whether the current mode is manual mode or not. If the mode management unit 152 determines that it is manual mode, the process proceeds to step S707; otherwise, the process proceeds to step S709.

[0047] In step S707, the pan / tilt head control unit 157 receives a control signal from the operating device 300. Then, in step S708, the pan / tilt head control unit 157 transmits the control signal received in step S707 to the pan / tilt head device 200 and terminates processing. Meanwhile, in step S709, the processing for the automatic tracking mode, which will be described later, is executed.

[0048] The following describes the detailed processing procedure of this embodiment in automatic tracking mode. Figure 9 is a flowchart showing an example of the detailed processing procedure in automatic tracking mode in step S709. In this embodiment, an example is described in which an aircraft moving on a runway is used as the subject to be tracked, and the aircraft's vertical stabilizer is used as a characteristic part of the aircraft. In addition, in automatic tracking mode, a real-time video signal is transmitted from the pan / tilt head device 200 to the information processing device 100.

[0049] First, in step S900, the pan / tilt control unit 157 transmits a control signal to the pan / tilt device 200 to move it to the home position. Here, the home position is the pre-set pan and tilt position of the pan / tilt device 200 and the zoom position of the camera 201. Then, in step S901, the target detection unit 154 determines whether the image processing unit 153 has detected an aircraft and its vertical stabilizer in the video signal received from the pan / tilt device 200. If the target detection unit 154 determines that it has detected an aircraft and its vertical stabilizer, the process proceeds to step S902; otherwise, the process returns to step S900.

[0050] In step 902, the distance calculation unit 155 estimates the height position of the vertical stabilizer in the image using the coordinates of the vertical stabilizer detected by the target detection unit 154. Then, in step S903, the distance calculation unit 155 calculates the distance between the pan / tilt head device 200 and the aircraft. Figures 10(a) and 10(b) are diagrams illustrating the processing in steps S902 and S903 in this embodiment. The method for calculating the height position of the vertical stabilizer in the image and the distance between the pan / tilt head device 200 and the aircraft will be described in detail below with reference to Figure 10.

[0051] First, the object detection unit 154 detects the coordinates (x1, y1) of the upper left point and the coordinates (x2, y2) of the lower right point of the circumscribed frame of the aircraft 1001 moving along the runway 1000. Furthermore, the object detection unit 154 detects the coordinates (x3, y3) of the upper left point and the coordinates (x4, y4) of the lower right point of the circumscribed frame of the vertical stabilizer 1002 of the aircraft 1001.

[0052] In step S902, the positions of the vertex and base points are estimated as height positions to determine the height of the vertical stabilizer 1002 in the image data. In the example shown in Figure 10(a), the tail vertex coordinates 1007 and root coordinates 1008 are estimated based on the coordinates (x3, y3) of the upper left point and (x4, y4) of the lower right point of the circumscribed frame of the vertical stabilizer 1002 of aircraft 1001.

[0053] Next, in step S903, first, from the fin top coordinates 1007 and the base coordinates 1008 estimated in step S902, the height la of the vertical fin in the image is calculated. The number of pixels l between the Y coordinates of the fin top coordinates 1007 and the base coordinates 1008 respectively a corresponds to the height of the vertical fin in the image. Also, in the data storage unit 151, the actual height L a (in m units) of the vertical fin is stored in advance. This number of pixels l a is set equal to the height L a of the actual vertical fin 1002. Next, from the number of pixels l b in the height direction in the image, the number of pixels l a of the vertical fin, and the actual height L a of the vertical fin, the actual height L b (in m units) within the range of the image based on the vertical fin is calculated from the following formula (1). L b = L a × l b / l a ···(1)

[0054] Subsequently, the distance L b (in m units) between the camera 201 and the center point of the actual height L c within the range of the image is calculated. FIG. 10(b) is a view of the aircraft 1001 and the pan-tilt device 200 in FIG. 10(a) as seen from the back side of the aircraft 1001. The viewing angle θ m when the camera 201 of the pan-tilt device 200 takes an image is formed by taking the camera 201 as the vertex and the upper and lower points 1012 and 1013 of the actual height L b within the range of the image based on the vertical fin, with the side of the height L b as the base. At this time, the straight line connecting the center point 1011 of the actual height L b within the range of the image based on the vertical fin and the camera 201 bisects the viewing angle θ m . The actual distance L c between the camera 201 and the center point 1011 can be calculated by the following formula (2). L c =(L b / 2) / tan(θ m / 2) ···(2)

[0055] Finally, the distance L calculated using equation (2) c The distance between camera 201 and aircraft 1001 is calculated using the following: the actual position corresponding to the root coordinates 1008 of the vertical stabilizer, and the height L. b The center point 1011 and the camera 201 form a triangle with the center point 1011 as a right angle. Here, the actual distance between the center point 1011 and the actual position corresponding to the root coordinate 1008 of the vertical stabilizer is L. d (in meters). Therefore, the actual distance L between camera 201 and the actual position corresponding to the root coordinate 1008 of the vertical stabilizer. e The unit (in meters) can be calculated using the following formula (3). L e =L c / cos(arctan(L d / L c )) ···(3)

[0056] As described above, the distance between the base of the vertical stabilizer 1002 and the camera 201 is calculated using equation (3). Since the base of the vertical stabilizer 1002 is located close to the main body of the aircraft 1001, in this embodiment, the distance L e This distance L is calculated as the distance between aircraft 1001 and the pan / tilt mount 200. e Using this, proceed to step S904.

[0057] In step S904, the tracking target determination unit 156 determines whether the target aircraft is the aircraft to be tracked, based on the distance between the pan / tilt head device 200 and the aircraft calculated in step S903. This determination process determines whether the aircraft is the target of tracking by determining whether the distance between the pan / tilt head device 200 and the aircraft is within the effective range of the shooting distance.

[0058] Here, the effective range of the shooting distance is defined by the distance between the tripod head device 200 and the target subject, and the pan angle of the tripod head device 200. In this embodiment, the effective range of the shooting distance is defined as the range of distances corresponding to the width of the runway 1000 for each pan angle of the tripod head device 200 that has been measured in advance, and it is assumed that information on the effective range of the shooting distance is stored in advance in the data storage unit 151. If the tracking target determination unit 156 determines that the aircraft is the target of tracking, the process proceeds to step S905; otherwise, the process returns to step S900. If multiple aircraft are detected simultaneously in step S901, the distance between the tripod head device 200 and each aircraft is calculated. Then, in step S904, it is determined whether there is an aircraft among them whose distance between the tripod head device 200 and the aircraft is within the effective range of the shooting distance. If the tracking target determination unit 156 determines that there is an aircraft within the effective range of the shooting distance, the process proceeds to S905.

[0059] In step S905, the pan / tilt / grip control unit 157 transmits a control signal to the pan / tilt / grip device 200 to perform automatic tracking photography using pan, tilt, and zoom for the aircraft that was determined to be the target of tracking in step S904.

[0060] In step S906, the pan / tilt head control unit 157 determines whether or not automatic tracking shooting has been performed for a certain period of time. Here, the period of time can be arbitrarily determined. If the pan / tilt head control unit 157 determines that automatic tracking shooting has been performed for a certain period of time, the process returns to step S901; otherwise, the process proceeds to step S907 in order to continue automatic tracking shooting.

[0061] In step S907, the target detection unit 154 determines whether or not other aircraft other than the target being photographed have been detected within the shooting area. If the target detection unit 154 determines that other aircraft have been detected, the process proceeds to step S908; otherwise, the process proceeds to step S911.

[0062] In step S908, the distance calculation unit 155 estimates the height position of the vertical stabilizers of multiple detected aircraft, including the target being tracked. In step S908, the same processing as in step S902 is performed. Next, in step S909, the distance calculation unit 155 calculates the distance between the aircraft and the pan / tilt mount 200 based on the height position of the vertical stabilizers. In step S909, the same processing as in step S903 is performed.

[0063] In step S910, the tracking target determination unit 156 determines whether or not there is an aircraft to be tracked, similar to the process in S904. If the tracking target determination unit 156 determines that there is an aircraft to be tracked, the process proceeds to step S905; otherwise, the process returns to step S900.

[0064] In step S911, the tracking target determination unit 156 determines whether or not to continue tracking the target aircraft. In this process, for example, it determines whether to end tracking if a preset tracking duration has elapsed or if the target aircraft moves out of the effective range of the shooting distance. If the tracking target determination unit 156 determines to continue tracking, the process returns to step S905; otherwise, the process ends.

[0065] As described above, according to this embodiment, the distance between the aircraft and the pan / tilt head device is calculated using dimensional information of characteristic parts of the detected aircraft, and it is determined whether or not it is the aircraft to be tracked. This makes it possible to more accurately identify the subject to be tracked, and to track the aircraft without the subject transferring to an aircraft that is not the target of tracking.

[0066] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0067] This embodiment includes the following configurations, methods, and programs.

[0068] (Composition 1) An acquisition means for acquiring a video signal from an imaging means, A detection means for detecting a subject and a predetermined part of the subject from the video signal acquired by the acquisition means, A calculation means for calculating the distance between the subject detected by the detection means and the imaging means based on a predetermined part detected by the detection means, When the distance calculated by the calculation means falls within a predetermined range, the identification means identifies the subject as the target subject, An information processing device characterized by having the following features.

[0069] (Configuration 2) The information processing apparatus according to configuration 1, characterized in that the predetermined range is a range based on the distance between the subject and the imaging means and the pan angle of the imaging means. (Composition 3) The information processing device according to configuration 1 or 2, further comprising a storage means for storing information on the actual dimensions of the predetermined part. (Composition 4) The information processing device according to configuration 3, characterized in that the calculation means calculates the distance between the subject and the imaging means by using information on the actual dimensions of the predetermined part to calculate the distance between the imaging means and the predetermined part.

[0070] (Composition 5) The detection means acquires coordinate information in the video signal of the predetermined part, The information processing apparatus according to configuration 3 or 4, characterized in that the calculation means calculates the distance between the subject and the imaging means by calculating the distance between the imaging means and the predetermined part based on the actual size of the predetermined part and the coordinate information of the predetermined part in the video signal. (Composition 6) The information processing device according to any one of configurations 3 to 5, characterized in that the storage means further stores shape information of the predetermined part. (Composition 7) The information processing apparatus according to any one of configurations 3 to 6, characterized in that the storage means further stores information within the predetermined range. (Composition 8) An information processing device according to any one of configurations 1 to 7, further comprising control means for controlling the imaging means to track a subject identified by the aforementioned identification means.

[0071] (method) The acquisition process involves acquiring a video signal from an imaging means, A detection step in which a subject and a predetermined part of the subject are detected from the video signal acquired in the acquisition step, A calculation step which calculates the distance between the subject detected in the detection step and the imaging means based on the predetermined part detected in the detection step, If the distance calculated in the calculation step falls within a predetermined range, the identification step involves identifying the subject as the target subject, A control method for an information processing device, characterized by having the following features.

[0072] (program) A program for causing a computer to function as one of the means of the information processing device described in any of configurations 1 to 8. [Explanation of Symbols]

[0073] 110: Tracking processing unit, 154: Target detection unit, 155: Distance calculation unit, 156: Tracking target determination unit

Claims

1. An acquisition means for acquiring a video signal from an imaging means, A detection means for detecting a subject and a predetermined part of the subject from the video signal acquired by the acquisition means, A calculation means for calculating the distance between the subject detected by the detection means and the imaging means based on a predetermined part detected by the detection means, When the distance calculated by the calculation means falls within a predetermined range, the identification means identifies the subject as the target subject, An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, characterized in that the predetermined range is a range based on the distance between the subject and the imaging means and the pan angle of the imaging means.

3. The information processing apparatus according to claim 1, further comprising a storage means for storing information on the actual dimensions of the predetermined part.

4. The information processing device according to claim 3, characterized in that the calculation means calculates the distance between the subject and the imaging means by using information on the actual size of the predetermined part to calculate the distance between the imaging means and the predetermined part.

5. The detection means acquires coordinate information in the video signal of the predetermined part, The information processing apparatus according to claim 3, characterized in that the calculation means calculates the distance between the subject and the imaging means by calculating the distance between the imaging means and the predetermined part based on the actual size of the predetermined part and the coordinate information of the predetermined part in the video signal.

6. The information processing apparatus according to claim 3, characterized in that the storage means further stores shape information of the predetermined part.

7. The information processing apparatus according to claim 3, characterized in that the storage means further stores information within the predetermined range.

8. The information processing apparatus according to claim 1, further comprising control means for controlling the imaging means to track a subject identified by the identification means.

9. The acquisition process involves acquiring a video signal from an imaging means, A detection step in which a subject and a predetermined part of the subject are detected from the video signal acquired in the acquisition step, A calculation step which calculates the distance between the subject detected in the detection step and the imaging means based on the predetermined part detected in the detection step, If the distance calculated in the calculation step falls within a predetermined range, the identification step involves identifying the subject as the target subject, A control method for an information processing device, characterized by having the following features.

10. A program for causing a computer to function as each means of the information processing apparatus described in claim 1.

Images