Information processing device, control method, and program

Abstract

To provide a technology capable of more freely arranging a sensor used for detecting a moving object.SOLUTION: An image processing device identifies a direction (first direction 12) from a sensor 10 to a moving body 20 by detecting the moving body 20 using the sensor 10. The information processing device causes a camera 30 to take an image while moving an optical axis direction 34 of the camera 30 along the first direction 12. As a result, a plurality of captured images 32 in which capturing is performed in different directions are generated by the camera 30. The information processing device 2000 detects the moving body 20 by performing image analysis on the captured image 32.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a control method, and a program. [Background technology]

[0002] Cameras are used to monitor moving objects. For example, Patent Document 1 discloses a surveillance camera device that monitors moving objects using a wide-angle camera and a telephoto camera placed side by side. This surveillance camera device detects moving objects using an image generated by the wide-angle camera, and changes the direction of the imaging optical axis of the telephoto camera based on the position of the moving object in the image. This allows the moving object to be captured in the center of the telephoto camera's imaging range. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-116666 Summary of the Invention [Problem to be solved by the invention]

[0004] The surveillance camera device of Patent Document 1 is based on the premise that the telephoto camera is placed near the wide-angle camera, i.e., that the telephoto camera and the wide-angle camera can capture images in approximately the same direction. Based on this premise, the telephoto camera can capture images of a moving object by pointing its imaging optical axis in approximately the same direction as the direction from the wide-angle camera toward the moving object. Therefore, the technology of Patent Document 1 does not allow the telephoto camera and the wide-angle camera to be located far apart from each other.

[0005] The present invention has been made in view of this problem, and one of its objects is to provide a technique that allows sensors used to detect moving bodies to be placed in more flexible positions. [Means for solving the problem]

[0006] The information processing device of the present invention has: 1) a first detection means that detects a moving object using a sensor and identifies a first direction from the sensor toward the moving object; 2) a control means that causes the first camera to take an image while moving the optical axis direction of the first camera along the first direction; and 3) a second detection means that detects the moving object from a first captured image generated by the first camera.

[0007] A control method of the present invention is executed by a computer and includes: 1) a first detection step of detecting a moving object using a sensor and identifying a first direction from the sensor toward the moving object, 2) a control step of causing the first camera to capture an image while moving the optical axis direction of the first camera along the first direction, and 3) a second detection step of detecting the moving object from the first captured image generated by the first camera.

[0008] The program of the present invention causes a computer to execute each step of the control method of the present invention. [Effects of the Invention]

[0009] According to the present invention, a technique is provided that allows sensors used to detect moving objects to be placed at more flexible positions. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining an overview of an information processing apparatus according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the functional configuration of the information processing apparatus according to the first embodiment. [Figure 3] FIG. 1 is a diagram illustrating a computer for realizing an information processing device. [Figure 4] 1 is a flowchart illustrating the flow of processing executed by the information processing apparatus of the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating a case in which a second camera is used as a sensor. [Figure 6] 10A and 10B are diagrams illustrating a case in which a detection area is designated by a user; [Figure 7] FIG. 1 is a diagram illustrating a detection space. [Figure 8] FIG. 10 is a diagram illustrating an example of updating a first direction. [Figure 9] FIG. 2 is a plan view of an area in which moving objects are monitored, viewed in the vertical direction. [Figure 10] FIG. 1 is a side view of an area where mobile objects are monitored. [Figure 11] FIG. 10 is a diagram illustrating an example of the functional configuration of an information processing apparatus according to a second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of the functional configuration of an information processing apparatus according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all drawings, like components are designated by like reference numerals, and their description will be omitted where appropriate. Furthermore, unless otherwise specified, in each block diagram, each block represents a functional configuration, not a hardware configuration.

[0012] [Embodiment 1] <Summary> 1 is a diagram for explaining an overview of the information processing device of embodiment 1 (information processing device 2000 shown in FIG. 2). The operation of the information processing device 2000 described below is an example to facilitate understanding of the information processing device 2000, and the operation of the information processing device 2000 is not limited to the following example. Details and variations of the operation of the information processing device 2000 will be described later.

[0013] The information processing device 2000 uses the sensor 10 to detect the moving object 20, thereby identifying the direction from the sensor 10 toward the moving object 20 (first direction 12 in FIG. 1 ). The information processing device 2000 causes the camera 30 to capture an image while moving the optical axis direction 34 of the camera 30 along the first direction 12 (i.e., from the sensor 10 toward the moving object 20). As a result, a plurality of captured images 32, each captured in a different direction, are generated by the camera 30. The information processing device 2000 detects the moving object 20 by performing image analysis on the captured images 32.

[0014] According to this method, the camera 30 can detect the moving object 20 even if the sensor 10 and the camera 30 are not installed in close proximity to each other. This is because the moving object 20 exists in the first direction 12, and therefore by repeatedly capturing images by the camera 30 while moving the optical axis direction 34 of the camera 30 along the first direction 12, the moving object 20 can be included in one of the captured images 32. Therefore, the information processing device 2000 of this embodiment has the advantage that the sensor 10 and the moving object 20 do not need to be installed in close proximity to each other, and can be positioned more freely.

[0015] The information processing device 2000 of this embodiment will be described in further detail below.

[0016] <Example of functional configuration of information processing device 2000> 2 is a diagram illustrating an example of the functional configuration of an information processing device 2000 according to the first embodiment. The information processing device 2000 includes a first detection unit 2020, a control unit 2040, and a second detection unit 2060. The first detection unit 2020 detects a moving object 20 using a sensor 10 and identifies a first direction 12 from the sensor 10 toward the moving object 20. The control unit 2040 causes the camera 30 to capture an image while moving the optical axis direction 34 of the camera 30 along the first direction 12. The second detection unit 2060 detects the moving object 20 from a captured image 32 generated by the camera 30.

[0017] <Hardware configuration of information processing device 2000> Each functional component of the information processing device 2000 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). The following further describes the case where each functional component of the information processing device 2000 is realized by a combination of hardware and software.

[0018] 3 is a diagram illustrating a computer 1000 for realizing the information processing device 2000. The computer 1000 is any computer. For example, the computer 1000 is a personal computer (PC), a server machine, a tablet terminal, a smartphone, or the like. The computer 1000 may be a dedicated computer designed to realize the information processing device 2000, or may be a general-purpose computer.

[0019] The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 transmit and receive data to and from each other. However, the method of interconnecting the processor 1040 and other components is not limited to bus connection. The processor 1040 may be any of various processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device implemented using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device implemented using a hard disk, a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like.

[0020] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0021] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1120 may be connected to the network via a wireless connection or a wired connection.

[0022] For example, a sensor 10 and a camera 30 are connected to the network interface 1120 via a network. The sensor 10 is any sensor capable of detecting a moving object 20. As will be described in detail later, the sensor 10 may be, for example, a camera, an acoustic sensor, or a radio wave sensor. The camera 30 is any imaging device that captures images and generates captured images. For example, the camera 30 is a video camera that periodically captures images and generates video data. In this case, the captured images 32 are the frames that make up the video data.

[0023] The storage device 1080 stores program modules that implement each functional component of the information processing device 2000. The processor 1040 reads each of these program modules into the memory 1060 and executes them to implement the function corresponding to each program module.

[0024] <Processing flow> 4 is a flowchart illustrating the flow of processing executed by the information processing device 2000 of embodiment 1. The first detection unit 2020 detects the moving object 20 using the sensor 10 (S102). The first detection unit 2020 identifies a first direction 12 from the sensor 10 toward the moving object 20 (S104).

[0025] Steps S106 to S116 are loop processes that are repeatedly performed until a predetermined condition is met. If the predetermined condition is met, the process in FIG. 4 ends. On the other hand, if the predetermined condition is not met, the process in FIG. 4 proceeds to step S108.

[0026] The control unit 2040 moves the optical axis direction 34 of the camera 30 along the first direction 12 (S108). The control unit 2040 causes the camera 30 to capture an image (S110). The second detection unit 2060 performs an object detection process on the captured image 32 generated by the camera 30 (S112). If an object is detected from the captured image 32 (S114: YES), the second detection unit 2060 detects the detected object as a moving object 20 (S118). Then, the process of FIG. 4 ends. On the other hand, if an object is not detected from the captured image 32 (S114: NO), the process of FIG. 4 proceeds to S106.

[0027] There are various predetermined conditions for ending loop process A. For example, the predetermined condition is that camera 30 has finished capturing images of the entire imageable range along first direction 12. More specifically, the predetermined condition is that the imaging direction of camera 30 has become parallel to first direction 12.

[0028] The flow of the process executed by the information processing device 2000 is not limited to the flow shown in Fig. 4. For example, the control unit 2040 may cause the camera 30 to capture images multiple times, and then perform object detection processing on each of the generated captured images. That is, the object detection processing on each of the multiple captured images generated by the camera 30 may be executed collectively after the loop processing A.

[0029] Furthermore, the control unit 2040 may treat the object detected in S112 as the moving body 20 unconditionally, but may treat the object as the moving body 20 if the object satisfies a predetermined condition. This point will be described later.

[0030] <Detection of moving object 20 using sensor 10> The first detection unit 2020 detects the moving object 20 using the sensor 10. The method for detecting the moving object 20 using the sensor 10 depends on the type of sensor used. Below, examples of detection methods for the moving object 20 will be described for each main type of sensor that can be used as the sensor 10.

[0031] <<Cases where a camera is used>> For example, a camera can be used as the sensor 10. Here, the camera used as the sensor 10 is called the second camera. FIG. 5 is a diagram illustrating a case where the second camera is used as the sensor 10. In FIG. 5, the second camera is represented by the reference numeral 40. Hereinafter, the captured image generated by the second camera 40 will be referred to as the second captured image 42.

[0032] The second camera 40 captures an image with a wider angle of view than the camera 30. In other words, the second camera 40 captures an image with a shorter focal length than the camera 30. Therefore, for example, a telephoto camera is used as the camera 30, and a wide-angle camera is used as the second camera 40. By using two cameras in this way, it becomes possible to first capture an image of a wide range with the wide-angle camera (camera 30) to quickly determine the presence and direction of the moving body 20, and then capture a large image of the moving body 20 with the telephoto camera (second camera 40) to grasp the details of the moving body 20.

[0033] The first detection unit 2020 detects the moving object 20 from the second captured image 42 generated by the second camera 40. Here, the first detection unit 2020 may detect any object included in the second captured image 42 as the moving object 20 (see FIG. 3), or may detect an object included in the second captured image 42 that satisfies a predetermined condition as the moving object 20. In the former case, the first detection unit 2020 extracts, for example, a foreground region from the second captured image 42, and determines the object represented by the foreground region as the moving object 20. Here, existing technology can be used as the technology for extracting the foreground region from an image.

[0034] On the other hand, suppose an object that satisfies a predetermined condition is to be detected as the moving body 20. In this case, for example, feature amounts of the object to be detected as the moving body 20 are determined in advance. The first detection unit 2020 detects an image area having these feature amounts from the second captured image 42, and determines the object represented by the detected image area as the moving body 20. The object to be detected as the moving body 20 is, for example, a flying object such as an airplane, a drone, or a bird. Note that existing technologies can be used for the technology to determine the feature amounts of the object to be detected and the technology to detect an object having these feature amounts from an image.

[0035] Alternatively, for example, the first detection unit 2020 may detect only moving objects from among the objects detected from the second captured images 42 as the moving body 20. For example, when the control unit 2040 detects an object from a certain second captured image 42, it tracks the object in each second captured image 42 generated thereafter, thereby determining whether the object is moving. If it is determined that the object is moving, the control unit 2040 treats the object as the moving body 20. On the other hand, if it is determined that the object is not moving, the control unit 2040 does not treat the object as the moving body 20. Note that existing technology can be used as a technology for determining whether an object detected from an image is moving.

[0036] The first detection unit 2020 may detect the moving object 20 over the entire image area of ​​the second captured image 42, or may detect the moving object 20 over only a part of the image area of ​​the second captured image 42. In the latter case, the image area that is the detection target for the moving object 20 is called a detection area.

[0037] For example, the information processing device 2000 accepts an input operation from a user to specify a detection area. FIG. 6 is a diagram illustrating an example of accepting a detection area specification from a user. In FIG. 6, a video 52 consisting of a second captured image 42 captured by the second camera 40 is displayed on the touch panel 50. The user watches the video 52 to find an object that the information processing device 2000 wants to detect as a moving object 20. Once the user finds the object that the user wants to detect as a moving object 20, the user touches the position on the touch panel 50 where the object is displayed. The first detection unit 2020 identifies an image position in the second captured image 42 that corresponds to the touched position and performs object detection for an image area of ​​the second captured image 42 based on the image position. The first detection unit 2020 then treats the detected object as a moving object 20. For example, in FIG. 6, a rectangular area 56 centered on the image position touched by the user's finger 54 is treated as the detection area. As a result, a drone 58 is detected as a moving object 20.

[0038] There are various ways in which the user can specify the detection area. For example, the user specifies a position on the second captured image 42 as described above. The first detection unit 2020 treats an image area of ​​a predetermined size and shape determined based on the specified position as the detection area. Note that existing technology can be used to determine an image area of ​​a predetermined size and shape based on the specified position.

[0039] The user may specify an area instead of a position on the second captured image 42. In this case, the area specified by the user is treated as the detection area. The area can be specified by, for example, a drag operation or a slide operation.

[0040] <<Cases where acoustic sensors are used>> For example, an acoustic sensor can be used as the sensor 10. This acoustic sensor uses a microphone array in which multiple microphones are arranged three-dimensionally. Here, the term "three-dimensionally arranged multiple microphones" means that there is no single plane that passes through all of the microphones that make up the microphone array.

[0041] When the same sound is detected by multiple microphones arranged three-dimensionally, the direction of the sound source can be identified from the microphone array. Therefore, when a sound with predetermined characteristics is detected by the multiple microphones that make up the acoustic sensor, the first detection unit 2020 detects the source of the sound as the moving object 20.

[0042] The "predetermined feature" may be any of a variety of features. For example, the predetermined feature may be expressed as the magnitude of sound pressure. In this case, the first detection unit 2020 determines whether the sound pressure of the sound detected by the acoustic sensor is equal to or greater than a predetermined value, and if it is equal to or greater than the predetermined value, treats the source of the sound as a moving object 20. In other words, a sound source that emits a sound louder than a preset volume is treated as a moving object 20.

[0043] Alternatively, for example, the predetermined feature may be represented by the frequency spectrum of the sound emitted by the moving body 20 to be detected. In this case, the first detection unit 2020 analyzes the frequency spectrum of the sound detected by the acoustic sensor, and if the sound has a frequency spectrum that matches or is similar to the predetermined feature, treats the source of the sound as the moving body 20. When detecting an airplane as the moving body 20, the predetermined feature is, for example, the frequency spectrum of the engine sound of the airplane. When detecting a drone as the moving body 20, the predetermined feature is, for example, the frequency spectrum of the sound of the drone's rotors rotating (hereinafter, rotor sound).

[0044] The frequency spectrum of an airplane engine sound differs depending on the type of airplane, etc. The predetermined feature may be defined as a fairly wide frequency spectrum range that includes the frequency spectrum of various airplane engine sounds, or may be expressed as the frequency spectrum of an airplane engine sound of a certain specific model, etc. In the latter case, only airplanes of that specific model will be detected as moving objects 20.

[0045] Similarly, the frequency spectrum of the rotor sound of a drone also differs depending on the model of the drone, etc. Therefore, the predetermined feature may be defined as a fairly wide frequency spectrum range so as to include the frequency spectrum of the rotor sound of various drones, or may be expressed as the frequency spectrum of the rotor sound of a drone of a certain specific model, etc. In the latter case, only drones of that specific model will be detected as moving objects 20.

[0046] Here, the characteristics of the sound emitted by a moving object to be detected as the moving object 20, i.e., information indicating the above-mentioned predetermined characteristics, is referred to as sound characteristic information. The sound characteristic information is stored in a storage device accessible by the first detection unit 2020. The first detection unit 2020 acquires the sound characteristic information and determines whether or not a sound having the predetermined characteristics indicated in the sound characteristic information has been detected by the acoustic sensor. Then, when the first detection unit 2020 detects a sound having the predetermined characteristics by the acoustic sensor, it identifies the direction from the sensor 10 toward the source of the sound as the first direction 12.

[0047] <<Cases where radio wave sensors are used>> For example, a radio wave sensor can be used as the sensor 10. This radio wave sensor uses a receiver array in which multiple radio wave receivers are arranged three-dimensionally. Here, the term "three-dimensionally arranged multiple radio wave receivers" means that there is no single plane that passes through all of the receivers that make up the receiver array. When radio waves having predetermined characteristics are detected by the multiple receivers that make up the radio wave sensor, the first detection unit 2020 treats the source of the radio waves as a moving object 20.

[0048] The predetermined characteristic may be a variety of characteristics. For example, the predetermined characteristic may be expressed by the amplitude of the radio wave. In this case, the first detection unit 2020 determines whether the amplitude of the radio wave detected by the radio wave sensor is equal to or greater than a predetermined value, and if it is equal to or greater than the predetermined value, treats the source of the radio wave as a moving body 20. In other words, a source that emits radio waves with an amplitude greater than a preset value is treated as a moving body 20.

[0049] Alternatively, for example, the predetermined characteristic may be represented by the frequency spectrum of radio waves emitted by the moving body 20 to be detected. In this case, the first detection unit 2020 analyzes the frequency spectrum of radio waves detected by the radio wave sensor, and if the radio waves have a frequency spectrum that matches or is similar to the predetermined characteristic, it treats the source of the sound as the moving body 20. In this case, the frequency spectrum of the radio waves output by the moving body 20 to be detected is known in advance, and information representing the frequency spectrum is stored in a storage device.

[0050] In another example, the predetermined characteristic may be represented by data carried in radio waves emitted by the mobile object 20 to be detected. When the mobile object 20 transmits data via radio waves, the receiver can demodulate the radio waves and extract the data, thereby ascertaining the content of the data. For example, suppose the mobile object 20 wirelessly transmits an Ethernet® frame. In this case, the source MAC (Media Access Control) address of this Ethernet® frame indicates the MAC address of the network interface possessed by the mobile object 20. Therefore, the first detection unit 2020 can obtain information about the source of the radio waves by demodulating the radio waves received by the radio wave sensor to extract the Ethernet® frame and examining the source MAC address of the Ethernet® frame.

[0051] Therefore, for example, the first detection unit 2020 identifies the source of data obtained from radio waves received by the radio wave sensor, and if the source satisfies a predetermined condition, detects the source as a moving body 20. For example, the predetermined condition is that "the source is an object having a specific source address." In this case, the first detection unit 2020 detects only objects having the specific address as moving bodies 20. In this way, it becomes possible to detect only objects having the specific address as moving bodies 20.

[0052] Another example of a predetermined condition is that "the transmitter belongs to a predetermined group." For example, a MAC address generally includes a vendor code. Therefore, this vendor code can be used to identify the manufacturer of the source of the radio waves. This makes it possible to detect only mobile units 20 manufactured by a specific manufacturer.

[0053] Here, the information indicating the predetermined characteristics is referred to as characteristic information. The characteristic information is stored in a storage device accessible by the first detection unit 2020. The first detection unit 2020 acquires the characteristic information and determines whether or not a sound having the predetermined characteristics indicated in the characteristic information has been detected by the acoustic sensor, thereby detecting a moving object 20 that emits a sound having the predetermined characteristics.

[0054] Here, the characteristics of radio waves emitted by a mobile object to be detected as mobile object 20, i.e., information indicating the above-mentioned predetermined characteristics, is referred to as radio wave characteristic information. The radio wave characteristic information is stored in a storage device accessible by first detection unit 2020. First detection unit 2020 acquires the radio wave characteristic information and determines whether radio waves having the predetermined characteristics indicated in the radio wave characteristic information have been detected by the radio wave sensor. Then, when radio waves having the predetermined characteristics are detected by the radio wave sensor, first detection unit 2020 identifies the direction from sensor 10 toward the source of the radio waves as first direction 12.

[0055] <Identification of the first direction 12> The first detection unit 2020 identifies a first direction 12 using the sensor 10. The first direction 12 is the direction from the sensor 10 to the moving object 20. For example, when the second camera 40 is used as the sensor 10, the first detection unit 2020 identifies the first direction 12 based on various setting parameters (such as the imaging direction and focal length) of the camera 30 and the position of the moving object 20 in the second captured image 42. Here, existing technology can be used to identify the direction from the camera to the object based on the setting parameters of the camera and the position of the object in the captured image.

[0056] When the above-described acoustic sensor is used as sensor 10, for example, control unit 2040 determines the direction from the acoustic sensor toward moving object 20 based on the difference in timing at which each microphone detects a sound emitted from moving object 20. Here, existing technology can be used to determine the direction toward the sound source of a sound based on the difference in timing at which each microphone arranged three-dimensionally detects the sound. Note that when the above-described radio wave sensor is used as sensor 10, first direction 12 can be determined in the same manner as when an acoustic sensor is used.

[0057] <Control of camera 30: S106, S108> The control unit 2040 causes the camera 30 to capture images multiple times while moving the optical axis direction 34 of the camera 30 along the first direction 12 (S106, S108). Here, the control unit 2040 may change the optical axis direction 34 of the camera 30 by changing the attitude of the camera 30, or may change the optical axis direction 34 of the camera 30 without changing the attitude of the camera 30. In the former case, for example, the control unit 2040 moves the optical axis direction 34 of the camera 30 by changing the attitude of a platform such as a cloud platform on which the camera 30 is installed. On the other hand, in the latter case, for example, an optical element (such as a mirror) is arranged on the optical axis of the camera 30, and the control unit 2040 moves the optical axis direction 34 of the camera 30 by controlling the attitude of this optical element, etc.

[0058] The initial position of the position in the first direction 12 through which the optical axis direction 34 of the camera 30 passes may be statically or dynamically determined. In the former case, for example, the starting point of the first direction 12 is set as the initial position.

[0059] On the other hand, in the latter case, for example, a spatial range (hereinafter, detection space) for detecting the moving object 20 is determined in advance using the information processing device 2000, and the position where the first direction 12 enters the detection space is set as the initial position. FIG. 7 is a diagram illustrating an example of the detection space. In FIG. 7, the detection space is represented by the reference numeral 60. The position where the first direction 12 enters the detection space 60 is position 62. Therefore, the control unit 2040 sets the initial position on the first direction 12, through which the optical axis direction 34 of the camera 30 passes, to position 62. Note that the position on the first direction 12 where the first direction 12 exits the detection space 60 is position 64. Therefore, the control unit 2040 may move the optical axis direction 34 of the camera 30 from position 62 to position 64.

[0060] Alternatively, for example, the control unit 2040 may determine the initial position based on the detection result of the sensor 10. For example, assume that the sensor 10 is a camera. In this case, the control unit 2040 acquires information indicating the size of the moving object 20 in real space by identifying the type of the moving object 20 included in the captured image 32. The control unit 2040 estimates the distance from the sensor 10 to the moving object 20 based on the size of the moving object 20 in real space and the size of the moving object 20 in the captured image 32. The control unit 2040 then determines a position that is the estimated distance away from the sensor 10 along the first direction 12 as the initial position in the first direction 12, through which the optical axis direction 34 of the camera 30 passes. Note that existing technology can be used to estimate the distance from the camera to an object based on the size of the object in real space and its size in the captured image.

[0061] There are various methods for causing the camera 30 to capture images multiple times. For example, a video camera that captures images periodically at a predetermined cycle (for example, 30 fps (frames per second)) is used as the camera 30. The camera 30 may start capturing images in response to control by the control unit 2040, or may capture images continuously. In the latter case, the second detection unit 2060 uses, of the captured images 32 generated by the camera 30, the captured images 32 generated after the control unit 2040 started to control the capturing direction of the camera 30.

[0062] Alternatively, for example, the control unit 2040 may transmit a control signal instructing the camera 30 to capture an image at each timing when the camera 30 is to capture an image. In this case, the camera 30 captures an image at the timing when the control signal is received.

[0063] <<About the focus of the camera 30>> The control unit 2040 may set the focal length of the camera 30 so that the focal point of the camera 30 is located in the first direction 12. Specifically, the control unit 2040 calculates the distance from the camera 30 to the intersection of the imaging direction of the camera 30 and the first direction 12, and sets this distance as the focal length of the camera 30. The control unit 2040 moves the optical axis direction of the camera 30 and changes the focal length of the camera 30.

[0064] By setting the focal point of the camera 30 in the first direction 12 in this manner, the camera 30 can capture an image of the moving object 20 in focus. This makes it easier for the second detection unit 2060 to detect the moving object 20 from the captured image 32.

[0065] Furthermore, even if an object other than the moving body 20 exists in the direction of the optical axis of the camera 30, the object will be captured out of focus. Therefore, by detecting only the in-focus objects among the objects included in the captured image 32 as the moving body 20, it is possible to prevent erroneous detection of the moving body 20. In other words, it is possible to prevent an object other than the object detected as the moving body 20 by the sensor 10 from being erroneously detected as the moving body 20 by the second detection unit 2060. Note that existing technology can be used to determine whether an object in an image is in focus.

[0066] <Detection of moving object 20 from captured image 32: S112, S114, S116> The second detection unit 2060 detects the moving body 20 from the captured image 32 by detecting an object from the captured image 32 (S112, S114, S116). For example, the second detection unit 2060 performs image analysis on the multiple captured images 32 in order of generation time, and treats the first detected object as the captured image 32. In this way, when multiple objects exist in the first direction 12, the object closest to the sensor 10 among them is detected as the moving body 20.

[0067] However, the second detection unit 2060 may use the detection result by the first detection unit 2020 to determine whether the object detected from the captured image 32 is the same as the moving body 20 detected by the first detection unit 2020. For example, when the sensor 10 is the second camera 40, the first detection unit 2020 calculates a feature amount of the moving body 20 detected from the second captured image 42. From this feature amount, it is possible to grasp the general characteristics of the moving body 20, such as whether the moving body 20 is a bird, an airplane, or a drone.

[0068] Therefore, the second detection unit 2060 determines whether or not the object detected from the captured image 32 has the feature amount of the moving body 20 calculated by the first detection unit 2020. If the object detected from the captured image 32 has this feature amount, the second detection unit 2060 detects the object as a moving body 20. On the other hand, if the object detected from the captured image 32 does not have this feature amount, the second detection unit 2060 determines that the object is not a moving body 20.

[0069] As described above, there are cases where the type of moving object 20 to be detected by the sensor 10 is predetermined. For example, if the sensor 10 is an acoustic sensor and detects the sound of the rotors of a specific model of drone, the moving object 20 detected by the sensor 10 is that specific model of drone. Therefore, the second detection unit 2060 may limit the objects to be detected from the captured image 32 to that specific model of drone. In this case, for example, a feature commonly shared by objects classified into the type to be detected using the sensor 10 is determined in advance. If the object detected from the captured image 32 has this feature, the second detection unit 2060 detects the object as a moving object 20. On the other hand, if the object detected from the captured image 32 does not have this feature, the second detection unit 2060 determines that the object is not a moving object 20.

[0070] Alternatively, for example, when the focus of the camera 30 is controlled so that the focus of the camera 30 is positioned in the first direction 12 as described above, the second detection unit 2060 may detect only those objects that are in focus among the objects detected from the captured image 32 as the moving body 20. Here, existing technology can be used as the technology for determining whether or not an object included in an image is in focus.

[0071] <Update of 1st direction 12> Here, the moving object 20 can be detected using the sensor 10 even while the optical axis direction 34 of the camera 30 is being moved along the first direction 12. When the position of the moving object 20 detected by the sensor 10 changes, the first direction 12 changes accordingly. In this case, the control unit 2040 may update the first direction 12 based on the detection result by the sensor 10.

[0072] 8 is a diagram illustrating an example of updating the first direction 12. The first direction 12-1 is the first direction identified by the first detection unit 2020 based on the detection result of the sensor 10 at time t1. After time t1, the control unit 2040 causes the camera 30 to capture an image while moving the optical axis direction 34 of the camera 30 along the first direction 12-1.

[0073] Thereafter, the first detection unit 2020 identifies the first direction 12-2 based on the detection result of the sensor 10 at time t2. Therefore, the control unit 2040 moves the optical axis direction 34 of the camera 30 along the first direction 12-2 instead of the first direction 12-1 after time t2. Thereafter, similarly, the first direction 12-3 is identified based on the detection result of the sensor 10 at time t3, and the optical axis direction 34 of the camera 30 is moved along the first direction 12-3 after time t3.

[0074] By updating the first direction 12 while the camera 30 is being moved along the first direction 12 in this way, the detection of the moving object 20 using the camera 30 is performed taking into consideration changes in the position of the moving object 20 while the optical axis direction 34 of the camera 30 is being moved along the first direction 12. This makes it possible to more reliably detect the moving object 20.

[0075] <When multiple sensors 10 and cameras 30 are used> In the above description, an example was given in which there was one sensor 10 and one camera 30. However, there may be more than one sensor 10 and one camera 30. Below, a specific example will be given in which there are multiple sensors 10 and multiple cameras 30.

[0076] 9 and 10 are diagrams illustrating a use case in which multiple sensors 10 and cameras 30 are provided. FIG. 9 is a diagram showing a vertical plan view of an area in which a moving object 20 is monitored. FIG. 10 is a diagram showing a side view of an area in which a moving object 20 is monitored. In this use case, the moving object 20 is a drone 120. The drone 120 flies between a building 100 and a building 110. Its flight route is set within a monitoring area 130.

[0077] Eight sensors 10 and two cameras 30 are provided in the monitoring area 130. Camera 30-1 is located at the center of sensors 10-1 to 10-4. Meanwhile, camera 30-2 is located at the center of sensors 10-5 to 10-7. The dotted rectangles within the monitoring area 130 represent the range in which each sensor 10 detects the drone 120.

[0078] The first detection unit 2020 detects the drone 120 using sensors 10-1 to 10-8. Here, when the drone 120 is detected by any of sensors 10-1 to 10-4, the control unit 2040 controls the camera 30-1 to generate a captured image 32 and detects the drone 120 from the captured image 32. On the other hand, when the drone 120 is detected by any of sensors 10-5 to 10-7, the control unit 2040 controls the camera 30-2 to generate a captured image 32 and detects the drone 120 from the captured image 32.

[0079] [Embodiment 2] 11 is a diagram illustrating an example of the functional configuration of an information processing apparatus 2000 according to the second embodiment. Except for the matters described below, the information processing apparatus 2000 according to the second embodiment has the same functions as the information processing apparatus 2000 according to the first embodiment.

[0080] The information processing device 2000 of the second embodiment includes an output unit 2080. The output unit 2080 outputs a captured image 32 in which the moving object 20 is detected by the second detection unit 2060. There are various methods for outputting the captured image 32. For example, the output unit 2080 outputs the captured image 32 in which the moving object 20 is detected to a storage device, thereby storing the captured image 32 in the storage device. Alternatively, for example, the output unit 2080 outputs the captured image 32 in which the moving object 20 is detected to a display device, thereby displaying the captured image 32 on the display device. Alternatively, for example, the output unit 2080 may transmit the captured image 32 in which the moving object 20 is detected to a terminal other than the information processing device 2000. For example, the output unit 2080 transmits the captured image 32 in which the moving object 20 is detected to a terminal (hereinafter, a monitoring terminal) used by a monitor who is monitoring the moving object. The captured image 32 received by the monitoring terminal is displayed, for example, on a display device provided in the monitoring terminal.

[0081] Here, the information processing device 2000 may have a function of causing the camera 30 to track the moving object 20 so that the camera 30 continues to capture images of the moving object 20 after the moving object 20 is detected by the second detection unit 2060. In this case, the moving object 20 is included in each captured image 32 generated after the moving object 20 is detected by the second detection unit 2060. That is, video data including the moving object 20 is generated by the camera 30. In this case, the output unit 2080 may output video data including not only the captured image 32 in which the moving object 20 is detected by the second detection unit 2060, but also the captured images 32 generated thereafter. Note that existing technology can be used to cause the camera to track a specific object.

[0082] <Example of hardware configuration> The hardware configuration of the computer that realizes the information processing device 2000 of the second embodiment is shown in Fig. 3, for example, as in the first embodiment. However, the storage device 1080 of the computer 1000 that realizes the information processing device 2000 of this embodiment further stores a program module that realizes the functions of the information processing device 2000 of this embodiment.

[0083] <Action and effect> According to the information processing device 2000 of this embodiment, a captured image 32 in which the moving object 20 is detected is output. Here, since the image capturing by the camera 30 can be performed while moving the optical axis direction 34 along the first direction 12 identified using the sensor 10, the capturing range of the camera 30 can be narrowed to a certain extent. For example, as described above, a telephoto camera can be used as the camera 30. Therefore, according to the information processing device 2000, a captured image 32 in which the moving object 20 is captured in a large size can be obtained, and the captured image 32 can be used to grasp the appearance of the moving object 20 in detail.

[0084] [Embodiment 3] 12 is a diagram illustrating an example of the functional configuration of an information processing device 2000 of embodiment 3. Except for the matters described below, the information processing device 2000 of embodiment 3 has the same functions as the information processing device 2000 of embodiment 1 or 2.

[0085] The information processing device 2000 of the second embodiment includes a calculation unit 2100. The calculation unit 2100 has a function of calculating the three-dimensional position of the moving object 20 detected by the second detection unit 2060. According to the information processing device 2000, the moving object 20 is detected by the sensor 10 and the camera 30, which are installed at different positions. Therefore, the three-dimensional position of the moving object 20 is determined as the intersection of the direction from the sensor 10 toward the moving object 20 (first direction 12) and the direction from the camera 30 toward the moving object 20 (hereinafter, second direction). Therefore, the calculation unit 2100 calculates the intersection of the first direction 12 and the second direction, and defines the intersection as the three-dimensional position of the moving object 20.

[0086] The three-dimensional position can be expressed, for example, as a combination of GPS coordinates and altitude. Specifically, the x and y coordinates are the x and y coordinates of the GPS coordinates, respectively, and the z coordinate is the altitude. Alternatively, for example, the three-dimensional position of the moving object 20 may be expressed as a relative position based on the positions of the sensor 10 and the camera 30.

[0087] It is preferable that the three-dimensional coordinates of the moving object 20 calculated by the calculation unit 2100 are output by the same method as the method by which the output unit 2080 detects the captured image 32 in which the moving object 20 is detected. For example, when the captured image 32 is displayed on a display device, it is preferable that information on the three-dimensional position of the moving object 20 is superimposed and displayed near the moving object 20 on the captured image 32.

[0088] <Example of hardware configuration> The hardware configuration of the computer that realizes the information processing device 2000 of the third embodiment is shown in Fig. 3, for example, as in the first embodiment. However, the storage device 1080 of the computer 1000 that realizes the information processing device 2000 of this embodiment further stores a program module that realizes the functions of the information processing device 2000 of this embodiment.

[0089] <Action and effect> According to the information processing device 2000 of this embodiment, the three-dimensional position of the moving object 20 can be determined based on the directions from the sensors 10 and the cameras 30, which are arranged at different positions, toward the moving object 20. The three-dimensional position calculated in this manner can be used for various purposes, such as monitoring and tracking the moving object 20.

[0090] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and it is also possible to adopt configurations that combine the above embodiments or various other configurations.

[0091] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0092] Below, examples of reference forms are added. 1. A first detection means for detecting a moving object using a sensor and identifying a first direction from the sensor toward the moving object; a control means for causing the first camera to capture an image while moving the optical axis direction of the first camera along the first direction; and second detection means for detecting the moving object from a first captured image generated by the first camera. 2. The information processing device described in 1., wherein the second detection means treats the moving object detected from the first captured image generated earliest among the first captured images in which a moving object is detected as the moving object detected by the sensor. 3. The sensor is a second camera having a focal length shorter than that of the first camera; 3. The information processing device according to 1. or 2., wherein the first detection means detects the moving object by performing image analysis on the captured image generated by the second camera. 4. The sensor is an acoustic sensor; The information processing device described in 1. or 2., wherein the first detection means acquires information representing the characteristics of the sound emitted by the moving object to be detected, and when a sound having the characteristics is detected by the acoustic sensor, identifies the direction from the acoustic sensor toward the source of the sound as the first direction. 5. The sensor is a radio wave sensor; The information processing device described in 1. or 2., wherein the first detection means acquires information representing the characteristics of radio waves emitted by the moving body to be detected, and when radio waves having the characteristics are detected by the radio wave sensor, identifies the direction from the radio wave sensor toward the source of the radio waves as the first direction. 6. An information processing device described in any one of 1. to 5., wherein the control means determines an initial position of the imaging direction of the first camera based on the size of the moving object detected by the sensor, and moves the imaging direction of the first camera from the initial position along the first direction. 7. An information processing device described in any one of 1. to 6., wherein the control means sets the focal length of the first camera so that the focus of the first camera is positioned in the first direction. 8. The first detection means repeatedly identifies the first direction; The information processing device described in any one of 1. to 7., wherein when the first direction is newly identified while the optical axis direction of the first camera is being moved along the first direction, the control means moves the optical axis direction of the first camera along the newly identified first direction. 9. The information processing device described in any one of 1. to 8., further comprising an output means for outputting the first captured image in which the moving object is detected. 10. An information processing device described in any one of 1. to 9., having a calculation means for calculating the three-dimensional position of the moving object based on the first direction and the optical axis direction of the first camera when the moving object is detected.

[0093] 11. A computer-implemented control method comprising: a first detection step of detecting a moving object using a sensor and identifying a first direction from the sensor toward the moving object; a control step of causing the first camera to capture an image while moving the optical axis direction of the first camera along the first direction; a second detection step of detecting the moving object from a first captured image generated by the first camera. 12. A control method described in 11, wherein in the second detection step, the moving object detected from the first captured image generated earliest among the first captured images in which a moving object is detected is treated as the moving object detected by the sensor. 13. The sensor is a second camera having a focal length shorter than that of the first camera; 13. The control method according to 11. or 12., wherein in the first detection step, the moving object is detected by performing image analysis on the captured image generated by the second camera. 14. The sensor is an acoustic sensor; 11. or 12. A control method according to claim 11, wherein in the first detection step, information representing the characteristics of the sound emitted by the moving object to be detected is acquired, and when a sound having the characteristics is detected by the acoustic sensor, the direction from the acoustic sensor toward the source of the sound is identified as the first direction. 15. The sensor is a radio wave sensor; A control method described in 11. or 12., wherein in the first detection step, information representing the characteristics of radio waves emitted by the moving object to be detected is obtained, and when radio waves having the characteristics are detected by the radio wave sensor, the direction from the radio wave sensor toward the source of the radio waves is identified as the first direction. 16. A control method described in any one of 11. to 15., wherein in the control step, an initial position of the imaging direction of the first camera is determined based on the size of the moving object detected by the sensor, and the imaging direction of the first camera is moved from the initial position along the first direction. 17. A control method according to any one of claims 11 to 16, wherein in the control step, the focal length of the first camera is set so that the focus of the first camera is positioned in the first direction. 18. In the first detection step, the first direction is repeatedly identified; A control method described in any one of 11. to 17., wherein, in the control step, if a new first direction is identified while the optical axis direction of the first camera is being moved along the first direction, the optical axis direction of the first camera is moved along the newly identified first direction. 19. The control method described in any one of 11 to 18, further comprising an output step of outputting the first captured image in which the moving object is detected. 20. A control method described in any one of 11. to 19., comprising a calculation step of calculating the three-dimensional position of the moving object based on the first direction and the optical axis direction of the first camera when the moving object is detected.

[0094] 21. A program that causes a computer to execute each step of the control method described in any one of 11 to 20. [Explanation of symbols]

[0095] 10 sensors 12 1st direction 20 Mobile 30 Camera 32 Captured images 34 Optical axis direction 40 Second Camera 42 Second captured image 50 Touch Panel 52 videos 56 areas 58 Drone 60 Detection Space 62 positions 64 positions 100 Building 110 Building 120 Drone 130 Surveillance Area 1000 calculator 1020 Bus 1040 processor 1060 memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 Information Processing Equipment 2020 First Detection Unit 2040 Control Unit 2060 Second detection unit 2080 output unit 2100 Calculation Unit

Claims

1. a detection means for detecting a moving object in a predetermined area set in advance within a first photographing area that is a photographing area of ​​the first camera; a control means for controlling the second camera when the moving object is detected so that the moving object is included in a second photographing area that is a photographing area of ​​the second camera; Equipped with the predetermined area is a part of the first imaging area, when the position of the moving object is identified within the first photographing area while the control means is moving the orientation of the second camera based on the position of the moving object, the control means moves the orientation of the second camera based on the identified position of the moving object. Information processing device.

2. When the moving object is detected in the predetermined area, the control means controls the second camera based on the position of the detected moving object. The information processing device according to claim 1 .

3. the control means controls the second camera so that the moving object is included in the second shooting area and the moving object is positioned at the center of the second shooting area.

3. The information processing device according to claim 1 or 2.

4. a setting means for setting the predetermined area within the first photographing area, which is a photographing area of ​​the first camera, based on an input for an image photographed by the first camera; The information processing device according to claim 1 , further comprising:

5. the detection means detects the moving object that is included in the predetermined area and has a predetermined feature amount. The information processing device according to claim 1 .

6. the setting means sets an area designated based on an input by a drag operation on a screen on which the image is displayed as the predetermined area; The information processing device according to claim 4 .

7. 1. A computer-implemented control method comprising: a detection step of detecting a moving object in a predetermined area set in advance within a first imaging area that is an imaging area of ​​the first camera; a control step of controlling the second camera when the moving object is detected so that the moving object is included in a second imaging area that is an imaging area of ​​the second camera; and the predetermined area is a part of the first imaging area, In the control step, when the position of the moving object is identified within the first photographing area while the orientation of the second camera is being moved based on the position of the moving object, the orientation of the second camera is moved based on the identified position of the moving object. Control method.

8. a setting step of setting the predetermined area within the first photographing area, which is a photographing area of ​​the first camera, based on an input for an image photographed by the first camera; The control method of claim 7 further comprising:

9. Computer, a detection means for detecting a moving object in a predetermined area of ​​a first photographing area, which is a photographing area of ​​the first camera; a control means for controlling the second camera when the moving object is detected so that the moving object is included in a second photographing area that is a photographing area of ​​the second camera; It functions as the predetermined area is a part of the first imaging area, when the position of the moving object is identified within the first photographing area while the control means is moving the orientation of the second camera based on the position of the moving object, the control means moves the orientation of the second camera based on the identified position of the moving object. program.

10. The computer a setting means for setting the predetermined area within the first photographing area, which is a photographing area of ​​the first camera, based on an input for an image photographed by the first camera; The program according to claim 9, further functioning as:

Images