Device, system, method and recording medium

Abstract

The present application relates to a device, a system, a method and a recording medium. A device is provided, comprising: an acquisition unit, which acquires a camera image from a camera whose direction can be controlled; a display control unit, which displays the camera image on a display screen; a detection unit, which detects a subject being viewed by a user in the display screen; and a direction changing unit, which changes the direction of the camera so that the subject does not leave the camera's camera area.

Description

Technical Field

[0001] The present invention relates to an apparatus, a system, a method and a recording medium. Background Art

[0002] Patent Document 1 states that “the user's gaze direction is detected, and the position (area) on the monitor 11 that the user is gazing at is estimated. Then, AF processing is performed to focus on a subject within a predetermined range including the gaze position.]”.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 6241107 Summary of the invention

[0006] In a first aspect of the present invention, a device is provided. The device may include an acquisition unit for acquiring a camera image from a camera capable of controlling its orientation. The device may include a display control unit for displaying the camera image on a display screen. The device may include a detection unit for detecting a subject that a user is viewing within the display screen. The device may include an orientation change unit for changing the orientation of the camera so that the subject does not leave the camera's imaging area.

[0007] The direction changing unit may change the direction of the camera so that the subject is located in the center of the imaging area.

[0008] The detection unit may detect a zoom operation on the subject according to the movement of the user's eyes. The device may further include a size change unit that changes the size of the subject in the display screen according to the zoom operation.

[0009] In a second aspect of the present invention, a system is provided. The system may include the device of the first aspect. The system may include a camera capable of controlling its direction. The system may include a communication device that transmits a camera image captured by the camera to the device.

[0010] The camera and communication device can be carried on the drone.

[0011] The drone can move on a pre-set path.

[0012] The camera may be a surveillance camera.

[0013] In a third aspect of the present invention, a method is provided. The method may include an acquisition phase of acquiring a camera image from a camera capable of controlling its orientation. The method may include a display control phase of causing a display screen to display the camera image. The method may include a detection phase of detecting a subject that a user is viewing in the display screen. The method may include an orientation change phase of changing the orientation of the camera so that the subject does not leave the camera's imaging area.

[0014] In a fourth aspect of the present invention, a recording medium having a program recorded thereon is provided. The program can cause a computer to function as an acquisition unit that acquires a camera image from a camera whose orientation can be controlled. The program can cause a computer to function as a display control unit that causes a display screen to display a camera image. The program can cause a computer to function as a detection unit that detects a subject that a user is viewing within a display screen. The program can cause a computer to function as a direction changing unit that changes the orientation of a camera so that the subject does not leave the camera's imaging area.

[0015] In addition, the above summary of the invention does not list all the essential features of the present invention. In addition, sub-combinations of these feature groups may also constitute inventions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A monitoring system 1 according to this embodiment is shown.

[0017] Figure 2 Indicates the action of device 3.

[0018] Figure 3 Shows an example of a display screen.

[0019] Figure 4 The computer 2200 is an example of a computer 2200 on which various aspects of the present invention may be implemented in whole or in part. DETAILED DESCRIPTION

[0020] Hereinafter, the present invention will be described by way of the embodiments of the invention, but the following embodiments are not intended to limit the invention involved in the claims. In addition, all combinations of the features described in the embodiments are not necessarily required for the solution of the invention.

[0021] [1. Structure of monitoring system 1]

[0022] Figure 1 A monitoring system 1 according to this embodiment is shown. The monitoring system 1 monitors a factory (plant) and includes a drone 2 and a device 3 .

[0023] Here, factories include, in addition to chemical and other industrial factories, factories that manage and control wellheads of gas fields, oil fields, and their surrounding areas; factories that manage and control hydroelectric, thermal, nuclear, and other power generation; factories that manage and control environmental power generation such as solar and wind power; factories that manage and control water supply and drainage, dams, etc.

[0024] [1.1. Drone 2]

[0025] The drone 2 moves in the factory. The drone 2 can move on a pre-set path in the factory. The drone 2 can move autonomously or can be remotely controlled by an operating device or device 3 not shown.

[0026] Drone 2 can be an unmanned aircraft (also called UAV (Unmanned Aerial Vehicle), drone), an unmanned vehicle (also called UGV (unmanned ground vehicle), an unmanned watercraft (also called USV (unmanned surface vehicle)), or an unmanned submarine (also called UUV (unmanned underwater vehicle)).

[0027] The drone 2 includes a camera 21 and a communication device 22 .

[0028] [1.1(1).Camera21]

[0029] The camera 21 takes pictures and provides the taken images to the communication device 22. In this embodiment, as an example, the camera 21 can always take pictures to generate dynamic images (also called images). The camera 21 can control the direction (also called the shooting direction). In this embodiment, as an example, the direction can be changed according to the direction change instruction described later provided by the communication device 22.

[0030] [1.1(2). Communication device 22]

[0031] The communication device 22 communicates with the device 3 wirelessly or by wire.

[0032] The communication device 22 can transmit the captured images captured by the camera 21 to the device 3. In the present embodiment, as an example, the communication device 22 sequentially transmits the captured images to the device 3, but may transmit the captured images at every reference interval (for example, every 10 seconds).

[0033] In addition, the communication device 22 may receive a direction change instruction from the device 3 and provide the instruction to the camera 21 .

[0034] [1.2. Device 3]

[0035] The device 3 assists the monitoring of the factory using the drone 2. The device 3 includes a communication unit 31, a display control unit 32, a display unit 33, a camera unit 34, a detection unit 35, and a direction changing unit 36. In addition, the device 3 may also be arranged in a management room, an instrument room, etc. in the factory. In this embodiment, as an example, the device 3 may be a fixed-type PC, etc.

[0036] [1.2(1).Communication Department 31]

[0037] The communication unit 31 communicates with the communication device 22 of the drone 2 by wireless or wired means. The communication unit 31 is an example of an acquisition unit, and acquires a captured image from the camera 21 via the communication device 22. The communication unit 31 can provide the captured image to the display control unit 32. In addition, the communication unit 31 can also send a direction change instruction supplied from the direction change unit 36 ​​described later to the communication device 22.

[0038] [1.2(2). Display control unit 32]

[0039] The display control unit 32 controls the display unit 33. The display control unit 32 may provide an image of a display object (also referred to as a display image) to the display unit 33 to display it on the display screen. As an example, the display image may be a captured image itself captured by the camera 21.

[0040] The display control unit 32 may be an example of a size change unit, and may change the size of the subject in the display screen according to the zoom operation. In the present embodiment, as an example, an instruction of zooming in (also called zoom in) or zooming out (also called zoom out) corresponding to the zoom operation may be obtained from the detection unit 35 to change the size of the subject.

[0041] The display control unit 32 can change the size of the subject by digital zoom. For example, in response to a zoom-in instruction, the display control unit 32 can provide the display unit 33 with an image of a region smaller than the region currently being displayed in the camera image provided by the communication unit 31, so that the image is zoomed in and displayed. In response to a zoom-out instruction, the display control unit 32 can provide the display unit 33 with an image of a region larger than the region currently being displayed in the camera image provided by the communication unit 31, so that the image is zoomed out and displayed. In this case, the display image provided to the display unit 33 can be a portion of the camera image that is zoomed in and displayed.

[0042] The display control unit 32 may provide the display image to the detection unit 35 and the orientation changing unit 36 ​​.

[0043] [1.2(3). Display unit 33]

[0044] The display unit 33 displays, on a display screen, a display image supplied from the display control unit 32. The display unit 33 may also display other contents.

[0045] [1.2(4). Camera unit 34]

[0046] The imaging unit 34 captures the user's eyes. The imaging unit 34 can provide the captured eye image to the detection unit 35. In the present embodiment, the imaging unit 34 is provided at the periphery of the display screen of the display unit 33 as an example, but may be provided at other positions.

[0047] [1.2(5). Detection unit 35]

[0048] The detection unit 35 performs various detections based on the eye image. The detection unit 35 includes an operation detection unit 351 , a gaze position detection unit 352 , and a subject detection unit 353 .

[0049] [1.2(5-1). Operation detection unit 351]

[0050] The operation detection section 351 detects a zoom operation on the subject according to the movement of the user's eyes. The operation detection section 351 may detect the zoom operation using an image of the eyes supplied from the imaging section 34. The zoom operation may be both a zoom-in operation and a zoom-out operation.

[0051] The operation detection unit 351 can detect a zoom-in operation by squinting the eyes of the user for the entire first reference time (for example, 0.5 seconds), and can detect a zoom-out operation by opening the eyes of the user. The detection unit 351 can detect a zoom-in operation by blinking the eyes of the user at a frequency higher than a reference frequency (for example, three times per second). The operation detection unit 351 can provide a zoom-in or zoom-out instruction to the display control unit 32 according to the detection of the zoom-in operation.

[0052] [1.2(5-2). Gaze position detection unit 352]

[0053] The gaze position detection unit 352 detects the gaze position that the user is looking at within the display screen.

[0054] The gaze position detection unit 352 can detect the user's line of sight. In the present embodiment, as an example, the gaze position detection unit 352 can detect the line of sight by analyzing the image of the eye provided by the camera unit 34. The gaze position detection unit 352 can use the position on the display screen through which the detected line of sight passes as the gaze position. The gaze position detection unit 352 can provide the position information of the detected gaze position (as an example, the coordinates in the display screen) to the subject detection unit 353 and the display control unit 32.

[0055] [1.2(5-3). Subject detection unit 353]

[0056] The subject detection unit 353 detects the subject (also referred to as the target subject) that the user is viewing in the display screen. For example, the subject detection unit 353 may store the feature data of the camera image or appearance in advance for each device or facility in the factory. The subject detection unit 353 may obtain the display image from the display control unit 32, and compare the appearance of the subject obtained by performing image processing such as edge extraction processing on the area including the gaze position in the subject included in the display image with the pre-stored camera image or feature data, thereby detecting the target subject. Alternatively, a specific code may be pre-set on the outer surface of each device or facility in the factory, and the subject detection unit 353 may detect the target subject by analyzing the code detected from the outer surface of the subject obtained by performing image processing such as edge extraction processing on the area including the gaze position. The code may be a barcode or a two-dimensional code (as an example, a QR code (registered trademark)), etc., and may be set on the device or facility by at least one of pasting, printing, or engraving.

[0057] The object detection unit 353 may provide information of the detected target object to the direction changing unit 36. The information of the target object may be feature data of the object, position coordinate data of the object in the captured image, or an identification number preset for equipment or facilities in the factory.

[0058] [1.2(6). Direction change unit 36]

[0059] The orientation changing unit 36 ​​changes the orientation of the camera 21 so that the target subject does not leave the imaging area of ​​the camera 21. For example, the orientation changing unit 36 ​​may change the orientation of the camera 21 so that the target subject is located in the center of the imaging area. The orientation changing unit 36 ​​may change the orientation of the camera 21 by providing the camera 21 with an orientation change instruction via the communication unit 31.

[0060] According to the above device 3, the target subject that the user is viewing is detected in the display screen showing the captured image, and the direction of the camera 21 is changed so that the target subject does not leave the capturing area, so that the target subject can be reliably included in the captured image.

[0061] Furthermore, since the orientation of the camera 21 is changed so that the target subject is located in the center of the imaging area, the target subject can be included in the center of the captured image.

[0062] Furthermore, since the drone 2 is equipped with the camera 21 and the communication device 22 , the target subject can be reliably included in the captured image by the camera 21 mounted on the drone 2 .

[0063] Furthermore, since the drone 2 moves on a preset path, the target subject existing on the path can be reliably included in the captured image by the camera 21 of the drone.

[0064] Furthermore, since the size of the target subject in the display screen is changed according to the zoom operation detected from the movement of the user's eyes, the captured image of the target subject can be viewed at an arbitrary magnification according to the movement of the eyes.

[0065] [2.Action]

[0066] Figure 2 The operation of the device 3 is shown. The device 3 monitors the factory through the processing of steps S11 to S27.

[0067] In step S11, the communication unit 31 obtains a captured image from the camera 21 of the drone 2, and in step S13, the display control unit 32 causes the captured image to be displayed on the display screen. In addition, when the size of the target subject is changed in step S17 described later, the display control unit 32 may cause the display screen to display a display image of a partial area of ​​the captured image so as to maintain the size of the subject.

[0068] In step S15, the display control unit 32 determines whether a zoom operation is detected by the operation detection unit 351. If it is determined that a zoom operation is not detected (step S15; "No"), the process may be transferred to step S21. If it is determined that a zoom operation is detected (step S15; "Yes"), the process may be transferred to step S17.

[0069] In step S17, the display control unit 32 changes the size of the target subject in the display screen according to the zoom operation. The display control unit 32 may change the size of the target subject in stages, or may increase or decrease the display size of the target subject in only one stage according to the situation of performing a zoom operation. The display control unit 32 may enlarge or reduce the camera image based on the position information of the gaze position provided from the gaze position detection unit 352 so that the gaze position does not deviate from the display screen. As an example, the gaze position detection unit 352 may detect the gaze position that the user is looking at in the display screen, and the display control unit 32 may enlarge or reduce the camera image using the detected gaze position as a reference point. Enlarging or reducing the camera image with the gaze position as a reference point may be to enlarge or reduce the camera image in a manner such that the subject at the gaze position does not move.

[0070] In step S21, the gaze position detection unit 352 detects the gaze position of the user in the display screen. If the gaze position detection unit 352 has already detected the gaze position in the process of step S17, the gaze position detection unit 352 may not detect the gaze position again.

[0071] In step S23, the subject detection unit 353 detects the target subject that the user is looking at in the display screen. The subject detection unit 353 can detect the target subject by comparing the appearance of the subject obtained by performing image processing such as edge extraction processing on the area including the gaze position of the subject included in the display image with the pre-stored camera image or feature data.

[0072] The subject detection unit 353 may detect a subject as a target subject based on the fact that the user has continuously viewed the same subject for more than a second reference time before the current time point. After detecting the first subject as the target subject, if the user views the second subject only for less than the second reference time, the subject detection unit 353 may keep the first subject as the target subject. The second reference time may be the same as or different from the first reference time (0.5 seconds as an example) for detecting the zoom operation by the operation detection unit 351. As an example, the second reference time may be 1 second.

[0073] In step S25, the orientation change unit 36 ​​determines whether the target subject is out of the display image, in other words, determines whether the target subject is out of the display screen. For example, the orientation change unit 36 ​​can calculate the moving speed of the target subject in the display image based on the migration of the display image provided by the display control unit 32, and determine whether the target subject is out of the display image after a third reference time has passed. The third reference time may be the same as or different from the second reference time (as an example, 1 second) for detecting the target subject by the subject detection unit 353 or the first reference time (as an example, 0.5 seconds) for detecting the zoom operation by the operation detection unit 351. As an example, the third reference time may be 0.5 seconds. In addition, in step S25, the orientation change unit 36 ​​may also determine whether the target subject is out of the center of the display image, or whether the target subject has already been out of the center of the display image.

[0074] If it is determined in step S25 that the target subject has not left the display image (step S25; No), the process may proceed to step S11. If it is determined that the target subject may leave the display image (step S25; Yes), the process may proceed to step S27.

[0075] In step S27, the orientation changing unit 36 ​​changes the orientation of the camera 21 so that the target subject does not leave the imaging area of ​​the camera 21. The orientation changing unit 36 ​​may change the orientation of the camera 21 so that the target subject is located in the center of the imaging area. In the present embodiment, as an example, the orientation changing unit 26 may change the orientation of the camera 21 so that the target subject does not leave the display image, and preferably, the orientation of the camera 21 may be changed so that the target subject is located in the center of the display image. After the processing of step S27 is completed, the processing may be transferred to step S11.

[0076] [3. Display screen example]

[0077] Figure 3 As shown in the figure, when the appearance of a factory captured by the camera 21 of the drone 2 is displayed on the display screen, when the user looks at the subject surrounded by a double frame in the figure, the direction of the camera 21 is changed so that the subject does not leave the camera area.

[0078] [4. Modifications]

[0079] In the above embodiment, the device 3 is described as having the display unit 33, but it is not necessary to have the display unit 33. In this case, the display control unit 32 can display the captured image on an externally connected display device.

[0080] In addition, the case where the gaze position detection unit 352 detects the user's line of sight based on the camera image of the eye is described, but the line of sight can also be detected by other methods. In this case, the device 3 may not have the camera unit 34, and the gaze position detection unit 352 may have a sensor group (not shown) that detects the user's line of sight or the gaze position of both eyes. The sensor group may include a 6-axis sensor, a magnetic sensor, an electrooculogram sensor, etc. installed on the user's head (as an example, around the eyes). The 6-axis sensor may be a motion sensor (also called an inertial sensor) having a 3-axis acceleration sensor and a 3-axis gyroscope (angular velocity) sensor, which can detect the direction of the user's head. The magnetic sensor may be, for example, a 3-axis geomagnetic sensor. The electrooculogram sensor can detect the direction of the eyeball by detecting the potential difference around the eye. The gaze position detection unit 352 can use these sensors to detect the line of sight by a conventionally known method. For example, when the sensor group includes a 6-axis sensor or a magnetic sensor, the gaze position detection unit 352 can detect the position of the middle point of the two eyes and the direction of the head, and detect a straight line passing through the middle point of the two eyes and toward the front of the head as the line of sight. When the sensor group further includes an electrooculogram sensor, the gaze position detection unit 352 may detect the middle point of both eyes and the gaze position of both eyes, and detect a straight line connecting the middle point of both eyes and the gaze position of both eyes as a line of sight.

[0081] In addition, although the case where the monitoring system 1 includes a single drone 2 has been described, it may include a plurality of drones 2. In this case, images captured by the cameras 21 of the respective drones 2 may be displayed on the display screen.

[0082] Furthermore, although the device 3 is described as a stationary PC, it may be any other device such as a mobile terminal, and as an example, it may be a wearable terminal having a head-mounted display.

[0083] In addition, the camera 21 is described as always capturing images, but it may capture images at regular intervals or when a shutter operation is performed on an unillustrated operating device or device 3. In addition, the camera 21 is described as generating moving images, but it may generate still images.

[0084] In addition, the case where the camera 21 is mounted on the movable drone 2 is described, but the camera 21 may also be a surveillance camera as long as the direction can be controlled. In this case, the subject existing in the surveillance area based on the surveillance camera can be reliably included in the camera image. In addition, the surveillance camera can be arranged on roads, parking lots, electric lights, electric poles, and can also be arranged at the entrances and exits of facilities, passages, etc. The facility can be a factory, a school or a residence, a station, an airport, a museum, a hospital, a store, a theme park or an amusement park, and their recreational facilities.

[0085] In addition, various embodiments of the present invention may be described with reference to flow charts and block diagrams, wherein a block diagram may represent (1) a stage of a process of performing an operation, or (2) a portion of a device having the function of performing an operation. Specific stages and portions may be implemented by dedicated circuits, programmable circuits provided together with computer-readable instructions stored on a computer-readable medium, and / or processors provided together with computer-readable instructions stored on a computer-readable medium. Dedicated circuits may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits may include reconfigurable hardware circuits that include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0086] A computer-readable medium may include any tangible device capable of storing instructions executed by an appropriate device, with the result that a computer-readable medium having instructions stored therein has a product containing instructions that can be executed in order to create a portion for performing the operations specified in a flowchart or block diagram. As an example of a computer-readable medium, electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. may be included. As a more specific example of a computer-readable medium, a floppy disk (registered trademark), a magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray (RTM) disk, a memory stick, an integrated circuit card, etc. may be included.

[0087] Computer readable instructions may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, and any of source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and existing procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0088] The computer readable instructions can be provided to a processor or programmable circuit of a general-purpose computer, a special-purpose computer or other programmable data processing device via a local or local area network (LAN), a wide area network (WAN) such as the Internet, and the computer readable instructions are executed to create a unit for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0089] Figure 4 An example of a computer 2200 that can implement all or part of the various embodiments of the present invention is shown. The program installed in the computer 2200 can cause the computer 2200 to function as an operation associated with the device involved in the embodiment of the present invention or one or more parts of the device, or can cause the computer 2200 to perform the operation or the one or more parts, and / or can cause the computer 2200 to perform a process involved in the embodiment of the present invention or a stage of the process. Such a program can be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described in this specification.

[0090] The computer 2200 according to the present embodiment includes a CPU 2212, a RAM 2214, a graphic controller 2216, and a display device 2218, which are connected to each other through a host controller 2210. The computer 2200 also includes input / output parts such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes conventional input / output parts such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0091] The CPU 2212 controls each unit by operating according to the programs stored in the ROM 2230 and the RAM 2214. The graphic controller 2216 obtains image data generated by the CPU 2212 in a frame buffer or the like provided in the RAM 2214 or in the graphic controller 2216 itself, and displays the image data on the display device 2218.

[0092] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0093] The ROM 2230 stores therein a boot program or the like executed by the computer 2200 upon activation, and / or a program that depends on the hardware of the computer 2200. The input / output chip 2240 can also connect various input / output sections to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0094] The program is provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from the computer-readable medium, installed in a hard disk drive 2224, a RAM 2214, or a ROM 2230, which is also an example of a computer-readable medium, and executed by a CPU 2212. Information processing written in these programs is read by the computer 2200, and cooperation between the program and the various types of hardware resources described above is provided. The device or method can be constituted by implementing the operation or processing of information according to the use of the computer 2200.

[0095] For example, in the case where communication is performed between the computer 2200 and an external device, the CPU 2212 can execute a communication program loaded into the RAM 2214, and based on the processing written in the communication program, instruct the communication interface 2222 to perform communication processing. Under the control of the CPU 2212, the communication interface 2222 reads the transmission data stored in the transmission buffer processing area provided in the recording medium such as the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or the IC card, and transmits the read transmission data to the network, or writes the reception data received from the network to the reception buffer processing area provided on the recording medium, etc.

[0096] In addition, the CPU 2212 may also read all or a necessary part of a file or database stored in an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), or an IC card into the RAM 2214, and perform various types of processing on the data on the RAM 2214. Then, the CPU 2212 writes the processed data back to the external recording medium.

[0097] Various types of information, such as various types of programs, data, tables, and databases, can be stored in a recording medium and receive information processing. CPU2212 can perform various types of processing on the data read from RAM2214 and write the results back to RAM2214, and the processing includes various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, information retrieval / replacement, etc. specified by the instruction sequence of the program recorded in various places of this disclosure. In addition, CPU2212 can also retrieve information from files, databases, etc. in the recording medium. For example, in the case where multiple entries with attribute values ​​of a first attribute respectively associated with attribute values ​​of a second attribute are stored in a recording medium, CPU2212 can retrieve an entry that specifies an attribute value of the first attribute and is consistent with the condition from the multiple entries, and read the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that meets the predetermined condition.

[0098] The programs or software modules described above may be stored in a computer-readable medium on or near the computer 2200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may also be used as a computer-readable medium, thereby providing the program to the computer 2200 via the network.

[0099] The present invention has been described above using the embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It can be clearly stated in the claims that the methods obtained by making such changes or improvements can also be included in the technical scope of the present invention.

[0100] It should be noted that the execution order of each process such as actions, processes, steps and stages in the device, system, program and method shown in the claims, specifications and drawings can be implemented in any order unless it is specifically stated as "before", "first", etc., and as long as the output of the previous process is not used for the subsequent process. Even if the action flow in the claims, specifications and drawings is described using "first", "next", etc. for convenience, it does not mean that it must be implemented in this order.

[0101] Description of symbols

[0102] 1 Monitoring system

[0103] 2. Drone

[0104] 3. Installation

[0105] 21 Camera

[0106] 22 communication devices

[0107] 31 Ministry of Communications

[0108] 32 Display control unit

[0109] 33 Display unit

[0110] 34 Camera Department

[0111] 35 Inspection Department

[0112] 36 Direction Change Department

[0113] 351 Operation Detection Unit

[0114] 352 gaze position detection unit

[0115] 353 Subject Detection Unit

[0116] 2200 Computer

[0117] 2201DVD-ROM

[0118] 2210 Host Controller

[0119] 2212CPU

[0120] 2214RAM

[0121] 2216 Graphics Controller

[0122] 2218 Display Devices

[0123] 2220 Input / Output Controller

[0124] 2222 communication interface

[0125] 2224 Hard Drive

[0126] 2226DVD-ROM drive

[0127] 2230ROM

[0128] 2240 Input / Output Chip

[0129] 2242 keyboard.

Claims

1. A device comprising: An acquisition unit that acquires a captured image from a camera that is mounted on a movable drone and can control its direction; A display control unit, causing the camera image to be displayed on a display screen; a detection unit, configured to detect that a user is continuously viewing a subject for at least a second reference time in the display screen; as well as The direction changing unit calculates the moving speed of the subject in the display image based on the migration of the display image displayed in the display control unit, determines whether the subject will leave the display image after a third reference time has passed, and changes the direction of the camera in response to determining that the subject will leave the display image so that the subject does not leave the camera's shooting area.

2. The device according to claim 1, wherein: The direction changing unit changes the direction of the camera so that the object is located at a center of an imaging area.

3. The device according to claim 1 or 2, wherein: The detection unit detects a zoom operation on the subject according to a movement of the user's eyes. The device also includes: The size changing unit changes the size of the subject in the display screen according to the zooming operation.

4. A system comprising: The device according to any one of claims 1 to 3; said camera; as well as The communication device transmits the camera image captured by the camera to the device.

5. The system according to claim 4, wherein: The communication device is carried on the drone.

6. The system according to claim 5, wherein: The drone moves on a pre-set path.

7. The system according to claim 5, wherein: The camera is a surveillance camera.

8. A method comprising: In the acquisition phase, a camera image is acquired from a camera mounted on a movable drone whose direction can be controlled; Display control stage, causing the camera image to be displayed on a display screen; In a detection stage, detecting in the display image that the user is continuously viewing the object for at least a second reference time; as well as In the orientation change stage, the moving speed of the subject in the display image is calculated based on the migration of the display image displayed in the display control stage, and it is determined whether the subject will leave the display image after a third reference time has passed. In response to the determination that the subject will leave the display image, the orientation of the camera is changed so that the subject does not leave the camera's imaging area.

9. A recording medium recording a program for causing a computer to function as: An acquisition unit that acquires a captured image from a camera that is mounted on a movable drone and can control its direction; A display control unit, causing the camera image to be displayed on a display screen; a detection unit, configured to detect that a user is continuously viewing a subject for at least a second reference time in the display screen; as well as The direction changing unit calculates the moving speed of the subject in the display image based on the migration of the display image displayed in the display control unit, determines whether the subject will leave the display image after a third reference time has passed, and changes the direction of the camera in response to determining that the subject will leave the display image so that the subject does not leave the camera's shooting area.

Images