Imaging device, imaging method, and program
The imaging device maintains focus on objects within a target area by fixing the region's position when the device moves, addressing the challenge of autofocus accuracy during motion.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- JVC KENWOOD CORP
- Filing Date
- 2021-09-27
- Publication Date
- 2026-05-15
AI Technical Summary
Existing imaging devices with autofocus methods struggle to appropriately focus on objects within a target region, especially when the device is in motion.
An imaging device equipped with a self-position acquisition unit, object information acquisition unit, target region acquisition unit, and focus position control unit that maintains the target region's position fixed when the device moves, allowing for precise focus adjustment on objects within the target area.
Enables proper focusing on objects within the target area even when the imaging device is in motion, ensuring accurate image capture.
Smart Images
Figure 0007859027000001 
Figure 0007859027000002 
Figure 0007859027000003
Abstract
Description
Technical Field
[0001] The present invention relates to an imaging device, an imaging method, and a program.
Background Art
[0002] An imaging device with an autofocus method for automatically setting the focus position is known. For example, Patent Document 1 describes focusing on a predetermined position specified by the user.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] In an imaging device with an autofocus method, it is required to appropriately focus.
[0005] In view of the above problems, an object of the present invention is to provide an imaging device, an imaging method, and a program capable of appropriately focusing.
Means for Solving the Problems
[0006] An imaging device according to an aspect of the present invention is an imaging device capable of imaging an object, including an imaging element, a self-position acquisition unit that acquires position information of the imaging device, an object information acquisition unit that acquires position information of an object existing in an imaging region of the imaging element, a target region acquisition unit that sets a target region within the imaging region, and a focus position control unit that controls the focus position of the imaging device to focus on the object when an object exists within the target region. When it is determined by the self-position acquisition unit that the imaging device has moved, the target region acquisition unit keeps the position of the target region fixed.
[0007] An imaging method according to one aspect of the present invention is an imaging method for imaging an object, comprising the steps of: acquiring position information of an imaging device; acquiring position information of an object located in the imaging area of an image sensor; setting a target area within the imaging area; and, if an object is located within the target area, controlling the focal position of the imaging device to focus on that object, wherein in the step of setting the target area, if it is determined that the imaging device has moved, the position of the target area is kept fixed.
[0008] A program according to one aspect of the present invention is a program that causes a computer to execute an imaging method for imaging an object, and causes the computer to execute the following steps: acquiring position information of an imaging device; acquiring position information of an object present in the imaging area of an image sensor; setting a target area within the imaging area; and, if an object is present in the target area, controlling the focal position of the imaging device so as to focus on that object, wherein in the step of setting the target area, if it is determined that the imaging device has moved, the position of the target area is kept fixed. [Effects of the Invention]
[0009] According to the present invention, it is possible to properly focus the image. [Brief explanation of the drawing]
[0010] [Figure 1] Figure 1 is a schematic block diagram of an imaging device according to the first embodiment. [Figure 2] Figure 2 is a schematic diagram illustrating an example of a target area. [Figure 3] Figure 3 is a schematic diagram illustrating an example of a target area. [Figure 4] Figure 4 is a schematic diagram illustrating another example of the target area. [Figure 5] Figure 5 is a schematic diagram illustrating another example of the target area. [Figure 6] Figure 6 is a schematic diagram showing an example of the target area when set to the first mode. [Figure 7] Figure 7 is a schematic diagram showing an example of the target area when set to the second mode. [Figure 8] Figure 8 is a flowchart illustrating the flow of setting the target area when the imaging device is moved. [Figure 9] Figure 9 is a flowchart illustrating the process flow for setting the focal point. [Figure 10] Figure 10 is a schematic block diagram of the imaging device according to the second embodiment. [Figure 11] Figure 11 is a schematic diagram showing an example of a target area in the second embodiment. [Figure 12] Figure 12 is a flowchart illustrating the alarm notification flow. [Figure 13] Figure 13 is a schematic diagram illustrating an example where the motion of an object is subject to predetermined conditions. [Modes for carrying out the invention]
[0011] Embodiments of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below.
[0012] (First Embodiment) (Configuration of the imaging device) Figure 1 is a schematic block diagram of an imaging device according to the first embodiment. The imaging device 100 according to the first embodiment is an imaging device that captures images of objects within the imaging range. The imaging device 100 is an autofocus camera capable of automatically setting the focal position. The imaging device 100 may be a video camera that captures moving images by capturing images at predetermined frame intervals, or it may be a camera that captures still images. The imaging device 100 may be used for any purpose, for example, as a surveillance camera set up at a predetermined location inside or outside a facility.
[0013] As shown in FIG. 1, the imaging device 100 includes an optical element 10, an imaging element 12, an image processing circuit 13, an object position measurement unit 14, a self-position measurement unit 15, an input unit 16, a display unit 18, a communication unit 20, a storage unit 22, and a control unit 24.
[0014] The optical element 10 is an element of an optical system such as a lens. The optical element 10 may be one or a plurality of elements.
[0015] The imaging element 12 is an element that converts the light incident through the optical element 10 into an image signal which is an electrical signal. The imaging element 12 is, for example, a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Metal Oxide Semiconductor) sensor, or the like.
[0016] The image processing circuit 13 generates image data for each frame from the image signal generated by the imaging element 12. The image data is data including, for example, the luminance and color information of each pixel in one frame, and may be data to which gradation for each pixel is assigned.
[0017] The object position measurement unit 14 is a sensor that measures the position of an object to be measured with respect to the imaging device 100 (the relative position of the object). The object here may be any object, which may be a living thing or an inanimate object, and the same applies hereinafter. Also, the object here may refer to a movable object, but is not limited thereto and may refer to an immovable object.
[0018] In this embodiment, the object position measuring unit 14 measures the distance from the imaging device 100 to the object as the relative position of the object. The object position measuring unit 14 may be any sensor capable of measuring the relative position of an object, but for example, it may be a TOF (Time Of Flight) sensor. If the object position measuring unit 14 is a TOF sensor, for example, it is provided with a light-emitting element (e.g., an LED (Light Emitting Diode)) that emits light and a light-receiving unit that receives light, and the distance to the object is measured by the time of flight of the light that is emitted from the light-emitting element to the object and returns to the light-receiving unit. In addition to measuring the distance from the imaging device 100 to the object as the relative position of the object, the object position measuring unit 14 may also measure, for example, the direction in which the object is located relative to the imaging device 100. In other words, the object position measuring unit 14 may measure the position (coordinates) of the object in a coordinate system with the imaging device 100 as the origin as the relative position of the object.
[0019] The self-position measurement unit 15 is a sensor that measures the position (i.e., self-position) of the imaging device 100. In this embodiment, the self-position measurement unit 15 measures the position (coordinates) and orientation (direction) of the imaging device 100. The self-position measurement unit 15 may be any sensor capable of measuring the position and orientation of the imaging device 100, but it may be, for example, a three-dimensional acceleration sensor that measures the acceleration in the three axes of the imaging device 100, or a gyro sensor. For example, by measuring the acceleration in the three axes of the imaging device 100, the self-position measurement unit 15 can measure the position and orientation of the imaging device 100, that is, the position and orientation of the imaging device 100 after movement relative to the position and orientation of the imaging device 100 before movement. However, the self-position measurement unit 15 is not limited to measuring both the position and orientation of the imaging device 100, but may measure at least one of the position and orientation of the imaging device 100.
[0020] The input unit 16 is a mechanism that receives input (operation) from the user, and may be, for example, a button, a keyboard, a touch panel, etc.
[0021] The display unit 18 is a display panel that displays images. In addition to the images captured by the imaging device 100, the display unit 18 may also display images for the user to set a target region AR, which will be described later.
[0022] The communication unit 20 is a communication module that communicates with an external device, and may be, for example, an antenna or a Wi-Fi (registered trademark) module. The imaging device 100 communicates with an external device by wireless communication, but may also use wired communication, and the communication method may be arbitrary.
[0023] The storage unit 22 is a memory that stores various information such as the captured image data, the calculation content of the control unit 24, and programs, and includes, for example, at least one of a main storage device such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive). The program for the control unit 24 stored in the storage unit 22 may be stored in a recording medium readable by the imaging device 100.
[0024] The control unit 24 is an arithmetic device and includes an arithmetic circuit such as a CPU (Central Processing Unit). The control unit 24 includes a self-position acquisition unit 28, a target region acquisition unit 30, an object information acquisition unit 32, a focus position control unit 34, an imaging control unit 36, and an image acquisition unit 38. The control unit 24 reads and executes a program (software) from the storage unit 22 to realize the self-position acquisition unit 28, the target region acquisition unit 30, the object information acquisition unit 32, the focus position control unit 34, the imaging control unit 36, and the image acquisition unit 38, and executes their processes. Note that the control unit 24 may execute these processes by one CPU, or may include a plurality of CPUs and execute the processes with these plurality of CPUs. Also, at least a part of the processes of the self-position acquisition unit 28, the target region acquisition unit 30, the object information acquisition unit 32, the focus position control unit 34, the imaging control unit 36, and the image acquisition unit 38 may be realized by a hardware circuit.
[0025] (Self-Position Acquisition Unit) The self-position acquisition unit 28 acquires position information of the imaging device 100. Position information of the imaging device 100 refers to information indicating the position (coordinates) and orientation (direction) of the imaging device 100. The self-position acquisition unit 28 controls the self-position measurement unit 15 to have the self-position measurement unit 15 measure the position and orientation of the imaging device 100. The self-position acquisition unit 28 acquires the measurement results of the position and orientation of the imaging device 100 by the self-position measurement unit 15 as position information of the imaging device 100. The object information acquisition unit 32 acquires position information of the imaging device 100 sequentially by acquiring position information of the imaging device 100 at predetermined time intervals. Note that the position information of the imaging device 100 is not limited to both the position and orientation of the imaging device 100, but may be information indicating at least one of the position and orientation of the imaging device 100.
[0026] The self-position acquisition unit 28 determines whether the imaging device 100 has moved based on the position information of the imaging device 100. The self-position acquisition unit 28 determines that the imaging device 100 has moved if the position information of the imaging device 100 has changed, and determines that the imaging device 100 has not moved if the position information of the imaging device 100 has not changed. A change in the position information of the imaging device 100 means that the difference between the position information of the imaging device 100 acquired immediately before (in this case, at least one of position and orientation) and the position information of the imaging device 100 acquired this time (in this case, at least one of position and orientation) is greater than or equal to a predetermined value.
[0027] The self-position acquisition unit 28 may acquire the degree of movement of the imaging device 100 if it determines that the imaging device 100 has moved. Here, the degree of movement of the imaging device 100 refers to the direction and amount of movement of the position (coordinates) and orientation (direction) of the imaging device 100. The self-position acquisition unit 28 may calculate the degree of movement of the imaging device 100 by determining the direction and amount of change when the position and orientation of the imaging device 100 changes from the position and orientation of the imaging device 100 acquired immediately before to the position and orientation of the imaging device 100 acquired this time.
[0028] (Target area acquisition unit) (Acquiring the target area) The target area acquisition unit 30 acquires information on the target area AR set within the imaging area of the imaging device 100. The target area AR is an area set to automatically adjust the focal position. The information on the target area AR is information indicating the position of the target area AR, that is, the position information of the target area AR. The target area AR will be explained below.
[0029] Figures 2 and 3 are schematic diagrams illustrating an example of a target area. Figure 2 is a view of the imaging device 100 and the target area AR from above in the vertical direction, and Figure 3 is a view of the imaging device 100 and the target area AR from the horizontal direction. Hereinafter, direction Z will be defined as the vertical direction, direction X as one horizontal direction perpendicular to direction Z, and direction Y as a direction perpendicular to both direction Z and direction X (horizontal direction). As shown in Figures 2 and 3, the range in which an image can be captured by the imaging device 100 is defined as the imaging area AR0. The imaging area AR0 refers to the area (space) that falls within the field of view of the image sensor 12, or in other words, the range in real space that is captured as an image. The target area AR is the area (space) set within the range of the imaging area AR0.
[0030] More specifically, the target region AR is the region within the imaging region AR0, between the first position AX1 and the second position AX2. The first position AX1 is the position at which the distance from the imaging device 100 is the first distance L1, and the second position AX2 is the position at which the distance from the imaging device 100 is the second distance L2, which is shorter than the first distance L1. As shown in Figures 2 and 3, in this embodiment, the first position AX1 can be said to be a virtual plane within the imaging region AR0 that includes each position (coordinate) at which the distance from the imaging device 100 is the first distance L1. Similarly, the second position AX2 can be said to be a virtual plane within the imaging region AR0 that includes each position (coordinate) at which the distance from the imaging device 100 is the second distance L2. In other words, the target region AR can be said to be a space within the imaging region AR0 enclosed by a virtual plane at which the distance from the imaging device 100 is the second distance L2 and a virtual plane at which the distance from the imaging device 100 is the first distance L1. Furthermore, the first position AX1 is not limited to a virtual plane where all positions (coordinates) included in the first position AX1 are at a first distance L1 from the imaging device 100; at least some of the positions (coordinates) included in the first position AX1 may be at a first distance L1 from the imaging device 100. Similarly, the second position AX2 may be a virtual plane where at least some of the positions (coordinates) included in the second position AX2 are at a second distance L2 from the imaging device 100.
[0031] Figures 4 and 5 are schematic diagrams illustrating other examples of the target region. In the explanation of Figures 2 and 3, the target region AR was demarcated from the imaging region AR0 by the first position AX1 and the second position AX2 in the direction of the optical axis of the imaging device 100 (the depth direction of the image), but it was not demarcated from the imaging region AR0 in the direction of radiation (the direction of field of view expansion) relative to the optical axis of the imaging device 100. In other words, the end face of the target region AR in the direction of field of view expansion coincides with the end face of the imaging region AR0 in the direction of field of view expansion. However, it is not limited to this, and the target region AR may also be demarcated from the imaging region AR0 in the direction of field of view expansion. That is, for example, as shown in Figures 4 and 5, the target region AR may also be demarcated from the imaging region AR0 in the direction of field of view expansion by the third position AX3. In this example, the third position AX3 is a virtual surface (here, a closed curved surface of the side shape of a cylinder) that includes a position (coordinate) located at a predetermined distance outward in the radial direction with respect to the optical axis LX of the imaging device 100. In this case, the target region AR is the region (space) enclosed by the first position AX1, the second position AX2, and the third position AX3. Note that the third position AX3 is not limited to a virtual plane where all positions (coordinates) included in the third position AX3 are at a first distance L1 from the optical axis LX, but may be a virtual plane where at least some positions (coordinates) included in the third position AX3 are at a third distance L3 from the optical axis LX. For example, the third position AX3 may be a virtual plane that extends outward in the radial direction (horizontal and elevation directions) at a predetermined angle as it moves away from the imaging device 100 along the optical axis direction.
[0032] The size and shape of the target area AR are not limited to those described above and may be arbitrary. Similarly, the position of the target area AR is not limited to those described above and may be arbitrary. For example, the target area AR is not limited to being located between the first position AX1 and the second position AX2. Furthermore, while the target area AR was described above as an area set within the imaging area AR0, it is not limited to that. For example, if the range within which the object position measurement unit 14 can measure distance is defined as the distance measurement area (distance measurement space), then the target area AR may be an area set within the distance measurement area. In this case, the imaging area AR0 in Figures 2 to 5 may be treated as the distance measurement area.
[0033] The target area acquisition unit 30 may acquire information about the target area AR by any method. For example, the position of the target area AR may be set in advance. In this case, the target area acquisition unit 30 may read the pre-set position information of the target area AR from the storage unit 22, or it may acquire the position information of the target area AR from another device via the communication unit 20. Also, for example, if the position of the target area AR is not set in advance, the target area acquisition unit 30 may automatically set the position of the target area AR. Also, for example, the user may set the position of the target area AR. In this case, for example, the user may input information specifying the position of the target area AR (for example, the values of the first distance L1, second distance L2, third distance L3, etc.) into the input unit 16, and the target area acquisition unit 30 may set the target area AR based on the position information of the target area AR specified by the user. Also, for example, the target area AR may be set by specifying coordinates. In other words, for example in the example in Figure 2, coordinates P1, P2, P3, and P4, which are the vertex positions of the target region AR, may be specified, and the region enclosed by coordinates P1 to P4 may be set as the target region AR.
[0034] (Target area when the imaging device moves) Here, it is required that the target area AR be appropriately set even when the imaging device 100 moves. In response to this, the target area acquisition unit 30 fixes the position of the target area AR and prevents it from moving when the self-position acquisition unit 28 determines that the imaging device 100 has moved. This fixes the target area AR and prevents the area of interest from unintentionally changing due to the movement of the imaging device 100.
[0035] Furthermore, in this embodiment, the target area acquisition unit 30 acquires mode information indicating whether or not to move the target area AR, and when the self-position acquisition unit 28 determines that the imaging device 100 has moved, it determines whether or not to move the target area AR based on the mode information. This will be explained in detail below.
[0036] The target area acquisition unit 30 acquires mode information. Mode information indicates whether or not the target area AR should move when the imaging device 100 moves. The mode information is assigned either information indicating that it is the first mode or information indicating that it is the second mode. The first mode is a mode in which the target area AR does not move when the imaging device 100 moves (the position of the target area AR is fixed), and the second mode is a mode in which the target area AR moves when the imaging device 100 moves.
[0037] Figure 6 is a schematic diagram showing an example of the target area when set to the first mode. When the target area acquisition unit 30 acquires mode information indicating the first mode, it sets the target area AR to the first mode, which fixes the target area AR, and does not move the target area AR even when the self-position acquisition unit 28 determines that the imaging device 100 has moved. In other words, in the first mode, the position of the target area AR is fixed regardless of the position of the imaging device 100. Figure 6 shows an example where the imaging device 100 moves from position 100a to position 100b. In this case, the position of the imaging area AR0 moves from position AR0a to position AR0b. However, when set to the first mode, the position of the target area AR remains fixed and does not move. Note that since the distance measurement area also moves along with the movement of the imaging device 100, it can also be said that the distance measurement area moves from position AR0a to position AR0b.
[0038] Figure 7 is a schematic diagram showing an example of a target area when set to the second mode. When the target area acquisition unit 30 acquires mode information indicating the second mode, it sets to the second mode which allows movement of the target area AR, and when the self-position acquisition unit 28 determines that the imaging device 100 has moved, it moves the target area AR. In this case, it is preferable for the target area acquisition unit 30 to move the target area AR so that the shape and size of the target area AR remain the same. Furthermore, it is preferable for the target area acquisition unit 30 to move the target area AR so that the position (relative position) of the target area AR with respect to the imaging device 100 remains the same, based on the degree of movement of the imaging device 100. That is, it is preferable not to change the position (relative position) of the target area AR with respect to the imaging device 100 before and after the movement of the target area AR. In this case, for example, the target area acquisition unit 30 moves the target area AR by the same amount as the degree of movement of the imaging device 100. Figure 7 shows an example where the imaging device 100 moves from position 100a to position 100b. In this case, the position of the imaging region AR0 moves from position AR0a to position AR0b. When set to the second mode, the position of the target region AR moves from position ARa to position ARb. That is, when the imaging device 100 is at position 100a, the target region AR is at position ARa, and when the imaging device 100 moves to position 100b, the target region AR moves to position ARb.
[0039] Furthermore, if the imaging device 100 does not move, it is preferable that the target area acquisition unit 30 does not move the position of the target area AR, whether in the first mode or the second mode.
[0040] The target area acquisition unit 30 may acquire mode information by any method. For example, mode information, i.e., whether to use the first mode or the second mode, may be set in advance. In this case, the target area acquisition unit 30 may read the pre-set mode information from the storage unit 22, or it may acquire mode information from another device via the communication unit 20. Also, for example, if the mode information is not set in advance, the target area acquisition unit 30 may set the mode information automatically. Also, for example, the user may set the mode information. In this case, for example, the user may input information specifying the mode (information specifying whether to use the first mode or the second mode) into the input unit 16, and the target area acquisition unit 30 may set the mode based on the mode information specified by the user.
[0041] The target area acquisition unit 30 may switch between the first mode and the second mode. In this case, once the target area acquisition unit 30 acquires mode information indicating that a mode should be switched, it may switch modes based on that mode information.
[0042] The flow for setting the target area AR when the imaging device 100 moves, as described above, will now be explained based on a flowchart. Figure 8 is a flowchart illustrating the flow for setting the target area when the imaging device moves. As shown in Figure 8, the control unit 24 acquires mode information and target area AR information from the target area acquisition unit 30 (step S10), sets the mode based on the mode information, and sets the target area AR based on the target area AR information. Then, the control unit 24 determines whether the imaging device 100 has moved using the self-position acquisition unit 28 (step S12). The self-position acquisition unit 28 determines whether the imaging device 100 has moved based on the position information of the imaging device 100. If it is determined that the imaging device 100 has moved (step S12; Yes) and the second mode is set (step S14; Yes), the target area acquisition unit 30 moves the target area AR (step S16). After that, if the process is not to be terminated (step S18; No), the process returns to step S12, and if the process is to be terminated (step S18; Yes), this process is terminated. On the other hand, if it is determined that the imaging device 100 does not move (step S12; No), the process proceeds to step S18 without moving the target area AR. Also, if it is determined that the imaging device 100 has moved and the system is set to the second mode (step S14; No), the process proceeds to step S18 without moving the target area AR.
[0043] In the first embodiment, it is not essential to set the system to either the first mode or the second mode as described above; it is sufficient that the system can be configured so that the position of the target area AR does not move even when the imaging device 100 moves.
[0044] (Object information acquisition unit) The object information acquisition unit 32 acquires positional information of objects located within the imaging area AR0. The object information acquisition unit 32 controls the object position measurement unit 14 to measure the relative position of the object with respect to the imaging device 100. The object information acquisition unit 32 acquires the measurement result of the relative position of the object with respect to the imaging device 100, obtained by the object position measurement unit 14, as the object's positional information. The object information acquisition unit 32 acquires the object's positional information sequentially by acquiring the object's positional information at predetermined time intervals. In addition, the object information acquisition unit 32 can also acquire information indicating the shape of the object (for example, the object's 3D shape) based on the object's positional information. For example, the object information acquisition unit 32 can acquire the object's 3D shape by accumulating multiple pieces of positional information, such as TOF image information.
[0045] (Focus position control unit) The focus position control unit 34 sets the focal position of the imaging device 100. The focus position control unit 34 controls the focal position by controlling the position of the optical element 10, that is, by moving the position of the optical element 10.
[0046] The focus position control unit 34 adjusts the focus position to an object that exists within the target area AR. In other words, the focus position control unit 34 sets the focus position to the position of an object that is determined to exist within the target area AR. In this embodiment, the focus position control unit 34 determines whether an object exists within the target area AR based on the object's position information acquired by the object information acquisition unit 32. If the position of the object acquired by the object information acquisition unit 32 coincides with the position of the target area AR, the focus position control unit 34 determines that the object exists within the target area AR and adjusts the focus position to the position of the object acquired by the object information acquisition unit 32. That is, for example, if the distance from the imaging device 100 to the object is less than or equal to the first distance L1 and greater than or equal to the second distance L2, the focus position control unit 34 determines that the object exists within the target area AR and adjusts the focus position to that object. On the other hand, the focus position control unit 34 does not adjust the focus position to an object that does not exist within the target area AR. In other words, for example, if the distance from the imaging device 100 to the object is greater than the first distance L1 or closer than the second distance L2, the focus position control unit 34 determines that the object is not within the target area AR and does not focus on that object.
[0047] The focus position control unit 34 maintains its focus on the object as long as the object is within the target area AR. That is, based on the object's position information acquired by the object information acquisition unit 32 at predetermined time intervals, the focus position control unit 34 determines whether the object continues to be within the target area AR, and maintains its focus on the object as long as the object continues to be within the target area AR. On the other hand, if the object that has been focused moves outside the target area AR, that is, if it is no longer within the target area AR, the focus position control unit 34 shifts its focus away from the object and focuses on a location other than the object.
[0048] Furthermore, the focus position control unit 34 does not need to focus on objects that are already within the target area AR from the time the imaging device 100 starts operating (the moment it becomes ready to image). In other words, the focus position control unit 34 may focus on objects that enter the target area AR after the start of operation. To put it another way, the focus position control unit 34 may focus on objects that are within the target area AR at a certain point in time but were not within the target area AR at a time prior to that point, starting from the moment they begin to enter the target area AR. To put it another way, the focus position control unit 34 may recognize an object as a target to focus on when it moves from outside the target area AR into the target area AR. In other words, the focus position control unit 34 may focus on objects that have moved from outside the target area AR into the target area AR.
[0049] Furthermore, if no object is present within the target area AR, the focus position control unit 34 may adjust the focus position to a pre-set position. The set position can be set arbitrarily, but it is preferable that it be set within the target area AR, such as the center position of the target area AR.
[0050] Furthermore, it is preferable that the focus position control unit 34 does not focus on stationary objects but focuses on moving objects. More specifically, when the focus position control unit 34 is set to the second mode, if a stationary (non-moving) object comes into the target area AR due to the movement of the imaging device 100, the focus position control unit 34 does not focus on that object. In other words, the focus position control unit 34 does not treat a stationary object as an object to focus on, even if that object is located within the target area AR, and does not focus on that object. On the other hand, the focus position control unit 34 focuses on a moving object when that object is located within the target area AR, that is, when a moving object reaches the target area AR. Whether an object is moving can be determined based on the object's position information obtained by the object information acquisition unit 32. In other words, if the position information of an object changes continuously over time, it can be determined that the object is moving.
[0051] The example of setting the focal position described above will now be explained with reference to Figure 2. Figure 2 shows an example where object A moves from position A0 to position A3, passing through positions A1 and A2, towards the imaging device 100. Position A0 is farther from the imaging device 100 than the first distance L1 and is outside the target area AR. Positions A1 and A2 are within the target area AR because their distance to the imaging device 100 is less than or equal to the first distance L1 and greater than or equal to the second distance L2. Position A3 is closer to the imaging device 100 than the second distance L2 and is outside the target area AR. In this case, when object A is at position A0, the focal position control unit 34 does not focus on object A, but focuses on a set position, for example. Then, when object A is at position A1, that is, when object A enters the target area AR, the focal position control unit 34 focuses on object A. The focus position control unit 34 maintains focus on object A even when object A is at position A2, and when object A moves to position A3, that is, when object A leaves the target area AR, it defocuses from object A and returns the focus position to the set position. In other words, from the moment object A enters the target area AR, the focus position control unit 34 focuses on object A, moves the focus position in accordance with the moving object A while object A is moving within the target area AR, and defocuses from object A when object A moves outside the target area AR.
[0052] The focal position may be set by the user. In this case, for example, it may be possible to switch between an auto mode in which the focal position is set automatically and a manual mode in which the user sets the focal position. In auto mode, the focal position is set by the focal position control unit 34 as described above. On the other hand, in manual mode, the user inputs an operation to set the focal position to the input unit 16, and the focal position control unit 34 sets the focal position according to the user's operation.
[0053] (Image Control Unit) The imaging control unit 36 controls the imaging device 100 to capture an image. The imaging control unit 36 controls, for example, the image sensor 12 to acquire an image signal. For example, the imaging control unit 36 may have the image sensor 12 acquire an image signal automatically, or it may have the image signal acquired in response to user operation.
[0054] (Image acquisition unit) The image acquisition unit 38 acquires image data acquired by the image sensor 12. The image acquisition unit 38 controls, for example, the image processing circuit 13 to cause the image processing circuit 13 to generate image data from the image signal generated by the image sensor 12, and acquires that image data. The image acquisition unit 38 stores the image data in the storage unit 22.
[0055] (Focus position setting flow) Next, the processing flow for setting the focal position described above will be explained. Figure 9 is a flowchart illustrating the processing flow for setting the focal position. As shown in Figure 9, the control unit 24 acquires information on the target area AR using the target area acquisition unit 30 (step S20), and acquires the position information of the object using the object information acquisition unit 32 (step S22). The order in which steps S20 and S22 are performed is arbitrary. The control unit 24, using the focal position control unit 34, determines whether the object is located within the target area AR based on the object's position information (step S24). If the object is not located within the target area AR (step S24; No), the process returns to step S22 and continues to acquire the object's position information. On the other hand, if the object is located within the target area AR (step S24; Yes), the focal position control unit 34 adjusts the focal position to that object (step S26). After that, the process continues to acquire the object's position information and determines whether the object has moved outside the target area AR (step S28). If the object does not move outside the target area AR (step S28; No), that is, if the object remains within the target area AR, the process returns to step S26 and the focus position is maintained on the object. If the object moves outside the target area AR (step S28; Yes), the focus position control unit 34 moves the focus position away from the object (step S30). If the process is not terminated thereafter (step S32; No), the process returns to step S22, and if the process is terminated (step S32; Yes), this process is terminated.
[0056] (effect) As described above, the imaging device 100 according to this embodiment includes an image sensor 12, a self-position acquisition unit 28, an object information acquisition unit 32, a target area acquisition unit 30, and a focus position control unit 34. The self-position acquisition unit 28 acquires position information of the imaging device 100, the object information acquisition unit 32 acquires position information of objects present in the imaging area AR0 of the image sensor 12, the target area acquisition unit 30 sets the target area AR within the imaging area AR0, and the focus position control unit 34 controls the focus position of the imaging device 100 to focus on an object present in the target area AR. The target area acquisition unit 30 fixes the position of the target area AR when the self-position acquisition unit 28 determines that the imaging device 100 has moved.
[0057] In autofocus imaging devices, it is necessary to appropriately adjust the focal position. In contrast, the imaging device 100 according to this embodiment controls the focal position of the imaging device 100 so that when an object exists within the target area AR, the focal position of the imaging device 100 is adjusted to that object. Furthermore, even if the imaging device 100 moves, the position of the target area AR is fixed. Therefore, according to this embodiment, even if the imaging area AR0 moves, the position of the target area AR can be fixed, so that the focal position can be appropriately adjusted to an object within the area of interest while changing the imaging area AR0.
[0058] Furthermore, when the target area acquisition unit 30 is set to the first mode, which fixes the position of the target area AR, it fixes the position of the target area AR when the self-position acquisition unit 28 determines that the imaging device 100 has moved. On the other hand, when the target area acquisition unit 30 is set to the second mode, which does not fix the position of the target area AR, it changes the position of the target area AR when the self-position acquisition unit 28 determines that the imaging device 100 has moved. Therefore, according to this embodiment, it is possible to set whether to change or fix the area of interest in accordance with the imaging area AR0 depending on the situation, so that the focal position can be appropriately adjusted.
[0059] Furthermore, the target area acquisition unit 30 may set the target area AR such that it is located between a first position AX1, where the distance from the imaging device 100 is a first distance L1, and a second position AX2, where the distance from the imaging device 100 is a second distance L2, which is shorter than the first distance L1. Therefore, the focal position can be appropriately adjusted for objects that enter the target area AR.
[0060] (Second Embodiment) Next, a second embodiment will be described. The second embodiment differs from the first embodiment in that, while the position of the target area AR is fixed, the imaging device 100A moves, and when the distance between the boundary of the imaging area AR0 and the target area AR falls below a predetermined distance, an alarm is issued. In the second embodiment, parts that have the same configuration as the first embodiment will not be described.
[0061] Figure 10 is a schematic block diagram of the imaging device according to the second embodiment. As shown in Figure 10, the imaging device 100A according to the second embodiment includes a notification control unit 40 in the control unit 24.
[0062] In the second embodiment, the notification control unit 40 calculates distance D when the position of the target area AR does not move even when the imaging device 100A moves (i.e., when set to the first mode). Distance D is the distance between the boundary B of the imaging area AR0 and the target area AR. The boundary B of the imaging area AR0 refers to the boundary position between the inside and outside of the imaging area AR0, or in other words, the periphery of the imaging area AR0. The notification control unit 40 calculates distance D as the shortest distance between the boundary B of the imaging area AR0 and the target area AR when the target area AR is located within the imaging area AR0. That is, the notification control unit 40 considers distance D to be the length of the shortest straight line connecting each point on the periphery of the target area AR to each point on the boundary B of the imaging area AR0. The distance D may be calculated by any method, but for example, the notification control unit 40 may pre-determine the position of the boundary B of the imaging area AR0 corresponding to the position of the imaging device 100A, and calculate the distance D from the boundary B and the position of the set target area AR.
[0063] The notification control unit 40 determines whether the distance D is less than a predetermined distance. That is, the notification control unit 40 determines whether the distance D has become less than a predetermined distance due to the movement of the imaging device 100A. The predetermined distance here may be set arbitrarily. If the distance D is less than the predetermined distance, the notification control unit 40 causes the imaging device 100A to output an alarm. The alarm here is information indicating that the distance D is less than a predetermined distance, and the content may be arbitrary. For example, the notification control unit 40 may display arbitrary information (e.g., characters or symbols indicating a warning) on the display unit 18 indicating that the distance D is less than a predetermined distance, or it may output arbitrary sound (e.g., an alarm) indicating that the distance D is less than a predetermined distance to an audio output unit (speaker) not shown provided on the imaging device 100A, or it may output arbitrary tactile stimuli (e.g., vibration) indicating that the distance D is less than a predetermined distance to a device (e.g., vibration) not shown provided on the imaging device 100A. On the other hand, the notification control unit 40 does not output an alarm if the distance D is not less than a predetermined distance, that is, if the distance D is greater than or equal to a predetermined distance.
[0064] The above process will be explained with reference to Figure 11. Figure 11 is a schematic diagram showing an example of a target area in the second embodiment. Figure 11 shows an example where the imaging device 100A moves from position 100Aa to position 100Ab. In this case, the position of the imaging area AR0 moves from position AR0a to position AR0b. In this example, since the system is set to the first mode, the position of the target area AR remains fixed and does not move. In this example, when the imaging device 100A is at position 100Aa, the distance Da from the boundary Ba of the imaging area AR0 to the target area AR is greater than or equal to a predetermined distance, and when the imaging device 100A is at position 100Ab, the distance Db from the boundary Bb of the imaging area AR0 to the target area AR is less than a predetermined distance. Therefore, the notification control unit 40 does not output an alarm when the imaging device 100A is at position 100Aa, and outputs an alarm when the imaging device 100B is at position 100Ab. Furthermore, as the imaging device 100 moves, the distance measurement area also moves, so the distance measurement area moves from position AR0a to position AR0b, and the boundary of the distance measurement area can also be referred to as boundary B.
[0065] In this way, by outputting an alarm when the distance D is less than a predetermined distance, it is possible to notify the user in advance of the risk that the target area AR may move outside the range of the imaging area AR0 due to, for example, the movement of the imaging device 100A. Therefore, it is possible to prevent the imaging device 100A from moving further and causing the target area AR to move outside the range of the imaging area AR0.
[0066] In the above description of the second embodiment, it was assumed that the system was set to either the first mode or the second mode, and that in the first mode, an alarm was issued when the distance between the boundary of the imaging area AR0 and the target area AR fell below a predetermined distance. However, in the second embodiment, it is not necessary to set the system to either the first mode or the second mode; it is sufficient that the system can be configured so that the position of the target area AR does not move even when the imaging device 100A moves. In other words, the imaging device 100A according to the second embodiment may be configured so that the position of the target area AR does not move even when the imaging device 100 moves, and an alarm is issued when the distance between the boundary of the imaging area AR0 and the target area AR falls below a predetermined distance.
[0067] (Alarm notification flow) Next, the alarm notification flow described above will be explained. Figure 12 is a flowchart illustrating the alarm notification flow. As shown in Figure 12, the control unit 24 acquires information on the target area AR using the target area acquisition unit 30 (step S40) and sets the target area AR. Then, the control unit 24 uses the self-position acquisition unit 28 to determine whether the imaging device 100A has moved (step S42). If it is determined that the imaging device 100A has moved (step S42; Yes), the control unit 24 uses the notification control unit 40 to determine whether the distance D between the target area AR and the boundary B is less than a predetermined distance (step S44). If the distance D between the target area AR and the boundary B is less than a predetermined distance, the notification control unit 40 outputs an alarm (step S46). After that, if the process is not to be terminated (step S48; No), the process returns to step S42, and if the process is to be terminated (step S48; Yes), this process is terminated. On the other hand, if it is determined that the imaging device 100A has not moved (step S42; No), the process also proceeds to step S48. Furthermore, if it is determined that the imaging device 100A has moved, and the distance D between the target area AR and the boundary B is not less than a predetermined distance (step S44; No), that is, if the distance D is greater than or equal to the predetermined distance, the process proceeds to step S48 without moving the target area AR.
[0068] As described above, in the second embodiment, when the imaging device 100A moves, causing the imaging area AR0 to move, and the distance D from the target area AR to the boundary B between the inside and outside of the imaging area AR0 falls below a predetermined distance, the notification control unit 40 outputs an alarm. By outputting an alarm when the distance D is less than a predetermined distance, it is possible to notify the user in advance that there is a risk that the target area AR may move outside the range of the imaging area AR0 due to the movement of the imaging device 100A. Therefore, it is possible to suppress the imaging device 100A from moving further and causing the target area AR to move outside the range of the imaging area AR0.
[0069] Furthermore, if the imaging device 100A is automatically moved by a moving mechanism (not shown), the control unit 24 may control the moving mechanism to stop the movement of the imaging device 100A in the direction that further shortens the distance D when the distance D falls below a predetermined distance. This movement stopping process may be performed together with the output of an alarm, or it may be performed instead of the output of an alarm.
[0070] (Third embodiment) Next, a third embodiment will be described. The third embodiment differs from the first embodiment in that it focuses on an object that is located within the target area AR and satisfies predetermined conditions. Parts of the third embodiment that are common with the first embodiment will not be described. The third embodiment is also applicable to the second embodiment.
[0071] In the third embodiment, the focus position control unit 34 focuses on an object that is located within the target area AR and satisfies predetermined conditions. The focus position control unit 34 does not focus on an object that does not satisfy at least one of the conditions of being located within the target area AR and satisfying predetermined conditions. The focus position control unit 34 maintains focus on the object for the duration that the object remains located within the target area AR while satisfying predetermined conditions. On the other hand, if the object no longer satisfies at least one of the conditions of being located within the target area AR and satisfying predetermined conditions, the focus position control unit 34 defocuses from that object. That is, for example, if the object satisfies predetermined conditions but moves outside the target area AR, or if the object is located within the target area AR but no longer satisfies predetermined conditions, the focus position control unit 34 defocuses from that object.
[0072] The focus position control unit 34 may determine whether predetermined conditions are met by any method, for example, by determining whether predetermined conditions are met based on at least one of the object's position information and the object's image. Here, the object's position information may refer to the measurement result of the object position measuring unit 14, and the object's image may refer to image data of the object acquired by the image sensor 12.
[0073] The predetermined conditions here can be any conditions other than the object being within the target area AR. For example, the predetermined conditions may be at least one of the following: the object is performing a predetermined motion, the object has a predetermined shape, and the object is facing a predetermined direction. Alternatively, any two of these may be predetermined conditions, or all of them may be predetermined conditions. If multiple predetermined conditions are set, the focus position control unit 34 determines that the predetermined conditions are met only when all of the conditions are met.
[0074] The following describes the case where the motion of an object is subject to predetermined conditions. In this case, the focus position control unit 34 determines whether an object is performing the predetermined motion based on the position information of the object acquired continuously in a time series. The focus position control unit 34 focuses on objects that are located within the target area AR and are performing the predetermined motion. The focus position control unit 34 does not focus on objects that do not satisfy at least one of the conditions of being located within the target area AR and performing the predetermined motion. The focus position control unit 34 continues to focus on an object for the duration that the object is located within the target area AR and continues to perform the predetermined motion. On the other hand, if the object no longer satisfies at least one of the conditions of being located within the target area AR and performing the predetermined motion, the focus position control unit 34 removes the focus from that object. Note that the motion of an object here refers to the manner of movement of the object, and may refer to, for example, the direction and speed of movement of the object. For example, if a predetermined motion refers to movement in the vertical downward direction at a speed of 10 m / h or more, the focus position control unit 34 adjusts the focus position to the object moving in the vertical downward direction at a speed of 10 m / h or more within the target area AR. Note that the motion of an object is not limited to the direction and speed of movement of the object, but may refer to any manner of movement. For example, the motion of an object may refer to at least one of the direction and speed of movement of the object.
[0075] Figure 13 is a schematic diagram illustrating an example where the motion of an object is subject to predetermined conditions. In the example in Figure 13, the predetermined condition is that the object moves vertically downward (opposite to the Z direction), i.e., the direction of the object's movement. In the example in Figure 13, object A moves vertically downward from position A0a, through positions A1a and A2a to position A3a, and stops at position A3a. Position A0a is outside the target region AR, while positions A1a, A2a, and A3a are inside the target region AR. In this case, when object A is at position A0a, the focus position control unit 34 does not focus on object A because object A is outside the target region AR, but instead focuses on a set position, for example. Then, when object A is at position A1a, i.e., when object A enters the target region AR while moving vertically downward, the focus position control unit 34 focuses on object A. The focus position control unit 34 continues to focus on object A even when object A is at position A2a, and when object A moves to position A3a and stops, it moves the focus position away from object A and returns the focus position to the set position.
[0076] Next, we will explain the case where the shape of an object is subject to predetermined conditions. In this case, the focus position control unit 34 determines whether an object has the predetermined shape based on the image data in which the object is captured. It focuses on objects that are located within the target area AR and have the predetermined shape. The focus position control unit 34 does not focus on objects that do not satisfy at least one of the conditions of being located within the target area AR and having the predetermined shape. The focus position control unit 34 continues to focus on an object as long as that object has the predetermined shape and remains located within the target area AR. On the other hand, if the object no longer satisfies at least one of the conditions of being located within the target area AR and having the predetermined shape, the focus position control unit 34 defocuses from that object. The shape of an object here may be, for example, at least one of the size of the object and the external shape of the object. For example, if the predetermined shape refers to being larger than or equal to a predetermined size, the focus position control unit 34 focuses on objects of a predetermined size or larger that are located within the target area AR. Furthermore, the 3D shape information acquired by the object information acquisition unit 32 may be used to acquire the shape information of the object.
[0077] The following describes the case where the orientation of an object is subject to predetermined conditions. In this case, the focus position control unit 34 determines whether the object is facing the predetermined direction based on the image data in which the object is captured. It focuses on objects that are located within the target area AR and facing the predetermined direction. The focus position control unit 34 does not focus on objects that do not satisfy at least one of the conditions of being located within the target area AR and facing the predetermined direction. The focus position control unit 34 continues to focus on the object for as long as the object it has focused on remains located within the target area AR while facing the predetermined direction. On the other hand, if the object no longer satisfies at least one of the conditions of being located within the target area AR and facing the predetermined direction, the focus position control unit 34 removes the focus from that object. Note that the 3D shape information acquired by the object information acquisition unit 32 may be used to acquire information on the orientation of the object.
[0078] The predetermined conditions may be set in any way, for example, they may be set in advance. In this case, the focus position control unit 34 may read information indicating the predetermined conditions (e.g., direction of movement and speed of movement) from the storage unit 22, or it may obtain the predetermined conditions from other devices via the communication unit 20. Also, for example, if the predetermined conditions are not set in advance, the focus position control unit 34 may set the predetermined conditions automatically. Also, for example, a user may set the predetermined conditions. In this case, for example, the user may input information specifying the predetermined conditions (e.g., direction of movement and speed of movement) into the input unit 16, and the focus position control unit 34 may set the predetermined conditions based on the information specified by the user.
[0079] As described above, in the third embodiment, the focus position control unit 34 may focus on an object that is in the target area AR and is performing a predetermined motion. The focus position control unit 34 continues to focus on the object while the object is performing the predetermined motion, and when the object stops performing the predetermined motion, it moves the focus away from the object. In this way, by making the fulfillment of a predetermined motion, in addition to being in the target area AR, a condition for focusing the position, it becomes possible to track an object performing a specific movement and appropriately adjust the focus position. For example, it becomes possible to detect a fall within the target area AR.
[0080] In the third embodiment, the focus position control unit 34 may focus on an object that is located in the target area AR and has a predetermined shape. By making the condition for focusing the object a predetermined shape in addition to being located in the target area AR, it becomes possible to track an object of a specific shape and appropriately adjust the focus position.
[0081] In the third embodiment, the focus position control unit 34 may focus on an object that is located in the target area AR and facing a predetermined direction. By making it a condition for focusing that the object is facing a predetermined direction in addition to being located in the target area AR, it becomes possible to track an object facing a specific direction and appropriately adjust the focus position.
[0082] The embodiments of the present invention have been described above, but the embodiments are not limited by the contents of these embodiments. Further, the above-described components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within a so-called equivalent range. Furthermore, the above-described components can be combined as appropriate, and the configurations of each embodiment can also be combined. Furthermore, various omissions, substitutions, or changes of the components can be made without departing from the gist of the above-described embodiments. Also, in each embodiment, the operation of adjusting the focus position has been described as a feature point, but the operation of adjusting the focus position and other operations may be combined. For example, the operation of adjusting the focus position and the operation of zooming in and out may be combined. Also, in the description of each embodiment, the operation of adjusting the focus position may be replaced with other operations. For example, in the description of each embodiment, the operation of adjusting the focus position may be replaced with the operation of zooming in and out. Also, the control unit 24 of the imaging device in each embodiment may, for example, notify a predetermined transmission destination through the communication unit 20 when a set condition such as an object entering or leaving a predetermined target region AR or the object moving in a predetermined direction is satisfied. The set condition here may, for example, refer to adjusting the focus position of the object triggered by the object moving into the target region AR.
Explanation of Reference Numerals
[0083] 10 Optical element 12 Image sensor 14 Object position measurement unit 15 Self-position measurement unit 28 Self-position acquisition unit 30 Target region acquisition unit 32 Object information acquisition unit 34 Focus position control unit 40 Notification control unit AR Target region AR0 Imaging region
Claims
1. An imaging device capable of imaging an object, Image sensor and A self-position acquisition unit that acquires positional information of the imaging device, An object information acquisition unit that acquires positional information of an object present in the imaging area of the image sensor, A target area acquisition unit that sets a target area within the aforementioned imaging area, A focal position control unit controls the focal position of the imaging device such that, when an object exists within the target area and the object is performing a predetermined motion, the focal position is set on the object, the focal position remains set on the object while the object is performing the predetermined motion, and when the object stops performing the predetermined motion, the focal position is moved away from the object. It has, The target area acquisition unit, when it is determined by the self-position acquisition unit that the imaging device has moved, will keep the position of the target area fixed. Imaging device.
2. The aforementioned target area acquisition unit, When set to the first mode for fixing the position of the target area, if the self-position acquisition unit determines that the imaging device has moved, the position of the target area is fixed. The imaging device according to claim 1, wherein, when set to a second mode in which the position of the target area is not fixed, the position of the target area is changed when the self-position acquisition unit determines that the imaging device has moved.
3. The imaging device according to claim 1 or claim 2, further comprising a notification control unit that outputs an alarm when the imaging area moves as the imaging device moves, and the distance from the target area to the boundary position between the inside and outside of the imaging area becomes less than a predetermined distance.
4. The imaging device according to any one of claims 1 to 3, wherein the target area acquisition unit sets the target area so that it is located between a first position where the distance from the imaging device is a first distance and a second position where the distance from the imaging device is a second distance shorter than the first distance.
5. An imaging method for imaging an object, A step to acquire the position information of the imaging device, A step of acquiring positional information of an object present in the imaging area of the image sensor, The steps include setting a target area within the aforementioned imaging area, The steps include controlling the focal position of the imaging device such that, if an object exists within the target area and the object is performing a predetermined motion, the focal position is set on the object, the focal position remains set on the object for the duration that the object is performing the predetermined motion, and when the object stops performing the predetermined motion, the focal position is moved away from the object; Includes, In the step of setting the target area, if it is determined that the imaging device has moved, the position of the target area is kept fixed. Imaging method.
6. A program that causes a computer to execute an imaging method for capturing images of an object, A step to acquire the position information of the imaging device, A step of acquiring positional information of an object present in the imaging area of the image sensor, The steps include setting a target area within the aforementioned imaging area, The steps include controlling the focal position of the imaging device such that, if an object exists within the target area and the object is performing a predetermined motion, the focal position is set on the object, the focal position remains set on the object for the duration that the object is performing the predetermined motion, and when the object stops performing the predetermined motion, the focal position is moved away from the object; The computer is made to execute the above, In the step of setting the target area, if it is determined that the imaging device has moved, the position of the target area is kept fixed. program.