Imaging assistance control device, imaging assistance control method, and imaging assistance system

Abstract

The imaging assistance control apparatus according to the present technology includes a control unit that performs control so that guidance of a change in composition by a user is performed via a tactile presentation to the user, based on a difference between a target composition and an actual composition.

Description

TECHNICAL FIELD

[0001] The present technology relates to the technical field of an imaging assistance control device and a method thereof that perform control so that guidance to a composition change operation by a user is performed, and an imaging assistance system including the imaging assistance control device. BACKGROUND

[0002] Imaging assistance technologies for assisting a user with image capturing using a video camera have been proposed. For example, Patent Literature 1 below discloses a technology that performs voice guidance to a composition change operation by a user so that an inappropriate composition is not produced.

[0003] LIST OF CITATIONS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Laid-Open No. 4135100 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] Here, imaging needs to be performed without looking at a video camera screen. For example, when imaging a pet or a child, imaging needs to be performed while looking with the naked eye rather than by watching a screen.

[0008] Further, among people with impaired vision, depending on the degree of impairment, there are people who want to perform imaging with a video camera. In this case, it is expected that the user cannot accurately grasp the position and size of a subject on a screen, and imaging becomes difficult.

[0009] In response to these requirements, performing voice guidance as in Patent Literature 1 is considered to be effective, but there are cases where voice guidance is not appropriate depending on the situation in which imaging is performed. For example, there are cases where the user cannot hear the guidance by voice in a noisy situation in an outdoor environment, or even in a noisy situation when imaging a running child indoors.

[0010] Further, in a situation where quietness is required, for example, in a situation of imaging a wild animal such as a wild bird, and the like, voice guidance is not appropriate. In addition, it is not desirable to record unnecessary voice when moving image capturing, and therefore voice guidance is not appropriate.

[0011] The present technology is made in view of the above-described situations, and an object thereof is to improve imaging assistance performance by being able to perform appropriate imaging assistance even in a situation where guidance by a display or sound is not appropriate.

[0012] SOLUTION TO THE PROBLEM

[0013] The imaging assistance control apparatus according to the present technology includes a control unit that performs control such that guidance of a composition change operation by the user is performed through a tactile presentation to the user, based on a difference between a target composition and an actual composition.

[0014] Therefore, even in a case where guidance through a display is inappropriate in a situation where the user cannot or is difficult to watch a screen, or even in a case where guidance through sound is inappropriate, it is possible to appropriately guide a composition change operation through a tactile presentation.

[0015] In the imaging assistance control apparatus according to the above-described present technology, it is conceivable to have a configuration in which the control unit changes a tactile presentation pattern for guidance based on a size of a difference between the target composition and the actual composition.

[0016] Therefore, it is possible to make the user grasp whether a difference between the target composition and the actual composition becomes smaller or larger according to a pattern of the tactile presentation for guidance.

[0017] In the imaging assistance control apparatus according to the above-described present technology, it is conceivable to have a configuration in which the target composition is set to a composition that satisfies a condition that a target subject is arranged at a position in an image frame that is set according to an operation of the user, an operation related to the setting of the arrangement position of the target subject in the image frame is performed as a drawing operation on a screen that displays a through image of a captured image, and the control unit determines a setting condition other than the arrangement position in the setting condition of the target composition according to which operation type the drawing operation belongs to, among a plurality of operation types.

[0018] The operation type of the drawing operation is obtained by classifying an operation pattern that can be performed as the drawing operation, and examples thereof can include types of an operation of drawing a point, an operation of drawing a circle, an operation of drawing an ellipse, and the like. Further, the setting condition of the target composition refers to a condition for determining a composition of the target composition, and examples thereof include an arrangement position of the target subject in the image frame and a condition for setting a single target subject or a plurality of target subjects.

[0019] According to the above-described configuration, the user can specify the arrangement position of the target subject through one operation as the drawing operation on the screen, and specify a setting condition other than the arrangement position in the setting condition of the target composition.

[0020] In the imaging assistance control apparatus according to the above-described present technology, it is conceivable to have a configuration in which the target composition is set to a composition that satisfies a condition that a target subject is arranged at a position in an image frame that is set according to an operation of the user, and the control unit receives a specification of the target subject through a voice input from the user.

[0021] Therefore, the user does not need to operate an operator such as a button or a touch panel in the designation of the target subject.

[0022] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the control unit receives a drawing operation on a screen that displays a through image of a captured image, and determines a condition of a camera that obtains the captured image, and in a case where the condition is a specific condition and a specific mark has been drawn by the drawing operation, sets a composition that satisfies a composition condition to a target composition, the composition condition being associated with the condition and the drawn mark.

[0023] Therefore, in a case where the camera is in a specific condition, for example, in a theme park such as an amusement park, when a specific mark (for example, a mark that imitates a famous person in the theme park) is drawn, it is possible to set a composition that satisfies a composition condition associated with the condition and the mark to a target composition, such as a composition in which the person is arranged at a position in an image frame corresponding to the drawn mark.

[0024] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the target composition is set to a composition that satisfies the following condition: the target subject is arranged at a position in an image frame set according to an operation of the user, the operation that specifies an arrangement range of the target subject in the image frame is performed as a drawing operation on a screen that displays a through image of a captured image, and the control unit determines the number of target subjects to be arranged within the arrangement range according to which operation type of a plurality of operation types the drawing operation belongs to.

[0025] Therefore, the user can specify the arrangement range of the target subject and specify the number of target subjects to be arranged within the arrangement range by one operation as a drawing operation on a screen.

[0026] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the target composition is set to a composition that satisfies the following condition: the target subject is arranged at a target position in an image frame, and the control unit detects movement of the target subject, and controls guidance for arranging the target subject at the target position based on the detected movement of the target subject.

[0027] Therefore, it is possible to perform guidance that takes into account the movement of the target subject as guidance for realizing a composition in which the target subject is arranged at a target position in an image frame.

[0028] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the control unit performs guidance based on a relationship between a predicted position of the target subject after a predetermined time elapses and the target position.

[0029] In a case where the movement of the target subject is large, if guidance based on a difference between the current position and the target position of the target subject is performed, it is difficult to follow the movement of the target subject, and a situation can occur in which the position of the target subject is difficult to adjust to match the target position. Therefore, as described above, guidance is performed based on the position after a predetermined time elapses, which is predicted from the movement of the target subject.

[0030] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the control unit sets the target composition based on a determination result of a scene to be imaged.

[0031] Therefore, the target composition can be automatically set based on the determination result of the scene to be imaged, such as a scene of imaging a train at a station or a scene of imaging a sunset, according to the scene.

[0032] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the target composition is set to a composition that satisfies the following conditions: the target subject is arranged at a target position in an image frame, and the control unit predicts a position of the target subject in a target state according to the scene to be imaged, and controls the guidance so that the predicted position matches the target position.

[0033] The target state according to the scene to be imaged means, for example, a target state of the target subject determined according to the scene, specifically, a state used as, for example, a shutter change, such as a state in which a train stops in a scene of imaging a train at a station, or a state in which a sunset occurs in a sunset scene.

[0034] According to the above-described configuration, the user can be appropriately assisted so that the target subject in the target state is imaged with the target composition.

[0035] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which, in a case where it is estimated that the user is not viewing a screen that displays a through image that captures an image, the control unit enables a function of the guidance.

[0036] Therefore, in a case where the user does not view the screen while imaging is performed, the guidance according to the technology is automatically performed.

[0037] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which the target composition is set to a composition that satisfies the following conditions: the target subject is arranged at a target position in an image frame in a target size, and the control unit performs zoom control so that the size of the target subject becomes the target size.

[0038] Therefore, in the implementation of imaging with the target composition, the need for a zoom operation by the user can be eliminated.

[0039] In the imaging assistance control device according to the above-described technology, it is conceivable to have a configuration in which, in a case where it is determined that the size of the target subject cannot be adjusted to the target size by the zoom control, the control unit determines whether it is a user-movable situation, and in a case where it is determined that it is a user-movable situation, causes the guidance for changing the positional relationship between the user and the target subject to be executed.

[0040] Therefore, for example, even in a situation in which the user cannot move due to imaging in a crowd or the like, it is possible to prevent the occurrence of inconveniences such as causing the user to trouble surrounding people by executing the guidance.

[0041] The imaging assistance control method according to the technology is an imaging assistance control method including: causing, by an information processing apparatus, control such that guidance to a composition change operation by a user is executed by haptic presentation to the user based on a difference between a target composition and an actual composition.

[0042] With this imaging assistance control method, it is also possible to obtain a function similar to that of the imaging assistance control device according to the above-described technology.

[0043] Further, the imaging assistance system according to the technology includes: a haptic presentation apparatus including a haptic presentation unit that executes haptic presentation to a user; and an imaging assistance control device including a control unit that executes control such that guidance to a composition change operation by the user is executed by the haptic presentation of the haptic presentation unit based on a difference between a target composition and an actual composition.

[0044] With this imaging assistance system, it is also possible to obtain a function similar to that of the imaging assistance control device according to the above-described technology. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a block diagram showing an internal configuration example of an information processing apparatus as an embodiment of the imaging assistance control device according to the technology.

[0046] Figure 2 is a diagram for describing a specific example of the imaging assistance technology as an embodiment.

[0047] Figure 3 is a flowchart showing an example of a specific processing procedure for realizing the imaging assistance technology described in Figure 2

[0048] Figure 4 is an explanatory diagram of processing according to the type of a drawing operation.

[0049] Figure 5 ​is a diagram illustrating an example in which name information of a subject type is displayed on a screen.

[0050] Figure 6 is a diagram illustrating an example in which a subject within a range drawn on a screen is set as a target subject.

[0051] Figure 7 is a diagram illustrating an example in which a target composition is set in a case where a specific mark is drawn under a specific condition.

[0052] Figure 8 is a diagram illustrating an example of subject designation information.

[0053] Figure 9 is a diagram illustrating a state in a case where a plurality of target subjects as "dog" are detected in an image frame.

[0054] Figure 10 is a diagram illustrating an example in which a number of target subjects to be arranged within an arrangement range is determined in accordance with an operation type of a drawing operation for designating the arrangement range.

[0055] Figure 11 is a diagram illustrating an example of a scene related to guidance of a manner of holding a camera.

[0056] Figure 12 is a diagram of a portrait holding state.

[0057] Figure 13 is a diagram illustrating another example of a scene related to guidance of a manner of holding a camera.

[0058] Figure 14 is a diagram of a landscape holding state.

[0059] Figure 15 is a diagram illustrating an example of guidance based on movement of a target subject.

[0060] Figure 16 is a diagram illustrating an example of a scene to be imaged.

[0061] Figure 17 is a diagram illustrating an example of a target composition set in accordance with a scene to be imaged.

[0062] Figure 18 is a diagram for predicting a position in which a target subject is in a target state in accordance with a scene to be imaged.

[0063] Figure 19 is a diagram illustrating another example of a scene to be imaged.

[0064] Figure 20 is a diagram illustrating a modified example of vibration presentation for guidance.

[0065] Figure 21 is an explanatory diagram of another modification example of the vibration presentation for guidance.

[0066] Figure 22 is a flowchart showing a processing example in a case where guidance is performed while taking into consideration whether or not it is a situation in which the user is movable.

[0067] Figure 23 is a block diagram showing a configuration example of an imaging assistance system as an embodiment. DETAILED DESCRIPTION

[0068] Hereinafter, the embodiments will be described in the following order.

[0069] <1. Configuration of imaging assistance control device as an embodiment>

[0070] <2. Example of imaging assistance technology>

[0071] <3. Processing procedure>

[0072] <4. Technology regarding setting conditions for specifying a target composition>

[0073] <5. Technology regarding specifying a subject>

[0074] <6. Technology for specifying a target composition according to a situation>

[0075] <7. Processing in a case where there are multiple target subjects>

[0076] <8. Control according to movement of a target subject>

[0077] <9. Variation of tactile presentation>

[0078] <10. Activation regarding guidance function>

[0079] <11. Error notification and the like>

[0080] <12. Modification example>

[0081] <13. Summary of embodiments>

[0082] <14. Prior art>

[0083] <1. Configuration of imaging assistance control device as an embodiment>

[0084] Figure 1 is a block diagram showing an internal configuration example of an information processing device 1 as an embodiment of an imaging assistance control device according to the present technology.

[0085] As illustrated, the information processing apparatus 1 includes an imaging unit 2, a display unit 3, an operation unit 4, a communication unit 5, a sensor unit 6, a control unit 7, a storage unit 8, a sound output unit 9, and a tactile presentation unit 10, and further includes a bus 11 that mutually connects these units so as to be able to perform data communication.

[0086] In the present example, the information processing apparatus 1 is configured as an information processing apparatus having a camera function, such as a smartphone, a tablet terminal, or the like.

[0087] The imaging unit 2 is configured to include a camera unit of an imaging optical system and an image sensor such as a charge-coupled device (CCD) sensor or a complementary metal-oxide semiconductor (CMOS) sensor, and to obtain a captured image in digital data.

[0088] In the present example, the above-described imaging optical system is provided with a zoom lens, and optical zooming can be performed by driving the zoom lens.

[0089] Note that the imaging unit 2 can also include a camera unit as a so-called out-camera that captures an image in a direction opposite to a direction in which a screen 3a of the display unit 3 described below faces, and a camera unit as a so-called in-camera that captures an image in the same direction as the direction in which the screen 3a faces.

[0090] The display unit 3 includes a display device capable of displaying an image, such as a liquid crystal display (LCD), an organic electroluminescence (EL) display, or the like, and displays various types of information on a screen 3a of the display device.

[0091] The operation unit 4 comprehensively represents an operator, such as a button, a key, and a touch panel, provided in the information processing apparatus 1. In the present example, the touch panel is formed to detect an operation involving contact with the screen 3a.

[0092] The communication unit 5 performs wireless or wired data communication with an external apparatus of the information processing apparatus 1. Examples of a data communication scheme with the external apparatus include communication via a communication network such as a local area network (LAN) or the Internet, and near field wireless communication such as Bluetooth (registered trademark).

[0093] The sensor unit 6 comprehensively represents various sensors provided in the information processing apparatus 1. Examples of the sensors in the sensor unit 6 include a microphone, a G sensor (acceleration sensor), a gyro sensor (angular velocity sensor), a temperature sensor, a position sensor that detects a position of the information processing apparatus 1, a proximity sensor, an illuminance sensor, and the like.

[0094] Here, examples of the position sensor include a global navigation satellite system (GNSS) sensor, a geomagnetic sensor for geomagnetic positioning, and the like.

[0095] Note that, for the position detection of the information processing apparatus 1, Wi-Fi positioning using radio field strength of Wi-Fi (Wireless Fidelity: a registered trademark) can also be performed.

[0096] The control unit 7, for example, includes a microcomputer including a central processing unit (CPU), a read only memory (ROM), and a random access memory (RAM), and performs control to realize various types of computation and various operations of the information processing apparatus 1, for example, when the above-described CPU executes processing according to a program stored in the above-described ROM or the like.

[0097] The control unit 7, for example, performs various types of control on the imaging unit 2. For example, an instruction to start and end an imaging operation, control of the above-described optical zoom, control of electronic zoom, and the like are performed.

[0098] Further, the control unit 7 controls output information of the output information unit serving as the display unit 3, the sound output unit 9, and the tactile presentation unit 10. For example, as an example of control with respect to the display unit 3, control is performed so that the display unit 3 displays an image captured by the imaging unit 2 as a through image in response to establishment of a predetermined condition such as activation of a camera application. Further, in response to an operation input or the like via the operation unit 4, display control of an image related to a graphical user interface (GUI) is performed.

[0099] Further, as illustrated in the figure, the control unit 7 has a function as an image recognition processing unit 7a and a function as a guidance control unit 7b.

[0100] The image recognition processing unit 7a performs image recognition processing on an image captured by the imaging unit 2. Specifically, the image recognition processing unit 7a performs, as image recognition processing, subject detection processing of detecting an image region having a specific image characteristic as a subject, such as face detection or pupil detection, subject recognition processing of recognizing a kind (for example, a kind of a person, a dog, a cat, a bird, a table, a flower, a bicycle, a car, a ship, a mountain, and the like) of a detected subject, and tracking processing of tracking a position of a detected subject.

[0101] The control unit 7 performs processing of causing the display unit 3 to display information indicating a result of the image recognition processing during display of the through image. For example, the display unit 3 is caused to display information (for example, frame information) indicating a range of a subject detected in the above-described subject detection processing. Further, the display unit 3 can also be caused to display information indicating a kind of a subject recognized in the subject recognition processing.

[0102] The guidance control unit 7b performs control so that guidance to a composition change operation by the user is performed on an image captured by the imaging unit 2 based on a difference between the target composition and the actual composition.

[0103] Note that the specific process performed by the control unit 7 as the guidance control unit 7b will be described again.

[0104] The memory unit 8 is configured using, for example, a semiconductor storage device such as a flash memory or a nonvolatile storage device such as a hard disk drive (HDD), and stores various types of data for processing by the control unit 7. In the present example, the memory unit 8 functions as a memory that stores images captured by the imaging unit 2. For example, at the time of still image capture, the control unit 7 causes the memory unit 8 to store one frame of an image captured at a timing instructed by a shutter operation by the user. Further, at the time of moving image capture, the control unit 7 causes the memory unit 8 to store image data of a moving image obtained by the imaging unit 2 in response to a recording start operation.

[0105] The sound output unit 9 includes a speaker and outputs various sounds in response to an instruction from the control unit 7.

[0106] The tactile presentation unit 10 includes a tactile presentation device that performs tactile presentation to the user. In the present example, the tactile presentation unit 10 includes a vibration device as the tactile presentation device, and can perform tactile presentation to the user by vibration.

[0107] <2. Example of Imaging Assistance Technology>

[0108] An example of the imaging assistance technology as an embodiment will be described with reference to Figure 2

[0109] In the present embodiment, guidance to a composition change operation by the user is performed by tactile presentation (vibration in the present example) of the tactile presentation unit 10 based on a difference between the target composition and the actual composition.

[0110] Here, note that, for the sake of clarity, a composition change operation refers to an operation for changing a composition, such as changing an orientation of a camera, changing a positional relationship between a camera and a subject, or zooming.

[0111] In the present example, the target composition is a composition that satisfies a condition in which a target subject is arranged at a target position in an image frame in a target size. In the present example, the target composition is set based on an operation input by the user.

[0112] Further, it is assumed that the target subject is specified by the user in the present example.

[0113] Figure 2 ​A is a diagram for describing an example of a technique for specifying a target subject.

[0114] In a case where a subject is detected on the screen 3a on which the through image of the captured image is displayed by the above-described image recognition processing unit 7a, a mark indicating the range of the detected subject, such as a dotted line mark in the drawing, is displayed.

[0115] The guidance control unit 7b in the control unit 7 receives an operation (for example, a touch operation) that specifies the mark indicating the range of the detected subject displayed in this way as an operation for specifying a target subject.

[0116] Figure 2 B and Figure 2 C is a diagram for describing an example of an operation of specifying a target region At as a region in which a target subject is to be accommodated.

[0117] In the present example, the operation of specifying the target region At is an operation of drawing a circle on the screen 3a illustrated in B. The guidance control unit 7b sets the range specified by this drawing operation as the target region At (see C). Figure 2 Figure 2 C).

[0118] In the present example, the center position of the target region At is set as the target position Pt, and a composition that satisfies a condition in which the position Ps (for example, the center position) of the target subject matches the target position Pt and the size of the target subject becomes a target size based on the size specified as the target region At is set as the target composition.

[0119] Hereinafter, as an example of description, it is assumed that the target size of the target subject is the same as the size specified as the target region At.

[0120] Note that the condition that the size of the target subject exactly matches the size specified as the target region At is not essential, and a range of, for example, ±10% of the size specified as the target region At can also be set as the target size, and this target size can be set as the condition.

[0121] The guidance control unit 7b performs guidance to achieve the target composition in response to the setting of the target region At in accordance with the operation of the user.

[0122] Here, regarding the adjustment of the size of the target subject, in the present example, the guidance control unit 7b performs zoom control of the imaging unit 2 to perform the adjustment. That is, in the present example, the adjustment of the size of the target subject is automatically performed on the information processing apparatus 1 side, and therefore, guidance related to the size of the target subject is not performed.

[0123] ​During the guidance, the position Ps of the target subject is sequentially (e.g., per frame) grasped by the tracking processing performed by the image recognition processing unit 7a, and the guidance control unit 7b recognizes the positional relationship between the target position Pt and the position Ps of the target subject.

[0124] During the guidance, the guidance control unit 7b performs the guidance of the change of the orientation of the information processing apparatus 1 (the camera) in accordance with the positional relationship sequentially recognized as described above.

[0125] In the present example, the guidance at this time is performed by changing the mode of the tactile presentation based on the size of the difference between the target composition and the actual composition. Specifically, the vibration pattern is changed in accordance with the interval distance between the position Ps of the target subject and the target position Pt. For example, when the interval distance between the position Ps of the target subject and the target position Pt is reduced, the vibration period is shortened. As a specific example, an example can be given in which a first threshold value and a second threshold value (where the first threshold value > the second threshold value > 0) are set with respect to the interval distance between the position Ps of the target subject and the target position Pt, the number of vibrations per unit time is set to one in a state where the interval distance exceeds the first threshold value, the number of vibrations per unit time is set to two in a state where the interval distance is equal to or smaller than the first threshold value and exceeds the second threshold value, the number of vibrations per unit time is set to three in a state where the interval distance is equal to or smaller than the second threshold value, and so on.

[0126] When the above-described vibration control is performed, even in a situation where imaging is performed without gazing at the screen 3a, the user can easily grasp whether the composition is approaching or distancing from the target composition in the process of changing the orientation of the information processing apparatus 1, and it becomes easy to achieve the target composition.

[0127] Figure 2 D shows an example of the state of the screen 3a when the target composition is achieved. That is, in the present example, the position Ps of the target subject matches the target position Pt.

[0128] Note that the condition that the position Ps of the target subject matches the target position Pt is not a necessary condition for achieving the target composition. For example, other conditions can be set by setting the fact that the distance between the position Ps of the target subject and the target position Pt becomes equal to or smaller than a predetermined threshold value other than zero as a condition for achieving the target composition, setting the fact that the number of matches per unit time between the position Ps of the target subject and the target position Pt reaches a predetermined number of times as a condition for achieving the target composition, and so on.

[0129] In a case where the target composition has been achieved, the guide control unit 7b causes the tactile presentation unit 10 to make a notification of this fact in the present example. The notification in a case where the target composition has been achieved is performed in a different mode from the tactile presentation for guidance performed in a state where the target composition has not been achieved.

[0130] The user receives the above-described notification of achievement of the target composition and performs a shutter operation. Thus, it is possible to capture a still image with the target composition.

[0131] <3. Processing>

[0132] An example of a specific processing procedure for implementing the above-described imaging assistance technique will be described with reference to the flowchart of Figure 3 .

[0133] Note that the processing illustrated in Figure 3 is performed by the control unit 7 (CPU) based on a program stored in a predetermined storage device such as a built-in ROM. The control unit 7 starts the processing illustrated in Figure 3 , for example, in response to activation of the camera application.

[0134] First, in step S101, the control unit 7 determines whether the mode is the guidance mode. That is, it is determined whether the mode is a mode in which the function of giving guidance on the above-described composition change operation is enabled. If the mode is not the guidance mode, the control unit 7 shifts to normal mode processing. That is, the control unit 7 shifts to imaging processing in which guidance is not performed.

[0135] On the other hand, if the mode is the guidance mode, the control unit 7 starts image recognition processing in step S102. That is, the processing as the above-described image recognition processing unit 7a starts to perform subject detection processing, subject recognition processing, and subject tracking processing.

[0136] In step S103 subsequent to step S102, the control unit 7 performs processing of receiving designation of a target subject. That is, an operation of designating a mark indicating a range of a detected subject displayed on the screen 3a is received.

[0137] In a case where an operation of designating a target object subject has been detected, the control unit 7 performs processing of receiving designation of a target region At in step S104. That is, in the present example, an operation of drawing a circle on the screen 3a as exemplified above in Figure 2 B is received.

[0138] In a case where an operation of designating a target region At has been detected, the control unit 7 sets a target position Pt (a center position of the target region At in the present example).

[0139] In step S105 subsequent to step S104, the control unit 7 executes zoom adjustment processing. That is, zoom control is executed on the imaging unit 2 so that the size of the target subject in the captured image matches the size of the target region At.

[0140] In step S106 subsequent to step S105, the control unit 7 starts guidance processing. That is, control is executed so that, based on the positional relationship between the position Ps of the target subject sequentially recognized according to the tracking processing of the target subject and the target position Pt, the composition change operation is guided by vibration presentation of the tactile presentation unit 10. Specifically, in the present example, guidance is executed by changing the vibration pattern according to the interval distance between the position Ps of the target subject and the target position Pt. For example, it is conceivable that, when the interval distance between the position Ps of the target subject and the target position Pt is reduced, control is executed to shorten the vibration period, as exemplified above.

[0141] Further, in the guidance processing in this case, it is determined whether or not the target composition has been achieved according to whether or not the position Ps of the target subject matches the target position Pt, and in the case where the target composition has been achieved, notification of this fact is made by vibration presentation of the tactile presentation unit 10.

[0142] In step S107 subsequent to step S106, the control unit 7 waits until a shutter operation is detected, and in the case where a shutter operation is detected, the guidance processing in step S108 is ended, and the processing proceeds to step S109.

[0143] In step S109, the control unit 7 determines whether or not to end imaging. That is, it is determined whether or not a predetermined condition, which is set in advance as an imaging end condition, has been satisfied, for example, an operation to end the camera application is detected.

[0144] For example, in the case where an operation to end the camera application has not been detected and it is determined not to end imaging, the control unit 7 returns to step S103. Thus, reception of designation of the target subject and the target region At, and zoom adjustment processing and guidance processing based on the designated target subject and target region At are executed for the next still image capture.

[0145] On the other hand, in the case where it is determined to end imaging in step S109, the control unit 7 ends the image recognition processing in step S110, and ends the series of processing illustrated. Figure 3

[0146] ​Note that the above has described an example in which guidance is performed at the time of still image capturing, but the guidance function as an embodiment can be appropriately applied also at the time of moving image capturing. In particular, it is not desirable to record unnecessary sound at the time of moving image capturing, so it is appropriate to perform guidance by tactile presentation as in the embodiment.

[0147] Further, in the above example, zoom adjustment is performed, so in this example, guidance about a composition change operation for size matching between the target subject and the target region At is not performed, but of course similar guidance can be performed even for a composition change operation for such size matching. For example, it is conceivable to perform guidance for changing the vibration pattern based on the magnitude of the size difference between the target subject and the target region At.

[0148] Here, in a case where guidance is performed for both position matching and size matching between the target subject and the target region At, it is conceivable to perform vibration for such guidance in a time-division manner. For example, it is conceivable to allocate the first half of a unit period to guidance for position matching, and the latter half to guidance for size matching.

[0149] Alternatively, it is also possible to provide the two guidance by a technique other than time division, such as expressing the guidance for position matching and the guidance for size matching by vibration intensity and the number of vibrations per unit time, respectively.

[0150] Further, regarding the guidance for position matching, it is also possible to guide any direction to which the orientation of the camera is preferably changed. At this time, it is possible to guide the direction in four directions of up, down, right, and left, or it is possible to guide the direction in only two directions of vertical and horizontal.

[0151] <4. Technique about Setting Condition of Designated Target Composition>

[0152] Here, the above has described an example of the operation of designating the target region At (that is, an operation related to the setting of the arrangement position of the target subject in the image frame), but it is also possible to set the type of the drawing operation on the screen 3a, and determine the setting condition other than the arrangement position of the target subject in the setting condition of the target composition according to such type of the drawing operation.

[0153] Here, the setting condition of the target composition refers to a condition for setting any composition serving as the target composition, and examples thereof can include the arrangement position of the target subject in the image frame and a condition for setting a single target subject or a plurality of target subjects.

[0154] Examples of the type of the drawing operation include, as Figure 4 A toFigure 4 drawing of a point, drawing of an ellipse, and drawing of a circle by multi-point touch such as double-finger touch.

[0155] For each type of such drawing operation, a setting condition other than the arrangement position of the target subject is set in advance in the setting condition of the target composition. Then, the guidance control unit 7b determines the setting condition of the target composition corresponding to the type of the drawing operation performed by the user, from which of the plurality of preset types the type belongs.

[0156] As a specific example, in the case of drawing a rectangle as shown in A, a condition in which the entire subject is placed in the rectangle is determined as the setting condition of the target composition. Figure 4 A, a condition in which the entire subject is placed in the rectangle is determined as the setting condition of the target composition.

[0157] Further, in the case of drawing a point as shown in B, a condition in which the target subject is arranged at the target position Pt and the position of the point is the target position Pt is determined as the setting condition of the target composition, without using the size of the target subject as a condition. Figure 4 B, a condition in which the target subject is arranged at the target position Pt and the position of the point is the target position Pt is determined as the setting condition of the target composition, without using the size of the target subject as a condition.

[0158] Further, in the case of drawing an ellipse as shown in C, a condition in which a plurality of target subjects are arranged in the ellipse is determined as the setting condition of the target composition. Figure 4 C, a condition in which a plurality of target subjects are arranged in the ellipse is determined as the setting condition of the target composition.

[0159] Further, in the case of drawing a circle by multiple taps as shown in D, a condition in which the target subject is arranged in the circle is determined as the setting condition of the target composition, without using the size of the target subject as a condition. Figure 4 D, a condition in which the target subject is arranged in the circle is determined as the setting condition of the target composition, without using the size of the target subject as a condition.

[0160] Note that the example described above with reference to Figure 2 the drawing of a circle, a condition in which the size of the target subject is adjusted to match the size of the circle and the target subject is arranged at the target position Pt set according to the drawing area of the circle is determined as the setting condition of the target composition.

[0161] Since the setting condition of the target composition is determined according to the type of the drawing operation as described above, the user can specify the arrangement position of the target subject and specify the setting condition other than the arrangement position in the setting condition of the target composition by one operation as the drawing operation on the screen 3a.

[0162] Therefore, the burden of user operation can be reduced.

[0163] <5. Technology on Specifying a Subject>

[0164] The reception of the designation of the target subject can also be performed by voice. For example, the control unit 7 is configured so that the type name of the subject identifiable by the above-described subject identification processing can be recognized by voice, and in response to the recognition of the utterance of the type name of the subject identifiable in the subject identification processing, the subject belonging to the type is determined as the target subject.

[0165] Here, this processing of determining the target subject based on the input voice can also be performed at a stage where the subject has not been detected in the captured image. For example, in a case where it is desired to set a dog, a child, or the like as the target subject, the user can sense the presence of a dog or a child in the vicinity from the surrounding sound or the like, and thus the designation of the target subject is effectively received by voice regardless of whether the target subject is detected in the image (image frame). In a case where the undetected target subject is designated, when a subject of the corresponding type is detected in the image, the subject is set as the target subject.

[0166] Further, the reception of the designation of the target subject can also be performed by displaying the name information of the type of the candidate subject and a mark indicating the range thereof, for example, as exemplified in Figure 5 In this case, the guide control unit 7b displays the mark and the name information of the type on the screen 3a, the mark indicating the range of the subject whose type has been recognized among the detected subjects.

[0167] Further, the reception of the designation of the target subject can be performed as the reception of the range drawing operation on the screen 3a, as exemplified in Figure 6 In this case, the subject in the drawn range is set as the target subject.

[0168] This reception technique is a technique applicable in a case where the subject detection processing does not detect the object desired by the user to be set as the target subject.

[0169] Note that, Figure 6 the technique illustrated in FIG. 8 can be poor in robustness of tracking, but improvement can be expected by using the background difference or optical flow technique.

[0170] Further, in a case where the designation of the target subject is performed by the range drawing operation as exemplified in Figure 6 it can be conceivable to make a notification to allow the user to recognize whether the designation of the target region At is received or the designation of the target subject is received.

[0171] Further, as the reception of the designation of the target subject, a list of information indicating the types of the subjects that can be recognized in the subject recognition process can also be displayed on the screen 3a to receive the designation from the list.

[0172] Furthermore, if a target subject becomes untrackable after it has been designated, an error notification can be issued. This notification can be, for example, through vibration of the haptic presentation unit 10.

[0173] <6. Techniques for specifying target mapping based on the situation>

[0174] The setting of the target composition based on the user's operation is not limited to the specific examples exemplified above. For example, the state of the information processing device 1 used as a camera can be determined, and if the state is a specific state and a specific mark is drawn through a drawing operation on screen 3a, the composition that satisfies the composition conditions associated with the state and the drawn mark can be set as the target composition.

[0175] Reference Figure 7 Describe a specific example.

[0176] Figure 7 An example is that, when the information processing device 1 is located in a specific theme park, and a marker that imitates a specific character in the specific theme park is drawn on the screen 3a, the composition in which the character is arranged at the position in the image frame corresponding to the drawn marker is set as the target composition.

[0177] Furthermore, although not shown, when the information processing device 1 is facing the outdoor sky, and a star-shaped mark is drawn on the screen 3a, it is conceivable to set the composition with the brightest star within the range of the star-shaped mark as the target composition.

[0178] In the case of setting the target composition according to the situation described above and drawing marks, such as Figure 8 The subject designation information I1 illustrated herein is stored in a storage device, such as a memory unit 8, that can be read by the control unit 7.

[0179] The guidance control unit 7b determines whether the state of the information processing device 1 is a predetermined specific state. For example, in Figure 7 In the example above, it is determined whether the information processing device 1 is located in a specific theme park. This determination can be made, for example, based on the detection signal from the position sensor in the sensor unit 6. Furthermore, in the example above, it is determined whether the information processing device 1 is facing the outdoor sky. This determination can be made, for example, based on the detection signals from the position sensor and microphone in the sensor unit 6, as well as the detection signals from the G-sensor and gyroscope sensors.

[0180] The guidance control unit 7b identifies the type of condition in which the information processing device 1 is placed by performing this determination.

[0181] Furthermore, when a drawing operation is performed on screen 3a, the guidance control unit 7b determines whether the drawn mark is a specific mark. For example, in Figure 7 In the example above, it determines whether the mark is a mark that imitates a specific character, and in the example of the star above, it determines whether the mark is a star-shaped mark. The guide control unit 7b identifies the type of mark drawn by performing this determination.

[0182] In the subject designation information I1, the information indicating the type of the target subject is associated with each combination of condition type and marker type, and the guidance control unit 7b can obtain information indicating the type of subject to be targeted in the target composition from the subject designation information I1 by identifying the condition type and marker type as described above.

[0183] The guidance control unit 7b sets the subject specified from the subject type information obtained by referring to the information processing device 1 in this way as the target subject, and sets the composition that satisfies the condition of arranging the target subject within the range of the marks drawn on the screen 3a as the target composition.

[0184] <7. Handling situations with multiple target subjects>

[0185] In the case that there are multiple subjects in the image frame that correspond to the type designated as the target subject, it is also possible to receive a specification of the range in which the target subjects are to be arranged and how many target subjects are to be arranged within that range.

[0186] Figure 9 This illustrates a situation where multiple target subjects, posing as "dogs," are detected in an image frame of a captured image.

[0187] For example, in such Figure 10 As shown in Figure A, a drawing operation is performed for a specified range to set the composition of two of the multiple detected target subjects within the specified range in the case of two protruding parts on the image 3a.

[0188] In addition, in such Figure 10 As shown in B, a composition is set up by performing a drawing operation for a specified range to arrange all of the detected target subjects within the specified range in a composition having a predetermined number or more (e.g., at least three or more) protrusions.

[0189] For example, as in the example above, if the operation of specifying the arrangement range of the target subject is performed as a drawing operation on screen 3a, the guidance control unit 7b can determine the number of target subjects to be arranged within the arrangement range based on which of the multiple operation types the drawing operation belongs to.

[0190] Since the number of target subjects to be arranged within the arrangement range is determined in this way based on the type of the drawing operation for designating the arrangement range, the user can designate the arrangement range of the target subjects and designate the number of target subjects to be arranged within the arrangement range as one operation of the drawing operation on the screen 3a.

[0191] Therefore, the burden of the user operation can be reduced.

[0192] Note that the user can estimate whether or not a plurality of subjects exist as target subjects in the surrounding environment according to the surrounding conditions (vision, sound, etc.). For example, the presence of a plurality of objects can be estimated from the barking sound of a dog or the sound of a child, etc. From this point of view, the reception of the designation of the arrangement range of the target subjects and the determination of the arrangement number of the target subjects according to the type of the drawing operation for designating the arrangement range can also be performed in a state in which the target subjects are not detected in the captured image.

[0193] Further, in a case in which a plurality of subjects corresponding to the type designated as the target subjects are detected in the image, it is also possible to notify the user of this fact. For example, it is conceivable to make the notification by voice such as "There are two dogs".

[0194] Further, regarding the tracking of the target subjects during the execution of the guidance to arrange a plurality of target subjects in the designated arrangement range, in a case in which some of the plurality of target subjects are separated from the other target subjects, it is possible to track only the target subjects close to the arrangement range.

[0195] Further, regarding the tracking of the target subjects during the execution of the guidance to arrange a plurality of target subjects in the designated arrangement range, in a case in which some of the plurality of target subjects are lost, the remaining target subjects are tracked to continue the guidance.

[0196] Further, which of the plurality of target subjects is to be set as the tracking target for the guidance can be dynamically determined based on the size of the arrangement range designated by the user. For example, in a case in which there are two target subjects, it is conceivable that if the image size of the designated arrangement range is equal to or smaller than a predetermined size, such as 100 pixels x 100 pixels, one target subject is set as the tracking target for the guidance, or if not, both target subjects are set as the tracking target for the guidance.

[0197] Note that, in a case in which "single" is designated as the designation regarding how many target subjects are to be accommodated in the designated arrangement range, it is conceivable that the subject closest to the arrangement range among the plurality of target subjects detected in the image is set as the tracking target for the guidance.

[0198] Alternatively, in a case where a plurality of target subjects have been detected for the "single" designation as described above, the user can also be presented with the characteristics of the plurality of target subjects (e.g., hair color, size, etc. in the case of a dog) and asked which one should be set as the target.

[0199] <8. Control in accordance with movement of target subject>

[0200] Various types of control can be conceived as control in accordance with movement of the target subject.

[0201] For example, it is conceivable to estimate the direction of movement of the target subject from image analysis of the captured image and to guide the user in a manner in accordance with the direction.

[0202] Specifically, for example, as shown in Figure 11 , in a scene in which a monkey is climbing a tree, the user is provided with guidance indicating vertical holding, as exemplified in Figure 12 . Further, in a scene in which a crab is moving, as shown in Figure 13 A, or a scene in which a child is walking, as shown in Figure 13 B, the user is provided with guidance indicating lateral holding, as exemplified in Figure 14 .

[0203] At this time, the estimation of the direction of movement of the target subject can be estimation in accordance with, for example, the direction or amount of flow of the background in the captured image, etc. Alternatively, a technique can also be conceived in which a table in which the direction is associated with each type of target subject is prepared, and the direction information corresponding to the determined type of target subject is obtained from the table.

[0204] Further, as a guidance control technique in a case where the target subject is a moving subject, the movement of the target subject can also be detected and guidance control can be executed based on the detected movement of the target subject to arrange the target subject at the target position Pt.

[0205] Figure 15 A shows the relationship between the position Ps(t) of the target subject, the target region At(t), and the target position Pt(t) at a certain time (t).

[0206] As shown in the figure, the interval distance between the position Ps(t) of the target subject and the target position Pt(t) is "d". In general, it is conceivable to provide guidance based on the interval distance d as the guidance at the point in time (t).

[0207] However, at a point in time (t+1) after a predetermined period of time from the time (t), assuming that the amount of movement of the target subject during the predetermined period of time is "do" as shown in the figure, then if the user changes the orientation of the camera in accordance with the guidance at the point in time (t) and asFigure 15 In the case where the image frame side is moved by "di" as shown in B, the target subject also moves by "do", and therefore, it is difficult to set the position Ps(t+1) of the target subject at the target position Pt(t+1).

[0208] Therefore, over a period of a plurality of past frames of captured images, the amount of movement "de" of the target subject between respective frames is calculated, and guidance based on the amount of movement de is performed. Instead of simple averaging, constant speed, acceleration, and deceleration are calculated in a distinguishable manner. The amount of movement de is a vector quantity having distance and direction.

[0209] During the guidance, in the case where the target subject moves in a direction approaching the target position Pt, the guidance control unit 7b sets "d-de" as a target value of the amount of movement of the image frame side, and performs control to provide guidance requesting the user not to move the camera too much. On the other hand, in the case where the target subject moves in a direction away from the target position Pt, the guidance control unit 7b sets "d+de" as a target value of the amount of movement of the image frame side, and performs control so as to guide the user to move the camera more.

[0210] In this way, guidance taking into account the movement of the target subject is implemented as guidance for achieving a composition in which the target subject is arranged at the target position in the image frame.

[0211] Therefore, according to the case where the target subject is a moving subject, it is possible to improve the accuracy of the guidance.

[0212] Further, as guidance taking into account the movement of the subject, it is also possible to perform guidance based on a relationship between a predicted position of the target subject after a predetermined time and the target position.

[0213] Specifically, in this case, the guidance control unit 7b obtains an amount of movement of the target subject based on, for example, captured images corresponding to a plurality of past frames, and predicts a position of the target subject after a predetermined time from this amount of movement. Then, guidance is performed based on a relationship between the position predicted in this way and the target position Pt. Specifically, for example, it is conceivable to perform guidance to change the vibration pattern based on the magnitude of the interval distance between the predicted position and the target position Pt.

[0214] It is also possible to improve the accuracy of the guidance according to the case where the target subject is a moving subject by performing such guidance based on a predicted position of the target subject.

[0215] Further, in the case where still image capture is performed with a moving subject as the target subject, it is also possible to perform the following guidance.

[0216] First, assume a case where a still image of a train is captured at a platform of a station, for example, as illustrated in Figure 16

[0217] In the present example, it is assumed that the guide control unit 7b sets a target composition based on a determination result of a scene to be imaged. Here, as illustrated in Figure 16 Figure 17

[0218] In the present example, a plurality of combinations of a scene to be imaged and an optimal composition (including information on a type of a subject to be set as a target subject) are set to process a plurality of scenes into images.

[0219] The guide control unit 7b determines a current scene to be imaged by image analysis processing or the like on an image captured by the imaging unit 2. Here, a technique of determining a scene to be imaged is not particularly limited. For example, a technique of using artificial intelligence (AI) trained to determine a scene from an image type can be exemplified. Alternatively, it is also conceivable to estimate an environment in which the information processing apparatus 1 is placed based on input information from various sensors such as a microphone, a position sensor, a G sensor, and a gyro sensor, and determine a scene.

[0220] Then, the guide control unit 7b sets an optimal composition determined for the determined scene to be imaged as a target composition.

[0221] In response to the setting of the target composition in this way, the guide control unit 7b performs a guide process.

[0222] The guide in this case is not performed based on a current position of a target subject, but is performed based on a position in which the target subject is in a target state set in accordance with a scene to be imaged.

[0223] Here, the target state refers to a target state of a target subject set in accordance with a scene to be imaged. Specifically, the target state refers to a state serving as a shutter opportunity, for example.

[0224] For the scene to be imaged of the train illustrated in Figure 16

[0225] In this case, for example, as illustrated in Figure 18 ​​​​As exemplified in A, the guidance control unit 7b predicts the position Ps' at which the target subject is in the target state (that is, the stopped state) from the current captured image, and controls the haptic presentation unit 10 so that guidance of arranging the predicted position Ps' at the target position Pt is performed.

[0226] Here, for example, by detecting the image of the stop line of the train at the platform, the position (Ps') at which the train is in the stopped state at the platform can be estimated from the position of the stop line. Alternatively, the speed of the train can be calculated from the captured image, and the position at which the stopped state is achieved can be estimated based on the speed. Note that when the stopped position is estimated based on the speed like this, information of the distance from the train is used, but the information of the distance can also be calculated using the time difference between the captured images, or the information of the distance can also be separately obtained using a distance measuring sensor.

[0227] Note that under the condition that the target subject is within the target region At, the shutter can be automatically closed (the captured image of the still image can be stored in the memory unit 8).

[0228] Further, in the example of the train described above, it is also conceivable that when the train has reached the stopped position, the size of the target region At is set to a size based on the size of the train.

[0229] While the scenario related to the train has been exemplified as an example of the scene to be imaged as described above, similar techniques can be performed for, for example Figure 19 A sunset scenario exemplified in A.

[0230] For the sunset scenario to be imaged, the target subject is the sun, and where as Figure 19 The composition in which the target region At (the target position Pt) is set as shown in B is set to the target composition.

[0231] Further, in the sunset scenario to be imaged, the sunset state is determined as the target state of the target subject.

[0232] Note that the current time information can also be used to determine whether the scenario is the sunset scenario to be imaged.

[0233] In this case, the guidance control unit 7b predicts the position Ps' at which the target subject is in the target state from the current captured image according to Figure 19 A sunset scenario exemplified in A. For example, the position of the sunset can be predicted based on the moving trajectory of the sun obtained from the captured image within a certain past period.

[0234] In response to the prediction of the position of the sunset (Ps'), in this case, the guidance control unit 7b controls the tactile presentation unit 10 so that guidance is performed to arrange the predicted position Ps' at the target position Pt.

[0235] Here, it is also possible to set the target composition of each scene to be imaged based on the result of analyzing the composition of a large number of captured images uploaded on the Internet, such as captured images posted on a social network service (SNS) site. Alternatively, it is also possible to learn and set the user's favorite composition based on the result of analyzing the composition of past images captured by the user.

[0236] <9. Variation of tactile presentation>

[0237] Here, it is conceivable to change the intensity, frequency, interval, or pattern of the vibration when changing the vibration pattern in the guidance.

[0238] Furthermore, the vibration is not only used for the guidance, but the vibration can also be used to simultaneously perform the notification of another information. For example, in the case where the interval distance between the position Ps of the target subject and the target position Pt is represented by the vibration, it is conceivable to notify the moving intensity of the subject by the vibration. In this case, it is conceivable to represent the interval distance from the target position Pt by the difference in the vibration frequency, and to represent the amount of movement of the subject between frames by the vibration intensity.

[0239] Furthermore, in the case where a plurality of target subjects are to be arranged in the target region At, in the case where the situation has changed, such as in the case where some of the target subjects have been lost or in the case where the lost target subjects have been successfully tracked again, it is also possible to notify this fact by the vibration.

[0240] Furthermore, the vibration for the guidance can also be performed in the form of picture vibration, as exemplified in Figure 20 In this case, the guidance is performed by the vibration of the picture after the target position Pt is specified, on the premise that the finger keeps touching the picture 3a after the target position Pt is specified. In this case, for example, it is conceivable to represent the interval distance from the target position Pt by the vibration intensity (for example, the vibration becomes stronger as the distance decreases). Note that, for example, in this case, a piezoelectric element can be used as the vibration device.

[0241] Furthermore, as exemplified in Figure 21 It is also conceivable to perform the guidance by the vibration using the vibration devices 15 arranged in the upper, lower, right, and left portions of the information processing apparatus 1, as exemplified in

[0242] In this case, a technique of expressing the proximity to the target position Pt by the vibration intensity can also be applied.

[0243] <10. Activation of the guidance function>

[0244] The guidance function for the framing change operation can be activated or deactivated based on the user's operation.

[0245] For example, it is conceivable that the guidance function is activated in a case where the operation of setting the "visual impairment mode" has been performed in the accessibility item in the "Settings" menu in the smartphone.

[0246] Alternatively, it is also conceivable that the guidance function is automatically activated.

[0247] For example, it is conceivable that the guidance function is activated in a case where it has been estimated that the user is not watching the screen 3a.

[0248] Specifically, in a case where imaging is performed in a crowd or the like, there is a case where imaging is performed with a hand raised, and in this case, imaging is performed without watching the screen 3a. Therefore, estimation of whether the user is performing imaging while the hand is raised is performed as estimation of whether it is a condition that the user is not watching the screen 3a, and the guidance function is activated in a case where it is estimated that the user is performing imaging while the hand is raised. Here, whether the user is performing imaging while the hand is raised can be estimated based on, for example, an image captured by the inner camera and a detection signal from the G sensor or the gyro sensor in the sensor unit 6. Alternatively, estimation can be performed using a detection signal from a height sensor.

[0249] Alternatively, estimation of whether it is a condition that the user is not watching the screen 3a can be performed as estimation of whether the user's line of sight is directed toward the screen 3a, for example. Whether the user's line of sight is directed toward the screen 3a can be estimated based on, for example, an image captured by the inner camera.

[0250] Furthermore, it is also conceivable that the guidance function is activated in response to recognizing a specific type of subject from a captured image.

[0251] Alternatively, it is also conceivable that the guidance function is activated in response to determining that the movement of the subject is large.

[0252] Furthermore, the condition for activating the guidance function can be an AND (and) condition of these conditions. For example, the guidance function is activated if "child" is recognized as the subject and the movement of the subject is large, and the like.

[0253] <11. Error notification or the like>

[0254] In a case where the condition of adjusting the size of the target subject to match the target size is set as the target composition, there can be a case where the size of the target subject is too large or too small with respect to the target size so that matching with the target size is not possible by the size adjustment of zooming.

[0255] In a case where it is not possible to perform any zooming any more, or in a case where it is predicted that matching with the target size is not possible by the size adjustment of zooming, it is conceivable to notify the user of this fact by the haptic presentation of the haptic presentation unit 10.

[0256] As an example, a description will be given below with respect to a case where, on the premise that the size matching of the target subject is automatically performed by the above-described zooming control, it is determined whether it is a user-movable condition in a case where it is determined that it is not possible to adjust the size of the target subject to match the target size by the zooming control (see step S105), and an error notification is made in a case where it is determined that it is not a user-movable condition. Here, in a case where it is determined that it is a user-movable condition, guidance is performed to change the positional relationship between the user and the target subject.

[0257] An example of a specific processing procedure will be described with reference to the flowchart of Figure 22

[0258] Note that some of the processing performed in step S103 and the subsequent steps among the processing shown in Figure 22 Figure 3

[0259] In this case, in response to the execution of the processing of receiving the designation of the target subject in step S103 and the processing of receiving the designation of the target region in step S104, in step S201, the control unit 7 calculates a zoom value Td for adjusting the size of the target subject to match the size of the target region At. This zoom value is obtained as a value within a variable zoom range (for example, 35 mm to 100 mm).

[0260] Then, in step S202 subsequent to step S201, the control unit 7 determines whether it is within the variable zoom range. Specifically, it is determined whether the above-described zoom value Td is within a range equal to or greater than a minimum zoom value TR and equal to or less than a maximum zoom value TE, which will be described below.

[0261] The minimum zoom value TR is the minimum value in the variable zoom range, and the maximum zoom value TE is the maximum value in the variable zoom range.

[0262] ​​​Here, in a case where only optical zoom is used, the maximum zoom value TE is set to the maximum value in the zoomable range by optical zoom. In a case where digital zoom (cropping of a captured image) is used, it is conceivable to determine the maximum zoom value TE taking into account image quality degradation caused by digital zoom. Since a condition in which recognition of any object fails when image quality becomes rough due to digital zoom varies depending on brightness, environment, subject type, and user's eyesight, the maximum zoom value TE can be determined adaptively according to these conditions, for example.

[0263] In a case where it is determined in step S202 to be within the zoomable range, the control unit 7 causes the process to proceed to step S105 and executes zoom adjustment processing. In this case, the processing in step S105 and the subsequent steps illustrated in FIG. 6 are executed, and guidance to adjust the position Ps of the target subject to match the target position Pt is executed. Figure 3

[0264] On the other hand, in a case where it is determined in step S202 not to be within the zoomable range, the control unit 7 executes zoom adjustment processing in step S203. In the zoom adjustment processing in this case, processing of adjustment to the minimum zoom value TR in a case where the size is larger than the target size or the maximum zoom value TE in a case where the size is smaller than the target size is executed.

[0265] Then, in step S204 subsequent to step S203, the control unit 7 determines whether it is a user-movable condition. That is, for example, it is determined whether it is a condition in which the user cannot move forward or backward due to imaging in a crowd or the like. Whether it is a user-movable condition can be determined based on, for example, analysis of an image captured by the imaging unit 2, detected sound (environmental sound around the information processing apparatus 1) obtained by a microphone in the sensor unit 6, or the like.

[0266] In a case where it is determined in step S204 to be a user-movable condition, the control unit 7 proceeds to step S205 and starts size guidance processing. That is, control is executed so that guidance for adjusting the size of the target subject to match the target size is executed by tactile presentation by the tactile presentation unit 10. The guidance is executed in a form in which the user can recognize whether to move the information processing apparatus 1 forward (in a direction of approaching the target subject) or backward at least according to whether the size of the target subject is smaller or larger than the target size.

[0267] Note that the control unit 7 can cause the process to proceed to step S107 illustrated in FIG. 6 after starting the size guidance processing in step S205. Figure 3

[0268] ​​Further, in a case where it is determined in step S204 that it is not the user-movable condition, the control unit 7 executes error notification processing in step S206. That is, a notification is made to the user that it is not possible to adjust the size of the target subject to match the target size. It is conceivable that this notification is executed by tactile presentation by the tactile presentation unit 10.

[0269] Note that a notification prompting the user to change the size of the target region At can be made in the error notification processing of step S206.

[0270] As shown in Figure 22 In this case, the control unit 7 determines whether it is the user-movable condition in a case where it is determined that it is not possible to adjust the size of the target subject to the target size by zoom control, and causes guidance for changing the positional relationship between the user and the target subject to be executed in a case where it is determined that it is the user-movable condition.

[0271] Thus, for example, even in a case where the user is unable to move due to imaging in a crowd or the like, it is possible to prevent the occurrence of inconveniences such as causing the user to disturb surrounding people by executing the guidance.

[0272] Note that, as the error notification related to imaging, it is also conceivable to make a notification regarding so-called finger fogging, blurring, or a large hand shake. Note that finger fogging refers to a state in which at least a part of the finger of the user holding the camera is in contact with the lens or the like to block at least a part of the imaging field of view.

[0273] For example, it is possible to determine whether finger fogging has occurred from the captured image. Further, it is possible to determine whether it is a large hand shake condition from a detection signal obtained by the G sensor or the gyro sensor in the sensor unit 6 or the captured image (for example, the difference between frames). The presence or absence of blurring can be executed as a determination of whether auto focus (AF) has been executed on the specified target subject.

[0274] In a case where it is determined that a finger fogging, blurring, and large hand shake condition has occurred, the control unit 7 executes a predetermined notification as the error notification by tactile presentation by the tactile presentation unit 10.

[0275] Note that it is also conceivable to separately make a notification regarding blurring according to the type of its cause. For example, different error notifications are made between a case where the solution when the user moves the position and a case where the target subject needs to be specified again. In this case, the action that needs to be taken by the user can be notified by tactile presentation, sound, or the like.

[0276] Further, regarding the error notification, it is also possible to notify (for example, notify by tactile presentation) that a specific object (for example, a dangerous article: a bicycle, a car, or the like) is detected in the captured image during the guidance.

[0277] <12. Modification Examples>

[0278] Note that the embodiments are not limited to the above-described specific examples, and configurations as various modification examples can be adopted.

[0279] For example, it is conceivable to identify the user type by the in-camera and switch various settings of the information processing apparatus 1. As an example, for example, if the user is an elderly person, it is conceivable to perform a setting to increase the shutter speed to prevent hand shake.

[0280] Alternatively, if the user is an elderly person, it is also conceivable to perform a setting to increase the vibration intensity during guidance.

[0281] Further, in a case where the user is identified as not being Japanese, the user interface (UI) can be changed to an English version.

[0282] Further, for example, in a case where there are many people who wish to image the same subject such as the same character in a specific place such as a theme park, it is also conceivable to guide them to the best positions respectively to optimize as a whole.

[0283] Further, in the description given so far, an example in which the vibration device for guidance and the camera that obtains a captured image are mounted on an integrated device that is the information processing apparatus 1 has been described, but the vibration device for guidance is not limited to being mounted on an integrated device having a camera that obtains a captured image.

[0284] For example, as in the imaging assistance system 100 shown in Figure 23 , a configuration in which the haptic presentation device 20 that performs vibration presentation for guidance is a separate body can also be adopted. Note that in Figure 23 , the same reference numerals are attached to the same parts as those already explained, and the explanation thereof is omitted.

[0285] As shown in the drawing, the imaging assistance system 100 includes the information processing apparatus 1A and the haptic presentation device 20. The haptic presentation device 20 includes the haptic presentation unit 10, the control unit 21, and the communication unit 22.

[0286] The information processing apparatus 1A is different from the information processing apparatus 1 shown in Figure 1 in that the control unit 7A is provided instead of the control unit 7. Note that in this case, the communication unit 5 is configured to be able to perform data communication with the communication unit 22 in the haptic presentation device 20.

[0287] The control unit 7A is different from the control unit 7 in that the control unit 7A includes the guidance control unit 7bA instead of the guidance control unit 7b.

[0288] The guidance control unit 7bA performs control so that the guidance with respect to the composition change operation is executed by the haptic presentation unit 10 in the haptic presentation device 20 through haptic presentation. Specifically, an instruction to perform haptic presentation for guidance is indicated to the control unit 21 via the communication unit 5, and the control unit 21 causes the haptic presentation unit 10 in the haptic presentation device 20 to execute haptic presentation in accordance with the instruction.

[0289] Here, as the device form of the haptic presentation device 20, for example, a device form of a type worn by a user, such as a smart watch or a glasses-type information processing device, which can communicate with the information processing device 1A such as a smart phone or a tablet terminal, and a device form such as a stand device, such as a tripod that assists the information processing device 1A as a camera, can be conceivable.

[0290] Note that the information processing device 1A does not necessarily include the haptic presentation unit 10 in the imaging assistance system 100.

[0291] Further, in the description given so far, an example in which guidance for imaging by a camera including a screen on which a through image is displayed has been described, but the guidance technique as an embodiment can also be applied to, for example, guidance with respect to a camera (such as an action camera) that does not have a screen on which a through image is displayed.

[0292] Further, in the description given so far, vibration has been exemplified as an example of haptic presentation for guidance, but presentation other than vibration, such as air blowing and the like, can also be executed as haptic presentation.

[0293] <13. SUMMARY OF EMBODIMENTS>

[0294] As described above, the imaging assistance control device (information processing device 1 or 1A) as an embodiment includes a control unit (guidance control unit 7b or 7bA) that performs control so that guidance to a composition change operation by a user is executed through haptic presentation to the user based on a difference between a target composition and an actual composition.

[0295] Therefore, even in a case where guidance through a display is not appropriate in a situation where the user cannot or is difficult to view a screen or even in a case where guidance through sound is not appropriate, it is possible to appropriately guide a composition change operation through haptic presentation.

[0296] Therefore, it is possible to improve the imaging assistance performance.

[0297] Further, in the imaging assistance control device as an embodiment, the control unit changes a mode of haptic presentation for guidance based on a size of a difference between a target composition and an actual composition.

[0298] Accordingly, the user can grasp that the difference between the target composition and the actual composition becomes smaller or larger according to the difference in the pattern of the haptic presentation for guidance.

[0299] Accordingly, even in a situation where the user cannot view or is difficult to view the screen, it is possible to improve the easiness of achieving matching with the target composition, and it is possible to improve the imaging assistance performance.

[0300] Further, in the imaging assistance control device as an embodiment, the target composition is set to a composition that satisfies the following conditions: the target subject is arranged at a position in an image frame that is set according to an operation of the user; the operation related to the setting of the arrangement position of the target subject in the image frame is performed as a drawing operation on a screen (3a) that displays a through image of a captured image, and the control unit determines a setting condition other than the arrangement position in the setting conditions of the target composition according to which operation type the drawing operation belongs to, among a plurality of operation types.

[0301] The operation type of the drawing operation is obtained by classifying the operation pattern that can be performed as the drawing operation, and examples thereof can include the type of the operation of drawing a point, the operation of drawing a circle, the operation of drawing an ellipse, and the like. Further, the setting condition of the target composition refers to a condition for determining the composition of the target composition, and examples thereof include the arrangement position of the target subject in the image frame and a condition for setting a single target subject or a plurality of target subjects.

[0302] According to the above configuration, the user can specify the arrangement position of the target subject by one operation as the drawing operation on the screen, and specify a setting condition other than the arrangement position in the arrangement conditions of the target composition.

[0303] Accordingly, it is possible to reduce the burden of the user operation.

[0304] Further, in the imaging assistance control device as an embodiment, the target composition is set to a composition that satisfies the following conditions: the target subject is arranged at a position in an image frame that is set according to an operation of the user, and the control unit receives the specification of the target subject by voice input from the user.

[0305] Accordingly, the user does not need to operate an operator such as a button or a touch panel in the specification of the target subject.

[0306] Accordingly, when the target subject is specified, the user does not have to search for the target subject on the screen or search for an operator for the specification, which is suitable in a case where imaging is performed without looking at the screen. In particular, it is possible to realize a user interface suitable for a person with impaired vision.

[0307] Further, in the imaging assistance control device as an embodiment, the control unit receives a drawing operation on a screen that displays a through image of a captured image, and determines a condition of a camera that obtains the captured image, and in a case where the condition is a specific condition and a specific mark has been drawn by the drawing operation, sets a composition that satisfies a composition condition to a target composition, the composition condition being associated with the condition and the drawn mark.

[0308] Thus, in a case where the camera is in a specific condition, for example, in a theme park such as an amusement park, when a specific mark (for example, a mark that imitates a famous person in the theme park) is drawn, a composition that satisfies a composition condition associated with the condition and the mark can be set to a target composition, such as a composition in which the person is arranged at a position in an image frame corresponding to the drawn mark.

[0309] Thus, a plurality of composition setting conditions can be executed by one operation called a mark drawing operation, and the burden of user operations can be reduced.

[0310] Further, in the imaging assistance control device as an embodiment, the target composition is set to a composition in which a target subject is arranged at a position in an image frame set according to an operation of a user, an operation that specifies an arrangement range of the target subject in the image frame is executed as a drawing operation on a screen that displays a through image of a captured image, and the control unit determines a number of target subjects to be arranged within the arrangement range according to which operation type of a plurality of operation types the drawing operation belongs to.

[0311] Thus, the user can specify the arrangement range of the target subject and specify the number of target subjects to be arranged within the arrangement range by one operation as a drawing operation on the screen.

[0312] Thus, the burden of user operations can be reduced.

[0313] Further, in the imaging assistance control device as an embodiment, the target composition is set to a composition in which a target subject is arranged at a target position in an image frame, and the control unit detects movement of the target subject, and controls guidance to arrange the target subject at the target position based on the detected movement of the target subject.

[0314] Thus, guidance can be executed considering the movement of the target subject as guidance for realizing a composition in which the target subject is arranged at the target position in the image frame.

[0315] Thus, according to a case where the target subject is a moving subject, the accuracy of the guidance can be improved.

[0316] Further, in the imaging assistance control device as an embodiment, the control unit performs guidance based on a relationship between the target position and a predicted position of the target subject after a predetermined time elapses.

[0317] In a case where the movement of the target subject is large, if guidance based on a difference between the current position of the target subject and the target position is performed, it is difficult to follow the movement of the target subject, and a situation in which the position of the target subject is difficult to adjust to match the target position can occur. Therefore, as described above, guidance is performed based on the position after a predetermined time elapses, which is predicted from the movement of the target subject.

[0318] Therefore, according to a case where the target subject is a moving subject, it is possible to improve the accuracy of guidance.

[0319] Further, in the imaging assistance control device as an embodiment, the control unit sets a target composition based on a determination result of a scene to be imaged.

[0320] Therefore, it is possible to automatically set a target composition based on a scene determination result according to a scene to be imaged, such as a scene of imaging a train at a station or a scene of imaging a sunset.

[0321] Therefore, when imaging is performed with an appropriate target composition according to a scene to be imaged, it is possible to reduce the operational burden of the user.

[0322] Further, in the imaging assistance control device as an embodiment, the target composition is set to a composition in which the target subject is arranged at a target position in an image frame, and the control unit predicts a position in which the target subject is in a target state according to a scene to be imaged, and controls guidance so that the predicted position matches the target position.

[0323] Therefore, it is possible to appropriately assist the user so that the target subject in the target state is imaged with the target composition.

[0324] Therefore, it is possible to improve the user assistance performance related to a change in composition at the time of imaging.

[0325] Further, in the imaging assistance control device as an embodiment, the control unit enables the function of guidance in a case where it has been estimated that the user is not viewing a screen displaying a through image that captures an image.

[0326] Therefore, in a case where the user performs imaging without viewing the screen, guidance according to the present embodiment is automatically performed.

[0327] Therefore, it is possible to appropriately assist the user. Further, even in a case where guidance is not needed, such as a case where the user performs imaging while viewing the screen, it is possible to prevent the guidance function from being enabled, and thus it is possible to improve convenience.

[0328] Further, in the imaging assistance control device as an embodiment, the target composition is set to a composition that satisfies the following condition: the target subject is arranged at a target position in an image frame in a target size, and the control unit performs zoom control so that the size of the target subject becomes the target size.

[0329] Therefore, the need for a zoom operation by the user can be eliminated in the implementation of imaging with the target composition.

[0330] Therefore, the burden of user operation can be reduced.

[0331] Further, in the imaging assistance control device as an embodiment, the control unit determines whether it is a user-movable situation in a case where it is determined that it is not possible to adjust the size of the target subject to the target size by the zoom control, and causes guidance for changing the positional relationship between the user and the target subject to be performed in a case where it is determined that it is a user-movable situation.

[0332] Therefore, for example, even in a situation where the user cannot move due to imaging in a crowd or the like, it is possible to prevent the occurrence of inconveniences such as causing the user to disturb surrounding people by performing the guidance.

[0333] Therefore, the user can be appropriately assisted.

[0334] Further, the imaging assistance control method as an embodiment is an imaging assistance control method including: controlling, by an information processing device, so that guidance to a composition change operation by a user is performed by haptic presentation to the user based on a difference between a target composition and an actual composition.

[0335] With this imaging assistance control method, similar functions and effects to the imaging assistance control device as the above-described embodiment can also be obtained.

[0336] The imaging assistance system (100) as an embodiment includes: a haptic presentation device (20) including a haptic presentation unit (10) that performs haptic presentation to a user; and an imaging assistance control device (information processing device 1A) including a control unit (guidance control unit 7bA) that performs control so that guidance to a composition change operation by the user is performed by haptic presentation of the haptic presentation unit based on a difference between a target composition and an actual composition.

[0337] With this imaging assistance system, similar functions and effects to the imaging assistance control device as the above-described embodiment can also be obtained.

[0338] Note that the effects described in this specification are merely examples and are not limiting, and there can be other effects.

[0339] <14. The present technology>

[0340] The present technology can also have the following configuration. (1)

[0342] An imaging assistance control device includes:

[0343] A control unit performs control such that guidance of a composition change operation by a user is performed through haptic presentation to the user based on a difference between a target composition and an actual composition. (2)

[0345] The imaging assistance control device according to (1) described above, wherein

[0346] The control unit changes a mode of the haptic presentation for the guidance based on a magnitude of the difference between the target composition and the actual composition. (3)

[0348] The imaging assistance control device according to (1) or (2) described above, wherein

[0349] The target composition is set to a composition in which a target subject is arranged at a position in an image frame that is set in accordance with an operation of the user;

[0350] An operation related to the setting of the arrangement position of the target subject in the image frame is performed as a drawing operation on a screen that displays a through image of a captured image, and

[0351] The control unit determines a setting condition other than the arrangement position among the setting conditions of the target composition in accordance with which operation type among a plurality of operation types the drawing operation belongs. (4)

[0353] The imaging assistance control device according to any one of (1) to (3) described above, wherein

[0354] The target composition is set to a composition in which a target subject is arranged at a position in an image frame that is set in accordance with an operation of the user, and

[0355] The control unit receives designation of the target subject through voice input from the user. (5)

[0357] The imaging assistance control device according to any one of (1) to (6) described above, wherein

[0358] The control unit:

[0359] receiving a drawing operation on a screen displaying a through image of a captured image, and

[0360] determining a condition of a camera that obtained the captured image, in a case where the condition is a specific condition and a specific mark is drawn by the drawing operation, setting a composition that satisfies a composition condition to the target composition, the composition condition being associated with the condition and the drawn mark. (6)

[0362] The imaging assistance control device according to any one of (1) to (5) described above, in which

[0363] the target composition is set to a composition in which a target subject is arranged at a position in an image frame set according to a user operation;

[0364] the operation of specifying a range of arrangement of the target subject in the image frame is performed as a drawing operation on a screen displaying a through image of a captured image, and

[0365] the control unit determines a number of the target subjects to be arranged within the range of arrangement according to which operation type of a plurality of operation types the drawing operation belongs to. (7)

[0367] The imaging assistance control device according to any one of (1) to (6) described above, in which

[0368] the target composition is set to a composition in which a target subject is arranged at a target position in an image frame, and

[0369] the control unit detects movement of the target subject, and controls guidance for arranging the target subject at the target position based on the detected movement of the target subject. (8)

[0371] The imaging assistance control device according to (7) described above, in which

[0372] the control unit performs the guidance based on a relationship between a predicted position of the target subject after a predetermined time elapses and the target position. (9)

[0374] The imaging assistance control device according to (1) or (2) described above, in which

[0375] the control unit sets the target composition based on a determination result of a scene to be imaged. (10)

[0377] The imaging-assist control device according to (9) above, in which

[0378] The target composition is set to a composition that satisfies a condition that a target subject is arranged at a target position in an image frame, and

[0379] The control unit predicts a position of the target subject in a target state according to the scene to be imaged, and controls the guidance so that the predicted position coincides with the target position. (11)

[0381] The imaging-assist control device according to any one of (1) to (10) above, in which

[0382] The control unit enables a function of the guidance in a case where it is estimated that the user is not watching a screen displaying a through image that captures an image. (12)

[0384] The imaging-assist control device according to any one of (1) to (11) above, in which

[0385] The target composition is set to a composition that satisfies a condition that a target subject is arranged at a target position in an image frame, and

[0386] The control unit performs zoom control so that a size of the target subject becomes the target size. (13)

[0388] The imaging-assist control device according to (12) above, in which

[0389] The control unit:

[0390] In a case where it is determined that the size of the target subject cannot be adjusted to the target size by the zoom control, it is determined whether it is a condition that the user is movable, and in a case where it is determined that it is a condition that the user is movable, the guidance for changing a positional relationship between the user and the target subject is caused to be performed. (14)

[0392] An imaging-assist control method including:

[0393] A control is performed by an information processing apparatus so that guidance of a composition change operation by a user is performed through haptic presentation to the user based on a difference between a target composition and an actual composition. (15)

[0395] An imaging-assist system including:

[0396] A haptic presentation device including a haptic presentation unit that performs haptic presentation to a user; and

[0397] An imaging assistance control device including a control unit that performs control so that guidance of a composition change operation by the user is performed by haptic presentation of the haptic presentation unit based on a difference between a target composition and an actual composition.

[0398] List of reference signs

[0399] 1, 1A information processing device

[0400] 2 imaging unit

[0401] 3 display unit

[0402] 3a screen

[0403] 4 operation unit

[0404] 5 communication unit

[0405] 6 sensor unit

[0406] 7, 7A control unit

[0407] 7a image recognition processing unit

[0408] 7b, 7bA guidance control unit

[0409] 8 memory unit

[0410] 9 sound output unit

[0411] 10 haptic presentation unit

[0412] 11 bus

[0413] At target region

[0414] Pt target position

[0415] Ps position of a target subject

[0416] I1 subject designation information

[0417] 15 vibration device

[0418] 20 haptic presentation device

[0419] 21 control unit

[0420] 22 communication unit

[0421] 100 imaging assistance system

Claims

1. An imaging-assisted control device, comprising: A control unit performs control such that, based on the difference between the target composition and the actual composition, it guides the user's compositional change operation by presenting it to the user's sense of touch. The control unit: Receive drawing operations for the screen displaying the captured image through the image, and The condition of the camera that acquired the captured image is determined. If the condition is a specific condition and a specific mark has been drawn through the drawing operation, a composition that satisfies the composition conditions is set as the target composition, the composition conditions being associated with the condition and the drawn mark.

2. The imaging-assisted control device according to claim 1, wherein, The control unit changes the mode of tactile presentation used for guidance based on the magnitude of the difference between the target composition and the actual composition.

3. The imaging-assisted control device according to claim 1, wherein, The target composition is set to satisfy the following conditions: the target subject is placed in the image frame at a position set according to the user operation; Operations related to setting the placement position of the target subject in the image frame are performed as drawing operations for a screen displaying a through image of the captured image, and The control unit determines the setting conditions of the target composition other than the arrangement position based on which of the multiple operation types the drawing operation belongs to.

4. The imaging-assisted control device according to claim 1, wherein, The target composition is set to satisfy the following conditions: the target subject is positioned within the image frame at a location determined by user operation, and The control unit receives the designation of the target subject via voice input from the user.

5. The imaging-assisted control device according to claim 1, wherein, The target composition is set to satisfy the following conditions: the target subject is placed in the image frame at a position set according to the user operation; The operation of specifying the arrangement range of the target subject in the image frame is performed as a drawing operation for the screen displaying the through image of the captured image, and The control unit determines the number of target subjects to be arranged within the arrangement range based on which of a plurality of operation types the drawing operation belongs to.

6. The imaging-assisted control device according to claim 1, wherein, The target composition is set to satisfy the following conditions: the target subject is positioned at the target location within the image frame, and The control unit detects the movement of the target subject and controls the guidance for placing the target subject at the target location based on the detected movement of the target subject.

7. The imaging-assisted control device according to claim 6, wherein, The control unit performs the guidance based on the relationship between the predicted position of the target subject after a predetermined time and the target position.

8. The imaging-assisted control device according to claim 1, wherein, The control unit sets the target composition based on the determination of the scene to be imaged.

9. The imaging-assisted control device according to claim 8, wherein, The target composition is set to satisfy the following conditions: the target subject is positioned at the target location within the image frame, and The control unit predicts the position of the target subject in the target state according to the scene to be imaged, and controls the guidance so that the predicted position is consistent with the target position.

10. The imaging-assisted control device according to claim 1, wherein, The control unit activates the guidance function when it anticipates that the user is not viewing a passthrough image of the captured image.

11. The imaging-assisted control device according to claim 1, wherein, The target composition is set to satisfy the following conditions: the target subject is positioned at the target location in the image frame at the target size, and The control unit performs zoom control, causing the size of the target subject to change to the target size.

12. The imaging-assisted control device according to claim 11, wherein, The control unit: If it is determined that the size of the target subject cannot be adjusted to the target size by means of the zoom control, it is determined whether the user is movable, and if it is determined that the user is movable, the guidance for changing the positional relationship between the user and the target subject is executed.

13. An imaging-assisted control method, comprising: Controlled by an information processing device, the system guides the user's composition-changing operations by presenting the difference between the target composition and the actual composition to the user's sense of touch. The information processing device: Receive drawing operations for the screen displaying the captured image through the image, and The condition of the camera that acquired the captured image is determined. If the condition is a specific condition and a specific mark has been drawn through the drawing operation, a composition that satisfies the composition conditions is set as the target composition, the composition conditions being associated with the condition and the drawn mark.

14. An imaging-assisted system, comprising: A tactile presentation device, including a tactile presentation unit that performs tactile presentation to a user; as well as An imaging assist control device includes a control unit that performs control to guide a user's composition-changing operation based on the difference between the target composition and the actual composition through haptic presentation by the haptic presentation unit. The control unit: Receive drawing operations for the screen displaying the captured image through the image, and The condition of the camera that acquired the captured image is determined. If the condition is a specific condition and a specific mark has been drawn through the drawing operation, a composition that satisfies the composition conditions is set as the target composition, the composition conditions being associated with the condition and the drawn mark.

Images