electronic machinery

Abstract

The objective is to enable the provision of useful tactile feedback. [Solution] The electronic device of the present invention has a determination means for determining the tactile sensation to be presented based on a plurality of data relating to the shooting, and the determination means determines different tactile sensations to be presented depending on the shooting conditions. For example, the electronic device of the present invention has a determination means for determining the tactile sensation to be presented by combining a first tactile sensation based on the characteristics of the subject in the captured image and a second tactile sensation predetermined for the subject and having a higher level of abstraction than the first tactile sensation, and the determination means determines the tactile sensation to be presented such that the higher the clarity of the subject in the captured image, the closer it is to the first tactile sensation, and the lower the clarity, the closer it is to the second tactile sensation.

Description

Technical Field

[0001] The present invention relates to electronic devices, and particularly to a technology for presenting a sense of touch.

Background Art

[0002] Technologies for presenting various senses of touch to users have been proposed. Technologies for presenting a sense of force such as vibration and pressure, technologies for presenting a surface shape by deforming or displacing an object, technologies for presenting a sense of warmth by temperature, etc. have been proposed. In these tactile presentations, users can intuitively grasp information more than in information presentation by display or voice.

[0003] Patent Document 1 discloses an imaging device that presents a sense of touch representing the focus state of a subject to a user based on frequency components (edge information and high-frequency components) of a captured image. While the user checks the captured image on a display, the user touches the touch panel with a finger so as to select a subject (target subject) to be focused. The imaging device performs a focusing operation on the selected target subject, extracts edge information and high-frequency components of the target subject from the captured image, creates tactile information, and repeats an operation of presenting a sense of touch. Thereby, when the user takes a photograph (records an image), the user can confirm the focus state of the target subject by touch.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the imaging device described in Patent Document 1, tactile feedback representing edges (contours) is presented based on edge information and high-frequency components of the captured image, and in the case of out-of-focus images, tactile feedback is not presented or is presented weakly. Therefore, it is difficult for the user to understand (recognize) that the image is out of focus, how far the focal position is shifted, or what the object of the focusing operation is.

[0006] Furthermore, since users generally only touch the touch panel for a short time to select a subject, the focusing process is completed after the user has lifted their finger from the touch panel. As a result, the user cannot obtain useful information about the focusing process (the final result of the focusing operation). This problem becomes particularly apparent when the focusing operation is slow.

[0007] Furthermore, when shooting in environments with varying brightness, the characteristics of the captured image change, making it impossible to provide stable tactile feedback, even if the degree of focus remains unchanged, as the presented tactile sensations may change.

[0008] The present invention aims to enable useful tactile presentation. [Means for solving the problem]

[0009] A first aspect of the present invention is an electronic device having a determination means for determining the tactile sensation to be presented based on a plurality of data relating to the shooting, wherein the determination means determines different tactile sensations to be presented depending on the shooting conditions.

[0010] A second aspect of the present invention involves combining a first tactile sensation based on the characteristics of a subject in an captured image with a second tactile sensation predetermined for the subject and having a higher level of abstraction than the first tactile sensation. The electronic device is characterized by having a determination means for determining the tactile sensation to be presented, wherein the determination means determines the tactile sensation to be presented as such that the higher the clarity of the subject in the captured image, the closer it is to the first tactile sensation, and the lower the clarity, the closer it is to the second tactile sensation.

[0011] A third aspect of the present invention is a control method for an electronic device, comprising a decision step of determining a tactile sensation to present based on a plurality of data relating to shooting, wherein the decision step determines a tactile sensation that differs depending on the shooting conditions.

[0012] A fourth aspect of the present invention is a program for causing a computer to function as one of the means of the electronic device described above. A fifth aspect of the present invention is a computer-readable storage medium that stores a program for causing a computer to function as one of the means of the electronic device described above. [Effects of the Invention]

[0013] According to the present invention, useful tactile feedback can be provided. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows the hardware configuration of the imaging device according to the first embodiment. [Figure 2] This figure shows the functional configuration of the imaging device according to the first embodiment. [Figure 3] This diagram shows the operation of the imaging device according to the first embodiment. [Figure 4] This figure shows the hardware configuration of the imaging system according to the second embodiment. [Figure 5] This figure shows the functional configuration of the imaging system according to the second embodiment. [Figure 6] This diagram shows the operation of the imaging system according to the second embodiment. [Modes for carrying out the invention]

[0015] Hereinafter, two embodiments of the present invention will be described. In the following embodiments, similar to the prior art, a focus target (a subject of the focusing operation) is selected using a touch panel, and a focusing operation is performed and a tactile presentation is performed. The first embodiment is an example using only an imaging device, and the second embodiment is an example using an imaging device and an operating device.

[0016] <First Embodiment> The first embodiment will be described. FIG. 1 is a block diagram showing the hardware configuration of an imaging device 1 according to the first embodiment. The imaging device 1 is a smartphone, a digital camera, or the like, and includes an arithmetic unit 101, an information processing unit 102, a primary storage device 103, a secondary storage device 104, an optical device 105, an input device 106, an output device 107, and a bus 110. The arithmetic unit 101 is a CPU (Central Processing Unit) or the like, and the information processing unit 102 is an information processing LSI (Large-Scale Integration) or the like. The primary storage device 103 is a volatile main storage device or the like, and the secondary storage device 104 is a non-volatile auxiliary storage device or the like. The optical device 105 is a lens, an image sensor, or the like, the input device 106 is a touch panel, a button, or the like, and the output device 107 is a tactile presentation device, a display, or the like.

[0017] FIG. 2 is a block diagram showing the hardware configuration, functional configuration, and data transmission / reception of the imaging device 1 according to the first embodiment.

[0018] The arithmetic unit 101 controls other hardware. Operations, parameters, etc. of each component shown in FIG. 2 are controlled by the arithmetic unit 101.

[0019] The information processing unit 102 includes an optical control unit 102a, an image processing unit 102b, an image analysis unit 102 c, and a tactile operation unit 102d. The optical control unit 102a performs control for imaging and focusing (control of the lens and image sensor). The image processing unit 102b performs processing using the image (captured image) captured by the optical device 105 and the like. The image analysis unit 102c analyzes the image before or after the processing by the image processing unit 102b. The tactile operation unit 102d generates tactile data (tactile information) representing the tactile sensation to be presented.

[0020] The primary storage device 103 has an image storage unit 103a and a tactile data storage unit 103b. These temporarily store image data, tactile data, and the like used by the arithmetic unit 101 and the information processing unit 102.

[0021] The secondary storage device 104 has an image storage unit 104a and a tactile data storage unit 104b. These store image data, tactile data, and the like used by the arithmetic unit 101 and the information processing unit 102.

[0022] The optical device 105 performs imaging and driving of the lens related to the focusing operation and the like.

[0023] The input device 106 inputs instructions from the user to the arithmetic unit 101 and the information processing unit 102.

[0024] The output device 107 has an image display unit 107a and a tactile presentation unit 107b. The image display unit 107a displays the image after the processing by the information processing unit 102, various information, and the like. The tactile presentation unit 107b presents the tactile sensation based on the tactile data generated by the information processing unit 102 to the user. The method of presenting the tactile sensation by the tactile presentation unit 107b is not particularly limited, and various known presentation methods can be used. For example, the tactile presentation unit 107b may be a vibrator that presents the tactile sensation by vibration. The tactile presentation unit 107b may present the texture such as smoothness and roughness, hardness, and the like. The tactile presentation unit 107b may present the temperature.

[0025] Each component of the imaging device 1 can exchange data via the bus 110.

[0026] The operation of the imaging device 1 according to the first embodiment will now be described. The operation of the imaging device 1 includes the following four stages: LV shooting and display, focus control, and shooting (image recording), which are similar to the operation of a general digital camera. The first embodiment is characterized by tactile calculation and presentation. • LV (Live View) shooting and display • Focus control • Tactile processing and presentation • Photography (recording images)

[0027] Figure 3 is a flowchart showing the operation of the imaging device 1 according to the first embodiment. When the user operates an input device 106 such as a button or switch, and the operating mode of the imaging device 1 switches to the shooting mode, the operation shown in Figure 3 begins.

[0028] Steps S201 to S204 involve operations related to LV shooting and display.

[0029] In step S201, the optical control unit 102a of the information processing device 102 controls the optical device 105 to capture an LV image.

[0030] In step S202, the optical device 105 captures an LV image.

[0031] In step S203, the image processing unit 102b of the information processing device 102 performs predetermined image processing on the LV image.

[0032] In step S204, the image display unit 107a of the output device 107 displays the LV image after image processing in step S203.

[0033] The operations in steps S205 to S210 relate to focus control.

[0034] In step S205, the arithmetic unit 101 determines whether or not a subject selection instruction has been given to the input device 106. A subject selection instruction is a user instruction to select a subject to be focused on, and is achieved by touching the touch panel to touch the subject in the displayed LV image. If a subject selection instruction has been given, the process proceeds to step S206; otherwise, the process proceeds to step S217.

[0035] In step S206, the image analysis unit 102c and image processing unit 102b of the information processing device 102 analyze and process the LV image to set the subject of interest, which is the subject to be focused on. The method for setting the subject of interest is not particularly limited, and the subject of interest can be set using various known subject recognition processes. The subject recognition process may be a process using a trained model obtained by machine learning.

[0036] In step S207, the image analysis unit 102c of the information processing device 102 outputs information about the subject of interest to the optical control unit 102a. The output information includes, for example, distance information (distance data) from the imaging device 1 to the subject of interest.

[0037] In step S208, the optical control unit 102a of the information processing device 102 calculates the lens control amount for focusing and outputs the lens control amount to the optical device 105. The type of focusing operation may be autofocus, in which the optical control unit 102a automatically calculates the lens control amount for focusing, or manual focus, in which the user directly inputs the lens control amount.

[0038] In step S209, the arithmetic unit 101 determines whether the focusing operation is complete or not. If focusing is not complete, the process proceeds to step S210; if focusing is complete, the process proceeds to step S211. Note that the processing from step S211 onwards is performed not only when focusing is complete, but also during the focusing operation (focus control). The processing from step S211 onwards is performed in parallel with the focusing operation.

[0039] In step S210, the optical device 105 drives the lens with a drive amount corresponding to the lens control amount obtained in step S208.

[0040] The actions in steps S211 to S216 relate to tactile processing and presentation.

[0041] In step S211, the arithmetic unit 101 determines whether or not a subject selection instruction (touch) has been made to the input device 106. If the subject selection instruction is still being made, the process proceeds to step S212; otherwise, the process proceeds to step S217.

[0042] In step S212, the tactile processing unit 102d of the information processing device 102 requests the image processing unit 102b to provide first feature data relating to the sharpness of the subject of interest in the LV image (captured image). For example, the first feature data requested may be feature data indicating the degree of focus of the subject of interest. Note that the first feature data may be any feature relating to the subject of interest, and is not limited to feature data relating to sharpness or degree of focus. For example, the first feature data The data may also be feature data relating to the degree of blur, exposure, sensitivity, noise level, contrast, degree of aberration, image resolution, and level of abstraction of the subject of interest. If the first feature data represents multiple types of features, the tactile calculation unit 102d may use the multiple types of features to calculate the sharpness of the subject of interest. In this case, the sharpness may be calculated using a specific feature such as the degree of focus, and the remaining features may be used to adjust (correct) the calculated sharpness.

[0043] In step S213, the tactile processing unit 102d of the information processing device 102 requests second feature data relating to the features of the subject of interest in the LV image (captured image) from the image processing unit 102b. Furthermore, the tactile processing unit 102d requests tactile data from the tactile data storage unit 103b of the primary storage device 103, or from the tactile data storage unit 104b of the secondary storage device 104. The request to the secondary storage device 104 may be made via the primary storage device 103.

[0044] The second set of feature data includes, for example, subject recognition result data of the subject of interest (data indicating the subject recognition result such as the category, size, position, and orientation of the subject of interest), color data, brightness data, edge data, and shape data. Distance data from the imaging device 1 to the subject of interest may be acquired as at least a part of the second set of feature data, or as data separate from the second set of feature data.

[0045] Furthermore, multiple tactile data sets corresponding to multiple categories of subjects are predetermined, and the tactile data requested from the primary storage device 103 and the secondary storage device 104 are predetermined tactile data corresponding to the category of the subject of interest. This tactile data is, for example, data relating to surface characteristics, and when tactile sensation is presented by vibration, it indicates vibration intensity, vibration frequency, vibration duration, etc. The predetermined tactile data is generated in advance, for example, using a trained model obtained by machine learning.

[0046] In step S214, the image processing unit 102b of the information processing device 102 outputs second feature data to the tactile calculation unit 102d in response to the request in step S213. In addition, either the tactile data storage unit 103b of the primary storage device 103 or the tactile data storage unit 104b of the secondary storage device 104 outputs predetermined tactile data to the tactile calculation unit 102d in response to the request in step S213.

[0047] In step S215, the tactile processing unit 102d of the information processing device 102 converts the second feature data input from the image processing unit 102b in step S214 into tactile data. The tactile processing unit 102d then combines the tactile data obtained by converting the second feature data with predetermined tactile data input from the primary storage device 103 or secondary storage device 104 in step S214, based on clarity (first feature data related to clarity). This combination generates the presented tactile data.

[0048] For example, when presenting tactile sensations through vibration, the tactile calculation unit 102d determines the vibration intensity according to the edge intensity in the LV image, the vibration frequency according to the image frequency, and the vibration time according to the distance from the imaging device 1 to the subject of interest. This converts the second feature data into tactile data representing vibration intensity, vibration frequency, and vibration time. The conversion may be performed using a table showing the correspondence between edge intensity, image frequency, and distance and vibration intensity, vibration frequency, and vibration time, or it may be performed using calculation formulas such as multiplying edge intensity, image frequency, and distance by predetermined weights. More advanced conversions (e.g., conversions using subject recognition result data, color data, brightness data, shape data, etc.) may be performed as part of the conversion from the second feature data to tactile data. When presenting temperature, the temperature to be presented may be determined using color data or brightness data. The conversion from the second feature data to tactile data is performed for each coordinate of the subject of interest. The region (each coordinate) can be determined using object recognition result data, color data, brightness data, shape data, etc.

[0049] The presented tactile data is the final tactile data used to control the tactile presentation unit 107b and present tactile sensations. When tactile sensations are presented by vibration, presented tactile data indicating vibration intensity, vibration frequency, vibration duration, etc., is generated for each coordinate of the displayed image.

[0050] For the region of the subject of interest, the value of the presented tactile data is calculated, for example, using the following formula. In the following formula, Vi1 is the vibration intensity indicated by predetermined tactile data from the primary memory device 103 or the secondary memory device 104. Vf1 is the vibration frequency indicated by the predetermined tactile data. Vt1 is the vibration duration indicated by the predetermined tactile data. Vi2 is the vibration intensity indicated by the tactile data converted from the second feature data. Vf2 is the vibration frequency indicated by the tactile data converted from the second feature data. Vt2 is the vibration duration indicated by the tactile data converted from the second feature data. α is a normalized value of sharpness, focus, etc., so that the maximum value is 1. ViO is the vibration intensity indicated by the presented tactile data. VfO is the vibration frequency indicated by the presented tactile data. VtO is the vibration duration indicated by the presented tactile data. According to the following formula, ViO is calculated by combining Vi1 and Vi2 with weights based on α, and VfO and VtO are calculated by similar weighted combination. For regions other than the region of the subject of interest, for example, ViO=VfO=VtO=0. ViO = (1-α) × Vi1 + α × Vi2 VfO = (1-α) × Vf1 + α × Vf2 VtO = (1-α) × Vt1 + α × Vt2

[0051] According to the above calculation formula, the values ​​of ViO, VfO, and VtO are closer to Vi1, Vf1, and Vt1 when the sharpness and degree of focus are low, and closer to Vi2, Vf2, and Vt2 when the sharpness and degree of focus are high. In the first embodiment, Vi1, Vf1, and Vt1 are generalized values ​​that are more abstract than Vi2, Vf2, and Vt2. ViO, VfO, and VtO correspond to the presented tactile sensations, Vi1, Vf1, and Vt1 correspond to tactile sensations with a higher degree of abstraction, and Vi2, Vf2, and Vt2 correspond to tactile sensations based on the features of the subject of interest in the LV image.

[0052] Using the haptic data generated in this way, it is possible to present tactile sensations that are easily perceived by the user, whether the degree of focus is high or low, and the level of abstraction of the presented tactile sensation changes depending on the change in the degree of focus. Therefore, the user can understand the degree of focus not only when the subject of interest is in focus, but also when it is not. Furthermore, the user can understand whether the subject of interest is set correctly, or what the subject of interest is if it is not set correctly. It is also possible to allow the user to intuitively understand changes in the degree of focus.

[0053] The method for generating the presented tactile data (the method for determining the tactile sensation to be presented) is not limited to the method described above. For example, the data used to generate the presented tactile data and the method for generating the presented tactile data may be changed depending on the type of feature indicated by the first feature data and the type of tactile sensation to be presented. Fixed data that is independent of the subject (tactile data with a fixed pattern) may be used as the predetermined tactile data. When repeatedly generating the presented tactile data, the method for generating the presented tactile data may be changed continuously or stepwise. The presented tactile data is not limited to data indicating vibration intensity, vibration frequency, and vibration time. Data indicating various information such as tactile frequency, phase, particle size, hardness, temperature, characteristics, presentation amount, and presentation time may be used as presented tactile data. It can be used. The presented quantity corresponds to the amount of energy presented per predetermined time.

[0054] Returning to the explanation of Figure 3, in step S216, the tactile presentation unit 107b of the output device 107 presents tactile sensations based on the presented tactile data generated in step S215.

[0055] The operations in steps S217 to S221 relate to image capture (recording of images).

[0056] In step S217, the arithmetic unit 101 determines whether or not a shooting instruction has been given to the input device 106. If a shooting instruction has been given, the process proceeds to step S218; otherwise, the process proceeds to step S222.

[0057] In step S218, the optical control unit 102a of the information processing device 102 controls the optical device 105 to capture a recording image.

[0058] In step S219, the optical device 105 captures a recording image. The recording image may be a still image or a video.

[0059] In step S220, the image processing unit 102b of the information processing device 102 performs predetermined image processing on the recorded image.

[0060] In step S221, the information processing device 102 records the recorded image after image processing in step S220 in the image storage unit 104a of the secondary storage device 104.

[0061] In step S222, the arithmetic unit 101 determines whether or not a shooting termination instruction has been given to the input device 106. If a shooting termination instruction has been given, the arithmetic unit 101 terminates the shooting mode, and the operation shown in Figure 3 ends. If no shooting termination instruction has been given, the process proceeds to step S201.

[0062] According to the first embodiment described above, based on multiple data points, different tactile sensations are determined to be presented depending on the shooting conditions. This enables the presentation of useful tactile sensations. For example, if the subject of interest is not in focus, a pre-prepared, highly abstract tactile sensation can be determined, and if the subject of interest is in focus, a tactile sensation corresponding to the characteristics of the subject of interest in the LV image (a tactile sensation faithful to the characteristics of the subject of interest) can be determined. As a result, even when the subject of interest is not in focus, or when focusing takes time, useful tactile sensations can be presented that allow the user to grasp the subject of interest and its degree of focus.

[0063] Furthermore, according to the calculation formula for the presented tactile data described above, when the shooting conditions (first feature data) change, the presented tactile sensation is continuously changed in response to the change in shooting conditions, but this is not limited to this. When the shooting conditions change, the presented tactile sensation may be changed in stages in response to the change in shooting conditions.

[0064] The shooting conditions are not limited to the first feature data, but may also include the mode of the imaging device 1, the driving status of the imaging device 1, the settings of the imaging device 1, and the processing content of the imaging device 1. The mode of the imaging device 1 may be a focus mode or an exposure mode, and different tactile presentation data may be generated depending on whether the focus mode is a focus mode or a manual focus mode. If the exposure mode is an aperture priority mode, the aperture value may be used to generate the tactile presentation data. The driving status of the imaging device 1 may be, for example, the driving status of the optical device 105 that represents the progress of the focusing operation. By considering (referencing) the driving status of the optical device 105 that represents the progress of the focusing operation, tactile presentation data that matches the progress of the focusing operation can be generated without analyzing the captured image. Data can be generated. The settings of the imaging device 1 are, for example, the state of various parameters (imaging conditions), including a parameter that specifies the mode of the imaging device 1. The processing content of the imaging device 1 corresponds, for example, to the program used in the imaging device 1.

[0065] <Second Embodiment> A second embodiment will now be described. Note that in the following description, the same configurations and processes as in the first embodiment will be omitted, and only configurations and processes different from those in the first embodiment will be described.

[0066] Figure 4 is a block diagram showing the hardware configuration of an imaging system according to the second embodiment. The imaging system includes an imaging device 1 and an operating device 3. The imaging device 1 is a robot or drone with imaging capabilities, and the operating device 3 is an external device for the imaging device 1 and acts as its controller. A smartphone may be used as the imaging device 1, and another smartphone may be used as the operating device 3. For example, the imaging device 1 and the operating device 3 are located separately, and the user operates the remotely located imaging device 1 using the operating device 3.

[0067] The imaging device 1 includes a computing device 101, an information processing device 102, a primary storage device 103, a secondary storage device 104, an optical device 105, a communication device 108, and a bus 110. The operating device 3 includes a computing device 301, an information processing device 302, an input device 306, an output device 307, a communication device 308, and a bus 310. The computing device 301 is a CPU (Central Processing Unit), the input device 306 is a touch panel or buttons, and the output device 307 is a haptic feedback device or display. The communication device 108 and the communication device 308 communicate with each other.

[0068] Figure 5 is a block diagram showing the hardware configuration, functional configuration, and data transmission / reception of the imaging device 1 according to the second embodiment.

[0069] User instructions for input device 306 are transmitted from communication device 308 to communication device 108.

[0070] The image data processed by the image processing unit 102b is transferred from the communication device 108 to the communication device 308. Before the image data is transmitted, the image processing unit 102b performs compression processing to convert the image data into a format suitable for communication. After the image data is transmitted, the image processing unit 302b of the information processing device 302 performs decompression processing to convert the image data into a format suitable for processing within the operating device 3. The image display unit 307a of the output device 307 displays an image (e.g., an LV image) based on the decompressed image data.

[0071] The presented tactile data generated by the tactile processing unit 102d is transferred from the communication device 108 to the communication device 308. Before the communication of the presented tactile data, the tactile processing unit 102d performs compression processing to convert the format of the presented tactile data into a format suitable for communication. After the communication of the presented tactile data, the tactile processing unit 302d of the information processing device 302 performs decompression processing to convert the format of the presented tactile data into a format suitable for processing within the operating device 3. The tactile presentation unit 307b of the output device 307 presents tactile feedback based on the decompressed tactile data.

[0072] Figure 6 is a flowchart showing the operation of the imaging system according to the second embodiment. When the user operates an input device 306 such as a button or switch, and the operating mode of the imaging system switches to the shooting mode, the operation shown in Figure 6 begins.

[0073] Steps S401 to S403 are the same as steps S201 to S203 in Figure 3. In step S404, the communication device 108 transmits the LV image after image processing in step S403 to the operating device 3 (communication device 308). In step S405, the output device 3 The image display unit 307a in step 07 displays the LV image transmitted in step S404.

[0074] In step S406, the arithmetic unit 301 determines whether or not a subject selection instruction has been given to the input device 306. If a subject selection instruction has been given, the process proceeds to step S407; otherwise, the process proceeds to step S421. In S407, the communication device 308 transmits the subject selection instruction to the imaging device 1 (communication device 108). Steps S408 to S412 are the same as steps S206 to S210 in Figure 3.

[0075] In step S413, the arithmetic unit 301 determines whether the subject selection instruction (touch) to the input device 306 is continuing. If the subject selection instruction is continuing, the process proceeds to step S414; otherwise, the process proceeds to step S421. In S414, the communication device 308 transmits the subject selection instruction to the imaging device 1 (communication device 108). Steps S415 to S418 are the same as steps S212 to S215 in Figure 3. In step S419, the communication device 108 transmits the tactile presentation data generated in step S418 to the operation device 3 (communication device 308). In step S420, the tactile presentation unit 307b of the output device 307 presents tactile feedback based on the presented tactile data transmitted in step S419.

[0076] In step S421, the arithmetic unit 301 determines whether or not a shooting instruction has been given to the input device 306. If a shooting instruction has been given, the process proceeds to step S422; otherwise, the process proceeds to step S427. In step S422, the communication device 308 transmits the shooting instruction to the imaging device 1 (communication device 108). Steps S423 to S426 are the same as steps S218 to S221 in Figure 3. In S427, the arithmetic unit 301 determines whether or not a shooting end instruction has been given to the input device 306. If a shooting end instruction has been given, the arithmetic unit 301 terminates the shooting mode, and the operation in Figure 6 ends. If a shooting end instruction has not been given, the process proceeds to step S401.

[0077] In the second embodiment, the imaging device 1 and the operating device 3 are located separately, and the imaging device 1 is assumed to be a freely movable drone or the like. Therefore, in the second embodiment, the shooting conditions (first characteristic data) such as the sharpness, degree of focus, degree of blur, exposure, sensitivity, noise level, contrast, degree of aberration, image resolution, and abstraction of the LV image may change significantly compared to the first embodiment.

[0078] Therefore, in the second embodiment, when generating the presented haptic data, a first feature data showing multiple features is acquired, and the features to be used are switched to use features that have relatively small temporal changes (features that change less over time compared to other features). Similarly, a second feature data showing multiple features is acquired, and the features to be used are switched to use features that have relatively small temporal changes. By doing this, stable haptic presentation can be achieved even when the external environment (e.g., brightness) changes significantly.

[0079] According to the second embodiment described above, in a configuration in which the imaging device and the operating device work together, useful haptic feedback can be provided, similar to the first embodiment. Furthermore, by coordinating the imaging device 1 and the operating device 3, haptic feedback can be provided to a remote user of the imaging device. In addition, by using features that change relatively little over time, stable haptic feedback can be provided even when the external environment changes significantly due to the movement of the imaging device or other factors.

[0080] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). Multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) may share the processing. By taking on this responsibility, the entire device may be controlled.

[0081] Furthermore, the above-mentioned processors are processors in a broad sense, including general-purpose processors and specialized processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Specialized processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0082] Furthermore, although embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0083] For example, data necessary for generating tactile presentation data may be transmitted from the imaging device to an external device such as an operating device, and the tactile presentation data may be generated in the external device. Alternatively, the above-described process may be shared and executed by three or more devices (electronic devices).

[0084] The present invention is applicable to a variety of electronic devices. For example, it may be applied to personal computers, smartphones, tablet devices, wearable devices (e.g., head-mounted displays), digital still cameras, digital video cameras, etc. It may also be applied to vehicles (e.g., automobiles), aircraft (e.g., drones), various robots, etc.

[0085] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit that implements one or more functions.

[0086] This embodiment includes the following configurations, methods, programs, and media. (Composition 1) A decision-making mechanism for determining the tactile sensation to present, based on multiple data related to the photograph. It has, The aforementioned determination means determines different tactile sensations to be presented depending on the shooting conditions. An electronic device characterized by the following features. (Configuration 2) The determination means continuously or stepwise changes the presented tactile sensation in response to changes in the shooting conditions. The electronic device according to configuration 1, characterized by the features described above. (Composition 3) The aforementioned shooting conditions include at least one of the following: first feature data relating to the first features of the subject in the captured image, the mode of the imaging device, the operating status of the imaging device, the settings of the imaging device, and the processing content of the imaging device. The electronic device according to configuration 1 or 2, characterized by the above. (Composition 4) The first characteristic includes at least one of the degree of focus, degree of blur, exposure, sensitivity, noise level, contrast, degree of aberration, image resolution, sharpness, and degree of abstraction of the subject. The electronic device according to configuration 3, characterized by the features described above. (Composition 5) The plurality of data includes at least one of the following: second feature data relating to a second feature of the subject in the captured image, predetermined data corresponding to the category of the subject, distance data from the imaging device to the subject, and predetermined fixed data. An electronic device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The second feature data includes at least one of the subject recognition result data, color data, brightness data, edge data, and shape data. The electronic device according to configuration 5, characterized by the features described herein. (Composition 7) The aforementioned first feature data shows multiple features, The determination means switches the features to be used so as to determine the tactile sensation to be presented by using the feature among the plurality of features whose temporal change is smaller than that of the other features. The electronic device according to configuration 3 or 4, characterized by the above. (Composition 8) The determination means determines at least one of the tactile frequency, phase, particle size, hardness, temperature, characteristics, presentation amount, and presentation time. An electronic device according to any one of configurations 1 to 7, characterized by the above. (Composition 9) Presentation means for presenting the tactile sensation determined by the determination means. It further possesses An electronic device according to any one of configurations 1 to 8, characterized by the above. (Composition 10) Output means for outputting the aforementioned tactile data to an external device. It further possesses An electronic device according to any one of configurations 1 to 8, characterized by the above. (Composition 11) The aforementioned electronic device is an imaging device. An electronic device according to any one of configurations 1 to 10, characterized by the above. (Composition 12) The aforementioned electronic device is an external device of the imaging device. An electronic device according to any one of configurations 1 to 10, characterized by the above. (Composition 13) A determination means for determining the tactile sensation to be presented by combining a first tactile sensation based on the characteristics of the subject in the captured image with a second tactile sensation predetermined for the subject and having a higher level of abstraction than the first tactile sensation. It has, The determination means determines the touch sensation to be presented as the one that is closer to the first touch sensation the higher the clarity of the subject in the captured image, and closer to the second touch sensation the lower the clarity of the subject. An electronic device characterized by the following features. (Method 14) A decision step to determine the tactile sensation to present, based on multiple data points related to the capture. It has, In the aforementioned decision step, different tactile sensations are determined to be presented depending on the shooting conditions. A method for controlling electronic equipment characterized by the following features. (Program 15) A program for causing a computer to function as one of the electronic devices described in any of configurations 1 to 13. (medium 16) A computer-readable storage medium storing a program for causing the computer to function as one of the electronic devices described in configurations 1 to 13. [Explanation of Symbols]

[0087] 1: Imaging device 102: Information processing device 102d: Tactile processing unit

Claims

1. A decision-making mechanism for determining the tactile sensation to present, based on multiple data related to the photograph. It has, The aforementioned determination means determines different tactile sensations to be presented depending on the shooting conditions. An electronic device characterized by the following features.

2. The determination means continuously or stepwise changes the presented tactile sensation in response to changes in the shooting conditions. The electronic device according to feature 1.

3. The aforementioned shooting conditions include at least one of the following: first feature data relating to the first features of the subject in the captured image, the mode of the imaging device, the operating status of the imaging device, the settings of the imaging device, and the processing content of the imaging device. The electronic device according to feature 1.

4. The first characteristic includes at least one of the degree of focus, degree of blur, exposure, sensitivity, noise level, contrast, degree of aberration, image resolution, sharpness, and degree of abstraction of the subject. The electronic device according to feature 3.

5. The plurality of data includes at least one of the following: second feature data relating to a second feature of the subject in the captured image, predetermined data corresponding to the category of the subject, distance data from the imaging device to the subject, and predetermined fixed data. The electronic device according to feature 1.

6. The second feature data includes at least one of the subject recognition result data, color data, brightness data, edge data, and shape data. The electronic device according to feature 5.

7. The first feature data mentioned above shows multiple features, The determination means switches the features to be used so as to determine the tactile sensation to be presented by using the feature among the plurality of features whose temporal change is smaller than that of the other features. The electronic device according to feature 3.

8. The determination means determines at least one of the tactile frequency, phase, particle size, hardness, temperature, characteristics, presentation amount, and presentation time. The electronic device according to feature 1.

9. Presentation means for presenting the tactile sensation determined by the determination means. It further possesses The electronic device according to feature 1.

10. Output means for outputting the aforementioned tactile data to an external device. It further possesses The electronic device according to feature 1.

11. The aforementioned electronic device is an imaging device. The electronic device according to feature 1.

12. The aforementioned electronic device is an external device of the imaging device. The electronic device according to feature 1.

13. A determination means for determining the tactile sensation to be presented by combining a first tactile sensation based on the characteristics of the subject in the captured image with a second tactile sensation predetermined for the subject and having a higher level of abstraction than the first tactile sensation. It has, The determination means determines the touch sensation to be presented as the one that is closer to the first touch sensation the higher the clarity of the subject in the captured image, and closer to the second touch sensation the lower the clarity of the subject. An electronic device characterized by the following features.

14. A decision step to determine the tactile sensation to present, based on multiple data points related to the capture. It has, In the aforementioned decision step, different tactile sensations are determined to be presented depending on the shooting conditions. A method for controlling electronic equipment characterized by the following features.

15. A program for causing a computer to function as one of the means of an electronic device according to any one of claims 1 to 13.

16. A computer-readable storage medium storing a program for causing the computer to function as one of the means of the electronic device described in any one of claims 1 to 13.

Images