Electronic apparatus, display control unit, method for controlling electronic apparatus, program, and storage medium

Abstract

To provide a technique that can sufficiently conceal a subject (prevent the subject from being visually recognized).SOLUTION: An electronic apparatus of the present invention has: acquisition means that acquires an image; detection means that detects a specific subject from the image; and setting means that performs settings to hide a range including the specific subject detected by the detection means, within a range in a predetermined direction of the image.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an electronic device, a display control device, a control method for an electronic device, a program, and a storage medium, and more particularly to a control method for displaying an image having a wide image range. [Background technology]

[0002] In recent years, imaging devices capable of capturing wide-range images that represent a range wider than the human viewing angle, such as omnidirectional images and spherical images, have become widespread. Also, a method of distributing such wide-range images to an unspecified number of viewers (for example, live distribution) is known.

[0003] In the distribution of wide-area images, viewers may unexpectedly see subjects that are not intended by the photographer (distributor). For example, the display of a third party's face may infringe on portrait rights, or copyrights may be infringed by the display of copyrighted material on street advertisements.

[0004] Patent Documents 1 and 2 disclose techniques for concealing a subject that appears in an image against the photographer's intention by using blurring or mosaic processing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2009-094946 A [Patent Document 2] JP 2017-022742 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the techniques disclosed in Patent Documents 1 and 2, which conceal the subject by blurring or mosaic processing, may not sufficiently conceal the subject (the subject may be visually recognized).

[0007] An object of the present invention is to provide a technique that can sufficiently conceal a subject (make it invisible). [Means for solving the problem]

[0008] The electronic device of the present invention is characterized by having an acquisition means for acquiring an image, a detection means for detecting a specific subject from the image, and a setting means for setting a range of the image in a specified direction that includes the specific subject detected by the detection means to be hidden. Effect of the Invention

[0009] According to the present invention, the subject can be sufficiently concealed (made invisible). [Brief description of the drawings]

[0010] [Figure 1] 1 is an external view and a block diagram of a digital camera. [Diagram 2] 1 is an external view and a block diagram of a display control device. [Diagram 3] 11A and 11B are schematic diagrams showing a touch operation for changing a display range. [Figure 4] 13 is a flowchart showing a distribution process of the digital camera. [Diagram 5] Schematic diagrams showing the positional relationships of subjects around a digital camera, etc. [Figure 6] 13 is a flowchart showing a process for determining a viewing restriction setting. [Figure 7] FIG. 2 is a schematic diagram for explaining the angle of view at which a subject is located. [Figure 8] 4 is a flowchart showing a display process of the display control device. [Figure 9] FIG. 1 is a schematic diagram showing a wide-range image before and after viewing restriction; [Figure 10] 11 is a schematic diagram showing a case where the angles of view in which a plurality of restricted subjects are located overlap each other. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1(a) is a front perspective view (external view) of a digital camera 100 (imaging device), which is a type of electronic device to which the present invention is applied. Fig. 1(b) is a rear perspective view (external view) of the digital camera 100. The digital camera 100 is an omnidirectional camera (panoramic camera).

[0012] Barrier 102a is a protective window for a front camera unit whose shooting range is in front of digital camera 100. The front camera unit is, for example, a wide-angle camera unit whose shooting range is a wide range of 180 degrees or more up, down, left, and right on the front side of digital camera 100. Barrier 102b is a protective window for a rear camera unit whose shooting range is a rear side of digital camera 100. The rear camera unit is, for example, a wide-angle camera unit whose shooting range is a wide range of 180 degrees or more up, down, left, and right on the rear side of digital camera 100.

[0013] The display unit 28 displays various information. The shutter button 61 is an operation unit (operation member) for issuing a shooting instruction. The mode change switch 60 is an operation unit for switching between various modes. The connection I / F 25 is a connector for connecting a connection cable to the digital camera 100, and external devices such as a smartphone, a personal computer, and a television are connected to the digital camera 100 using the connection cable. The operation unit 70 is various switches, buttons, dials, touch sensors, etc. that accept various operations from the user. The power switch 72 is a push button for switching the power on / off.

[0014] The light-emitting unit 21 is a light-emitting member such as a light-emitting diode (LED) and notifies the user of various states of the digital camera 100 by light-emitting patterns and colors. The fixing unit 40 is, for example, a tripod screw hole and is used to fix and install the digital camera 100 on a fixing device such as a tripod.

[0015] FIG. 1C is a block diagram showing an example of the configuration of the digital camera 100.

[0016] The barrier 102a covers the imaging system (photographing lens 103a, shutter 101a, imaging unit 22a, etc.) of the front camera unit to prevent the imaging system from becoming dirty or damaged. The photographing lens 103a is a lens group including a zoom lens and a focus lens, and is a wide-angle lens. The shutter 101a is a shutter with an aperture function that adjusts the amount of subject light incident on the imaging unit 22a. The imaging unit 22a is an imaging element (imaging sensor) composed of a CCD or CMOS element that converts an optical image into an electrical signal. The A / D converter 23a converts an analog signal output from the imaging unit 22a into a digital signal. Note that the barrier 102a may not be provided and the outer surface of the photographing lens 103a may be exposed, and the photographing lens 103a may prevent other imaging systems (shutter 101a and imaging unit 22a) from becoming dirty or damaged.

[0017] The barrier 102b covers the imaging system (taking lens 103b, shutter 101b, imaging section 22b, etc.) of the rear camera section to prevent the imaging system from getting dirty or damaged. The taking lens 103b is a lens group including a zoom lens and a focus lens, and is a wide-angle lens. The shutter 101b is a shutter with an aperture function that adjusts the amount of subject light incident on the imaging section 22b. The imaging section 22b is an imaging element composed of a CCD or CMOS element that converts an optical image into an electrical signal. The A / D converter 23b converts an analog signal output from the imaging section 22b into a digital signal. Note that if the barrier 102b is not provided, the outer surface of the taking lens 103b is exposed, and the taking lens 103b covers the other imaging systems (shutter It is also possible to prevent the lens 101b and the imaging unit 22b from becoming dirty or damaged.

[0018] The imaging units 22a and 22b capture VR (Virtual Reality) images. The VR image is an image that can be displayed in VR (displayed in the display mode "VR view"). The VR image includes an omnidirectional image (spherical image) captured by an omnidirectional camera (spherical camera) and a panoramic image having a wider image range (effective image range) than the display range that can be displayed at one time on the display unit. The VR image includes not only still images but also videos and live view images (images acquired from the camera almost in real time). The VR image has an image range (effective image range) of up to 360 degrees in the up and down directions (vertical angle, angle from the zenith, elevation angle, depression angle, altitude angle, pitch angle) and 360 degrees in the left and right directions (horizontal angle, azimuth angle, yaw angle).

[0019] In addition, VR images also include images that have a wider angle of view (field of view) than the angle of view that can be captured by a normal camera, or a wider image range (effective image range) than the display range that can be displayed at one time on a display unit, even if the angle of view is less than 360 degrees up and down and less than 360 degrees left and right. For example, an image captured by a spherical camera capable of capturing an object with a field of view (field of view) of 360 degrees in the left and right directions (horizontal angle, azimuth angle) and a vertical angle of 210 degrees centered on the zenith is a type of VR image. In addition, an image captured by a camera capable of capturing an object with a field of view (field of view) of 180 degrees in the left and right directions (horizontal angle, azimuth angle) and a vertical angle of 180 degrees centered on the horizontal direction is a type of VR image. In other words, an image that has a field of view of 160 degrees (±80 degrees) or more in the up and down and left and right directions and has a wider image range than the range that a human can see at one time is a type of VR image.

[0020] When this VR image is displayed in VR (displayed in "VR view" display mode), seamless omnidirectional images can be viewed in the left-right direction (horizontal rotation direction) by changing the position of the display device (the display device displaying the VR image) in the left-right rotation direction. In the up-down direction (vertical rotation direction), seamless omnidirectional images can be viewed within a range of ±105 degrees from directly above (the zenith), but the range beyond 105 degrees from directly above becomes a blank area with no images. A VR image can also be described as "an image whose image range is at least a part of a virtual space (VR space)."

[0021] VR display (VR view) is a display method (display mode) that can change the display range, displaying an image of a VR image with a viewing range according to the orientation of the display device. When viewing by wearing a head-mounted display (HMD), which is a display device, an image with a viewing range according to the orientation of the user's face is displayed. For example, assume that a VR image is displayed with a viewing angle (field of view) centered on 0 degrees left and right (a specific direction, for example, north) and 90 degrees up and down (90 degrees from the zenith, i.e., horizontal) at a certain point in time. From this state, if the orientation of the display device is flipped over (for example, if the display surface is changed from facing south to facing north), the display range is changed to an image with a viewing angle centered on 180 degrees left and right (the opposite direction, for example, south) and 90 degrees up and down (horizontal) in the same VR image. In the case where a user is viewing an HMD, if the user turns his or her face from north to south (i.e., turns around), the image displayed on the HMD will also change from a north image to a south image. This type of VR display can provide users with a sense of immersion, as if they were visually inside the VR image (VR space). A smartphone attached to a VR goggle (head-mounted adapter) can be considered a type of HMD.

[0022] The display method of the VR image is not limited to the above. The display range may be moved (scrolled) in response to user operations on the touch panel or directional buttons, instead of posture changes. During VR display (display mode "VR view"), in addition to changing the display range due to posture changes, the display range can be moved (scrolled) by touch-move on the touch panel, dragging with the mouse, or using directional buttons. The display range may be changed in response to pressing a button or the like.

[0023] The image processing unit 24 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on data from the A / D converter 23a and the A / D converter 23b, or data from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data. The system control unit 50 performs exposure control and distance measurement control based on the arithmetic results obtained by the image processing unit 24. This allows TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-emission) processing to be performed. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (auto white balance) processing based on the arithmetic results obtained. The image processing unit 24 also performs basic image processing on two images (two fisheye images; two wide-angle images) obtained from the A / D converter 23a and the A / D converter 23b, and performs stitching processing to combine the two images that have been subjected to the basic image processing, thereby generating a single VR image. In addition, the image processing unit 24 performs image cropping, enlargement, distortion correction, etc. to display the VR image in VR during VR display in live view or during playback, and performs rendering to draw the processing results in the VRAM of the memory 32.

[0024] In the stitching image processing, the image processing unit 24 uses one of two images as a reference image and the other as a comparison image, calculates the amount of deviation between the reference image and the comparison image for each area by pattern matching processing, and detects a stitching position for stitching the two images based on the amount of deviation for each area. The image processing unit 24 corrects the distortion of each image by geometric transformation in consideration of the detected stitching position and the lens characteristics of each optical system, and converts each image into an image in a celestial sphere format (a celestial sphere image format). Then, the image processing unit 24 generates one celestial sphere image (VR image) by synthesizing (blending) two celestial sphere format images. The generated celestial sphere image is, for example, an image using equirectangular projection, and the position of each pixel of the celestial sphere image can be associated with the coordinates of the surface of a sphere (VR space).

[0025] The output data from the A / D converters 23a, 23b is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or via the memory control unit 15 without via the image processing unit 24. The memory 32 stores image data obtained by the imaging units 22a, 22b and converted into digital data by the A / D converters 23a, 23b, and image data to be output to an external display from the connection I / F 25. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio.

[0026] The memory 32 also serves as a memory (video memory) for image display. The image display data stored in the memory 32 can be output from the connection I / F 25 to an external display. The VR images captured by the imaging units 22a and 22b and generated by the image processing unit 24 and stored in the memory 32 are sequentially transferred to the external display and displayed. In this way, a function as an electronic viewfinder can be realized, and live view display (LV display) can be performed. Hereinafter, an image displayed in live view display is referred to as a live view image (LV image). In addition, the live view display (remote LV display) can also be performed by transferring the VR images stored in the memory 32 to an external device (such as a smartphone) wirelessly connected via the communication unit 54 and displaying them on the external device side.

[0027] The non-volatile memory 56 is a memory serving as an electrically erasable and recordable recording medium, such as an EEPROM. Constants and programs for the operation of the system control unit 50 are recorded in the non-volatile memory 56. The programs referred to here are computer programs for executing various processes.

[0028] The system control unit 50 is a control unit having at least one processor or circuit. and controls the entire digital camera 100. The system control unit 50 realizes each process by executing the programs recorded in the nonvolatile memory 56 mentioned above. The system memory 52 is, for example, a RAM, and constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like are loaded into the system memory 52. ​​The system control unit 50 also performs display control by controlling the memory 32, the image processing unit 24, the memory control unit 15, and the like. The system timer 53 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0029] The mode changeover switch 60 , the shutter button 61 , the operation unit 70 , and the power switch 72 are used to input various operational instructions to the system control unit 50 .

[0030] The mode changeover switch 60 changes the operation mode of the system control unit 50 to one of a still image recording mode, a video shooting mode, a playback mode, a communication connection mode, and the like. Modes included in the still image recording mode include an auto shooting mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode. In addition, there are various scene modes and a custom mode that are shooting settings according to shooting scenes. The user can directly switch to one of these modes using the mode changeover switch 60. Alternatively, the user may use the mode changeover switch 60 to once switch to a list screen of shooting modes, and then selectively switch to one of the multiple modes displayed on the display unit 28 using other operating members. Similarly, the video shooting mode may also include multiple modes.

[0031] The shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on when the shutter button 61 is pressed halfway (instruction to prepare for shooting) during operation, and generates a first shutter switch signal SW1. In response to the first shutter switch signal SW1, the system control unit 50 starts preparation operations for shooting, such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing. The second shutter switch 64 is turned on when the shutter button 61 is pressed fully (instruction to shoot) and generates a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 50 starts a series of operations for shooting processing, from reading out signals from the imaging units 22a and 22b to writing image data to the recording medium 90.

[0032] The shutter button 61 is not limited to an operating member that can be operated in two stages, full press and half press, but may be an operating member that can be pressed only in one stage. In that case, in response to pressing in one stage, the shooting preparation operation and shooting process are performed consecutively. This is the same operation as when a shutter button that can be pressed halfway and full press is pressed fully (when the first shutter switch signal SW1 and the second shutter switch signal SW2 are generated almost simultaneously).

[0033] The operation unit 70 is assigned appropriate functions for each situation by selecting and operating various function icons and options displayed on the display unit 28, and acts as various function buttons. Examples of the function buttons include an end button, a back button, an image forward button, a jump button, a filter button, and an attribute change button. For example, when the menu button is pressed, a menu screen in which various settings can be made is displayed on the display unit 28. The user can intuitively make various settings by operating the operation unit 70 while looking at the menu screen displayed on the display unit 28.

[0034] The power switch 72 is a push button for switching the power on and off. The power control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching the block to which electricity is applied, etc., and detects whether a battery is installed, the type of battery, the remaining battery level, etc. Furthermore, the power supply control unit 80 controls a DC-DC converter based on the detection result and instructions from the system control unit 50, and supplies the necessary voltage for the necessary period to each unit including the recording medium 90. The power supply unit 30 is composed of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, etc.

[0035] The recording medium I / F 18 is an interface with a recording medium 90 such as a memory card or a hard disk. The recording medium 90 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, an optical disk, a magnetic disk, etc. The recording medium 90 may be an exchangeable recording medium that is detachable from the digital camera 100, or may be a recording medium built into the digital camera 100.

[0036] The communication unit 54 transmits and receives video signals, audio signals, and the like to and from an external device connected wirelessly or via a wired cable. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can transmit images (including LV images) captured by the imaging units 22a and 22b and images recorded on the recording medium 90, and can receive images and various other information from external devices.

[0037] The attitude detection unit 55 detects the attitude of the digital camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging units 22a and 22b is an image captured by holding the digital camera 100 horizontally or vertically. In addition, it is possible to determine how much the digital camera 100 is tilted in three axial directions (rotation directions) of the yaw direction, pitch direction, and roll direction when the image captured by the imaging units 22a and 22b is captured. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 55 to the image file of the VR image captured by the imaging units 22a and 22b, or rotate the image (adjust the orientation of the image to correct the tilt (zenith correction)) and record it. The attitude detection unit 55 can use one sensor or a combination of multiple sensors from among an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, and an altitude sensor. The acceleration sensor, gyro sensor, direction sensor, and the like that configure the attitude detection unit 55 can also be used to detect the movement of the digital camera 100 (panning, tilting, lifting, whether or not it is stationary, etc.).

[0038] The microphone 20 is a microphone that collects sounds around the digital camera 100 to be recorded as sounds for VR images (VR video), which are videos. The connection I / F 25 is a connection plug to which an HDMI (registered trademark) cable, a USB cable, or the like is connected to connect to an external device to transmit and receive video.

[0039] FIG. 2(a) is an external view of a display control device 200 to which the present invention is applied. The display control device 200 is, for example, a display device such as a smartphone. The display 205 is a display unit that displays images and various information. The display 205 is integrally configured with a touch panel 206a, and is configured to detect a touch operation on the display surface of the display 205. The display control device 200 is capable of VR display of a VR image (VR content) on the display 205. The operation unit 206b is a power button that accepts an operation to switch the power of the display control device 200 on and off. The operation unit 206c and the operation unit 206d are volume buttons that increase and decrease the volume of the sound output from the speaker 212b and the earphones and external speakers connected to the audio output terminal 212a. The operation unit 206e is a home button for displaying a home screen on the display 205. The audio output terminal 212a is an earphone jack, and is a terminal that outputs an audio signal to the earphones, the external speakers, and the like. The speaker 212b is a built-in speaker that outputs audio.

[0040] 2B is a block diagram showing an example of the configuration of the display control device 200. A CPU 201, a memory 202, a non-volatile memory 203, an image processing unit 204, a display 205, an operation unit 206, a recording medium I / F 207, an external I / F 209, and a communication I / F 210 are connected to the internal bus 250. In addition, an audio output unit 212 and a posture detection unit 213 are also connected to the internal bus 250. The units connected to the internal bus 250 are capable of exchanging data with each other via the internal bus 250.

[0041] The CPU 201 is a control unit that controls the entire display control device 200, and is composed of at least one processor or circuit. The memory 202 is composed of, for example, a RAM (such as a volatile memory using semiconductor elements). The CPU 201 controls each unit of the display control device 200, for example, using the memory 202 as a work memory in accordance with a program stored in the nonvolatile memory 203. The nonvolatile memory 203 stores image data, audio data, other data, various programs for the operation of the CPU 201, and the like. The nonvolatile memory 203 is composed of, for example, a flash memory, a ROM, and the like.

[0042] The image processing unit 204 performs various image processing on images stored in the non-volatile memory 203 or the recording medium 208, video signals acquired via the external I / F 209, images acquired via the communication I / F 210, etc., based on the control of the CPU 201. The image processing performed by the image processing unit 204 includes A / D conversion processing, D / A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement / reduction processing (resizing), noise reduction processing, color conversion processing, etc. In addition, various image processing such as panoramic development, mapping processing, conversion of omnidirectional images or VR images that are wide-range images having wide-range images even if they are not omnidirectional. The image processing unit 204 may be configured with a dedicated circuit block for performing specific image processing. Depending on the type of image processing, it is also possible for the CPU 201 to perform image processing according to a program without using the image processing unit 204.

[0043] The display 205 displays images, GUI screens constituting a GUI (Graphical User Interface), and the like under the control of the CPU 201. The CPU 201 generates a display control signal according to a program, and controls each unit of the display control device 200 to generate a video signal for display on the display 205 and output it to the display 205. The display 205 displays an image based on the generated and output video signal. Note that the configuration of the display control device 200 itself is limited to an interface for outputting a video signal for display on the display 205, and the display 205 may be configured as an external monitor (such as a television or HMD).

[0044] The operation unit 206 is an input device for receiving user operations, including a character information input device such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, etc. In this embodiment, the operation unit 206 includes a touch panel 206a and operation units 206b, 206c, 206d, and 206e.

[0045] A recording medium 208 such as a memory card, CD, or DVD can be detachably attached to the recording medium I / F 207. The recording medium I / F 207 reads data from the attached recording medium 208 and writes data to the recording medium 208 under the control of the CPU 201. The recording medium 208 is a storage unit that stores data such as images to be displayed on the display 205. The external I / F 209 is an interface that connects to an external device via a wired cable (such as a USB cable) or wirelessly, and inputs and outputs (data communication) video signals and audio signals. The communication I / F 210 is an interface that communicates (wirelessly) with an external device, the Internet 211, etc., and transmits and receives (data communication) various data such as files and commands. The communication I / F 210 can communicate with the communication unit 54 of the digital camera 100 shown in FIG. 1. The display control device 200 is connected to the digital camera 100. Thus, it is possible to receive the captured image (video) and display it on the display 205 of the display control device 200.

[0046] The audio output unit 212 outputs the sound of video or music data played back by the display control device 200, operation sounds, ringtones, various notification sounds, etc. The audio output unit 212 includes an audio output terminal 212a for connecting earphones or the like, and a speaker 212b, but the audio output unit 212 may output audio data to an external speaker via wireless communication or the like.

[0047] The attitude detection unit 213 detects the attitude (tilt) of the display control device 200 with respect to the direction of gravity and the attitude of the display control device 200 with respect to each axis of the yaw direction, pitch direction, and roll direction, and notifies the CPU 201 of the attitude information. Based on the attitude detected by the attitude detection unit 213, it is possible to determine whether the display control device 200 is held horizontally, held vertically, facing up, facing down, or in an oblique attitude. It is also possible to determine the presence or absence and magnitude of the tilt of the display control device 200 in a rotation direction such as the yaw direction, pitch direction, or roll direction, and whether the display control device 200 has rotated in the rotation direction. One or a combination of multiple sensors from among an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, and an altitude sensor can be used as the attitude detection unit 213.

[0048] As described above, the operation unit 206 includes the touch panel 206a. The touch panel 206a is configured to be planar and overlaid on the display 205, and is an input device that outputs coordinate information according to the touched position. The CPU 201 can detect the following operations or states on the touch panel 206a. A finger or pen that has not been touching the touch panel 206a touches the touch panel 206a again, that is, the start of touching (hereinafter referred to as Touch-Down). A state in which a finger or a pen touches the touch panel 206a (hereinafter referred to as Touch-On) A finger or a pen is moved while touching the touch panel 206a (hereinafter referred to as Touch-Move). The finger or pen that was touching the touch panel 206a is removed from the touch panel 206a, that is, the touch ends (hereinafter referred to as "touch-up"). A state in which nothing is touching the touch panel 206a (hereinafter referred to as Touch-Off)

[0049] When touch-down is detected, touch-on is also detected at the same time. After touch-down, touch-on will usually continue to be detected unless touch-up is detected. If touch-move is detected, touch-on is also detected at the same time. Even if touch-on is detected, touch-move will not be detected if the touch position does not move. When it is detected that all fingers or pens that were touching have touched up, touch-off is detected.

[0050] These operation states and the position coordinates of the touch panel 206a touched by a finger or pen are notified to the CPU 201 via an internal bus, and the CPU 201 determines what kind of operation (touch operation) has been performed on the touch panel 206a based on the notified information. Regarding touch-move, the moving direction of the finger or pen moving on the touch panel 206a can also be determined for each vertical and horizontal component on the touch panel 206a based on changes in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed.

[0051] The operation of touching the touch panel 206a with a finger, moving the finger quickly for a certain distance, and then releasing the finger is called a flick. This is an operation to quickly trace the top of 6a with a finger as if flicking it. If a touch-move is detected over a certain distance or at a certain speed or faster, and then a touch-up is detected, it can be determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation).

[0052] Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer together is called pinch in, and a touch operation in which the touch positions are moved away from each other is called pinch out. Pinch out and pinch in are collectively called pinch operation (or simply pinch). The touch panel 206a may be of any of various touch panel types, such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. There are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or a pen to the touch panel, and either type may be used.

[0053] FIG. 2(c) is an external view of VR goggles (head mount adapter) 230 to which the display control device 200 can be attached. The display control device 200 can also be used as a head mount display by attaching it to the VR goggles 230. The insertion port 231 is an insertion port for inserting the display control device 200. The entire display control device 200 can be inserted into the VR goggles 230 with the display surface of the display 205 facing the headband 232 side (i.e., the user side) for fixing the VR goggles 230 to the user's head. In a state where the user wears the VR goggles 230 to which the display control device 200 is attached on the head, the user can view the display 205 without holding the display control device 200 with his / her hands. In this case, when the user moves the head or the entire body, the posture of the display control device 200 also changes. The posture detection unit 213 detects the posture change of the display control device 200 at this time, and the CPU 201 performs processing for VR display based on this posture change. In this case, the detection of the orientation of the display control device 200 by the orientation detection unit 213 is equivalent to detecting the orientation of the user's head (the direction in which the user's line of sight is facing). Note that the display control device 200 itself may be an HMD that can be worn on the head without VR goggles.

[0054] FIG. 2(d) shows an example of VR display when a user wears VR goggles 230 equipped with the display control device 200. When a person 240 wears the VR goggles 230 and looks forward (toward the viewer on the paper), the person 240 sees a VR display 245 by the display control device 200. Flowers are displayed in the VR display 245. When the person 240 rotates his / her head in a downward direction 241, the flowers seen in the VR display 245 appear to move in an upward direction 246. Similarly, when the person 240 rotates his / her head in a left direction 242, an upward direction 243, and a right direction 244, the flowers seen in the VR display 245 appear to move in a right direction 247, a downward direction 248, and a left direction 249, respectively. In this way, when the user wears the VR goggles 230, the CPU 201 changes the display range of the VR display in response to a change in the user's posture.

[0055] A method of changing the display range of the VR display that does not detect the user's posture can also be used. FIGS. 3(a) to 3(c) show an example of changing the display range by a touch operation without detecting the user's posture. First, as shown in FIG. 3(a), the CPU 201 detects a touch down on the touch panel 206a. At the time of touch down, the CPU 201 does not change the display range of the VR display. Next, as shown in FIG. 3(b), the CPU 201 detects a touch move on the touch panel 206a. At this time, the CPU 201 moves the display range of the VR display so that the wide-range image (VR image) moves (slides) in a moving direction and by a moving amount according to the touch move. The moving direction of the wide-range image (the opposite direction to the moving direction of the display range) is the same as the moving direction of the touch move, and the moving amount of the wide-range image (the moving amount of the display range) is proportional to the moving amount of the touch move. Finally, as shown in FIG. 3(c), the CPU 201 detects a touch up from the touch panel 206a. At the time of touch up, the CPU 201 changes the display range of the VR display. This method does not require equipment such as VR goggles.

[0056] The flow of the distribution process performed by digital camera 100 will be described with reference to the flowchart in Fig. 4. The distribution process in Fig. 4 is realized by system control unit 50 expanding a program stored in non-volatile memory 56 into system memory 52 and executing it. System control unit 50 starts the distribution process in Fig. 4 in response to, for example, an instruction to start distribution from a user. Note that, although an example of live distribution will be described, the distribution of images (video) does not have to be live.

[0057] FIG. 5(a) shows an example of the positional relationship of subjects around the digital camera 100 when the wide-range image is captured by the digital camera 100. FIG. 5(a) is a diagram showing the positional relationship seen from above. People 501 to 505 are located around the digital camera 100 and are captured by the digital camera 100. Subjects other than people are omitted. The digital camera 100 is set so that the optical axis of the photographing lens 103a and the optical axis of the photographing lens 103b are horizontal. The reference direction 506 of the digital camera 100 is the center of the photographing range in front of the digital camera 100 (the direction in which the optical axis of the photographing lens 103a is facing). When the wide-range image captured in the situation of FIG. 5(a) is developed using equirectangular projection, an image as shown in FIG. 5(b) is obtained.

[0058] In S401, the system control unit 50 sets the subject designated by the user (photographer, distributor) of the digital camera 100 as a subject for which viewing restriction setting (setting to restrict the viewer from viewing the distributed image (video)) is not performed. The user may designate one subject, or may designate multiple subjects. The user operation for designating the subject may be, for example, an operation of setting the digital camera 100 to a specific mode and then photographing the subject. The user operation may also be an operation of inserting the recording medium 90 recording an image of the subject into the digital camera 100, causing the system control unit 50 to detect the subject in the image, and selecting a specific subject from the detected subjects. For example, if the user of the digital camera 100 wants to show his / her face to the viewers, he / she may designate his / her own face. This makes it possible to prevent viewing restriction setting from being performed on the face of the user of the digital camera 100. Here, it is assumed that only the person 501 in FIG. 5(b) is set as a subject for which viewing restriction setting is not performed.

[0059] In S402, the system control unit 50 sets an angle of view (a specific range within a range in a specific direction (horizontal direction) of a wide-range image) in which viewing restriction setting of a subject is not performed in response to a user operation. One angle of view may be set, or multiple angles of view may be set. The user operation may be, for example, an operation of setting on a setting screen of the digital camera 100 so as not to set viewing restriction setting of a subject for an angle of view in a range photographed by the lens 103a. The user operation may also be an operation of inputting an arbitrary angle between 1 degree and 180 degrees on the setting screen of the digital camera 100. When an angle of 45 degrees is input, for example, the system control unit 50 sets so as not to set viewing restriction setting of a subject for an angle of view of 90 degrees in total, which is 45 degrees on the left and right of a specific subject (for example, the subject set in S401). Instead of an angle of view centered on a specific subject, an angle of view centered on the reference direction 506 in FIG. 5(a) may be set. The digital camera 100 may be installed at a live venue. In that case, if it is clear that the performers of the live performance will fit within a certain angle of view of digital camera 100, the user of digital camera 100 may specify that angle of view as the angle of view for which viewing restrictions will not be set for the subjects, thereby preventing viewing restrictions from being set for the performers.

[0060] In S403, the system control unit 50 controls the imaging units 22a, 22b, etc. to obtain a wide-range image.

[0061] In S404, the system control unit 50 detects a restricted subject from the wide-range image acquired in S403. A restricted subject is a specific subject for which the viewer should be restricted from viewing the distributed image (video). The detection of the restricted subject is, for example, pattern matching using a trained model by machine learning. Note that the restricted subject may be specified by a user of the digital camera 100. For example, in the case where the digital camera 100 is provided with a display and a touch panel superimposed thereon, the user may touch a subject displayed on the display, and the touched subject may be set as the restricted subject.

[0062] The restricted subject detected in S404 is, for example, a subject that may infringe the rights of others if it appears in the image to be distributed. Specifically, it is a third party who is not acquainted with the photographer, a copyrighted work depicted in a street advertisement, etc. If these appear in the image to be distributed, the former may infringe the portrait right or privacy right, and the latter may infringe the copyright. The restricted subject may be a subject with an age restriction for viewing, such as a person smoking a cigarette or a gravure image. Such a subject that should be restricted from viewing only by some viewers may also be set as a restricted subject. For example, viewers who should be restricted from viewing an image of a person smoking a cigarette are limited to viewers of a specific nationality and a specific age by law, but in S404, the system control unit 50 may also set a person smoking a cigarette as a restricted subject. For viewers who are determined not to be restricted from viewing based on requirements such as nationality and age, the viewing restriction setting of the restricted subject can be released by the process of FIG. 8 described later.

[0063] The system control unit 50 performs the processes of S405 to S407 for each restricted subject detected in S404. At this time, the restricted subject to be processed may or may not be selected randomly. For example, the restricted subject may be selected in order from the one furthest from the subject set in S401 in the wide-range image (the restricted subject having the largest absolute value of the difference between the location angle (azimuth angle) of the subject set in S401 and the location angle of the restricted subject). When the processes of S405 to S407 have been performed for all restricted subjects, the process proceeds to S408.

[0064] In S405, the system control unit 50 determines whether or not to set a viewing restriction for the restricted subject to be processed. If it is determined that a viewing restriction should be set, the process proceeds to S406, otherwise the process of S405 to S407 for the restricted subject to be processed is terminated. If the restricted subject is the subject set in S401 or a subject included in the angle of view set in S402, the system control unit 50 determines not to set a viewing restriction. Otherwise, the system control unit 50 determines to set a viewing restriction.

[0065] 5B is set as a subject for which viewing restriction setting is not performed in S401, and an angle of view for which viewing restriction setting is not performed for the subject is not set in S402. Then, suppose that persons 501 to 505 are detected as restricted subjects in S404. In this case, it is determined that viewing restriction setting is performed for persons 502 to 505.

[0066] In S406, the system control unit 50 performs a determination process for determining a parental restriction setting to be performed for the restricted subject to be processed. The details of the determination process for the parental restriction setting will be described later with reference to FIG.

[0067] In S407, the system control unit 50 performs (enables) the viewing restriction setting determined in S406 for the restricted subject to be processed.

[0068] In S408, the system control unit 50 distributes distribution data including the wide-range image and information on each restricted subject (such as identification information, area, location angle, location angle of view, viewing restriction setting, etc.) to a display device (for example, the display control device 200) held by a viewer. Details of the processing in the display device will be described later with reference to FIG. 8.

[0069] In S409, the system control unit 50 determines whether or not to continue the distribution in response to a user operation. If the distribution is to be continued, the process proceeds to S403, and if not, the process proceeds to S410.

[0070] In S410, the system control unit 50 stops the distribution.

[0071] Fig. 6 shows a flowchart of the viewing restriction setting decision process (S406 in Fig. 4). In the decision process in Fig. 6, either a process for deciding to set the angle of view in which the restricted subject is located so as to be hidden (non-display setting) or a process for deciding to set the angle of view in which the restricted subject is located so as to be hidden (reduced visibility setting) is selectively executed. Image processing for reducing visibility includes, for example, mosaic processing and blurring processing. The former process is the process in S611, and the latter process is the process in S610.

[0072] In S601, the system control unit 50 sets a range including the restricted object to be processed, among the range of a predetermined direction (horizontal direction) of the wide-range image, as the location angle of the restricted object. For example, as the location angle of the person 701, which is the restricted object, the minimum angle of view 702 including the person 701 may be set as shown in FIG. 7(a), or the angle of view 703 or angle of view 704 wider than the angle of view 702 may be set as shown in FIGS. 7(b) and 7(c). Furthermore, the width 705 between the azimuth angle of the end of the restricted object (end of the angle of view 702) and the azimuth angle of the end of the location angle of view may be common to all restricted objects, or may be changed according to the restricted object. For example, the lower the accuracy (reliability) of the detection result of the restricted object, the wider the width 705 may be, or the faster the moving speed of the restricted object in the wide-range image, the wider the width 705 may be. Furthermore, the width 705 may be different between the right end and the left end of the location angle of view. For example, when the restricted subject moves toward the right side of the wide range image, the right side width 705 may be set to a width greater than the left side width 705 .

[0073] In S602, the system control unit 50 judges whether the restricted subject to be processed is included in the angle of view of another restricted subject for which non-display setting has already been determined. If the restricted subject to be processed is included in the angle of view of another restricted subject, the process proceeds to S611, otherwise the process proceeds to S603. Here, it may be judged whether the entire restricted subject to be processed is included in the angle of view of another restricted subject, or whether at least a part of the restricted subject to be processed is included in the angle of view of another restricted subject. It may be judged whether a predetermined ratio or more of the restricted subject to be processed is included in the angle of view of another restricted subject.

[0074] In S603, the system control unit 50 judges whether or not the horizontal distance between the restricted object to be processed and the object set in S401 in the wide-range image is longer than a threshold value. If the distance is longer than the threshold value, the process proceeds to S611, and if not, the process proceeds to S603. Here, the horizontal distance between two objects in the wide-range image corresponds to the absolute value of the difference between the location angle (azimuth angle) of one object and the location angle of the other object. The system control unit 50 may also make the judgment assuming that the object set in S401 is in the reference direction of the digital camera 100. For example, in the situation of FIG. 5(b) (the situation in which the person 501 is set as the object for which viewing restriction setting is not performed in S401), it is judged whether or not the distance to the location angle 0° of the person 501 and the reference direction 506 is longer than a threshold value. If multiple objects are set as the objects for which viewing restriction setting is not performed in S401, it may be judged whether or not the distance from the restricted object to be processed to the closest object among the multiple objects is longer than a threshold value.

[0075] Here, two values ​​are obtained as the absolute value of the difference between the angle at which the restricted subject to be processed is located and the angle at which the subject is located set in S401: a value less than 180° and a value greater than 180°. The smaller of these two values ​​(clockwise angle and counterclockwise angle) (a value equal to or less than 180°) is compared. For example, when the process of S603 is performed for person 505 in FIG. 5(b), 150° and 210° are obtained as the absolute values ​​of the difference between the location angle of person 505, 210°, and the location angle of person 501, 0°. In S602, the system control unit 50 determines whether 150° is greater than a threshold value.

[0076] In S604, the system control unit 50 calculates the ratio of the size of the area of ​​the restricted object to be processed to the size of the area corresponding to the angle of view of the restricted object in the wide-range image, and judges whether the ratio is greater than a threshold value. If the calculated ratio is greater than the threshold value, the process proceeds to S611, and if not, the process proceeds to S605. For example, when performing the process of S604 for the person 505 in FIG. 5(b), the system control unit 50 judges whether the ratio of the size of the area of ​​the person 505 to the size of the area corresponding to the angle of view 507 of the person 505 is greater than a threshold value. If the viewing of most of a specific angle of view is restricted according to the visibility reduction setting, it is unlikely that the object that the viewer wants to view is included in that angle of view. In such a case, it is often preferable for the viewer to hide the angle of view itself from the viewpoint of operability, which will be described later with reference to FIG. 10. Therefore, if the ratio calculated in S604 is greater than the threshold value, the process proceeds to S611. In addition, when the angles of view of the multiple restricted subjects including the restricted subject to be processed overlap or are close to each other, the ratio of the total size of the areas of the multiple restricted subjects to the size of the area corresponding to the angle of view including the angles of view of the multiple restricted subjects may be calculated. The angle of view including the angles of view of the multiple restricted subjects is, for example, in the range from the minimum azimuth angle to the maximum azimuth angle of the angles of view of the multiple restricted subjects.

[0077] In S605, the system control unit 50 detects the horizontal moving speed of the restricted subject to be processed in the wide-range image, and determines whether the moving speed is slower than a threshold value. If the moving speed is slower than the threshold value, the process proceeds to S611, and if not, the process proceeds to S606.

[0078] In S606, the system control unit 50 calculates the ratio of the number of viewers of the restricted subject to be processed to the total number of viewers of the distribution, and judges whether or not the ratio is smaller than a threshold value. If the calculated ratio is smaller than the threshold value, the process proceeds to S611, and if not, the process proceeds to S607. The total number of viewers of the distribution and the number of viewers of the restricted subject to be processed can be calculated, for example, based on a signal from the display device of the viewer. For example, whether or not a certain viewer is viewing the person 505 (angle of view 507) in FIG. 5(b) can be judged by whether or not the angle of view 507 is displayed at the center of the display surface of the display device of the viewer (for example, the center of the display surface of the display 205). Note that instead of the above ratio, the ratio of the number of viewers of the restricted subject to be processed to the number of viewers in a small group extracted from the entire viewers of the distribution may be calculated.

[0079] In S607, the system control unit 50 judges whether or not there is a restricted subject for which a non-display setting has already been decided (whether or not there is already a location angle set to non-display). If it is judged that there is a restricted subject for which a non-display setting has already been decided, the process proceeds to S608, and if not, the process proceeds to S611.

[0080] In S608, the system control unit 50 judges whether or not the total size (not including overlapping) of the angles of view of the restricted subjects that have already been set to non-display is smaller than a threshold value. If the total size (total angle) is smaller than the threshold value, the process proceeds to S611, and if not, the process proceeds to S609. Note that it may be judged whether or not the total size of the angles of view of all restricted subjects that have already been set to non-display and the angle of view of the restricted subject to be processed is smaller than a threshold value.

[0081] In S609, the system control unit 50 judges whether or not the horizontal distance between the angle of view of the restricted subject that has already been set to non-display in the wide-range image and the restricted subject to be processed is greater than a threshold value. The horizontal distance between the edge of the angle of view of the restricted subject that has already been set to hidden and the restricted subject to be processed is compared with a threshold value. For example, the absolute value of the difference (the smaller of the two) between the azimuth angle corresponding to the edge of the angle of view of the restricted subject that has already been set to hidden and the angle of view of the restricted subject to be processed is compared with the threshold value. If the distance is longer than the threshold value, the process proceeds to S610, and if not, the process proceeds to S611. The process may proceed to S611 only if the angle of view of the restricted subject to be processed is adjacent to or partially overlaps with the angle of view of another restricted subject that has already been set to hidden.

[0082] In S610, the system control unit 50 determines to perform visibility reduction setting as viewing restriction setting for the restricted subject to be processed.

[0083] In S611, the system control unit 50 determines to set the restricted subject to non-display as a viewing restriction setting for the restricted subject to be processed.

[0084] The various threshold values ​​used in the determination process of FIG. 6 may be predetermined fixed values, values ​​calculated individually by the system control unit 50 according to the restricted subject, or values ​​set according to input from the user.

[0085] The flow of display processing of a distributed image (distributed video) performed by the display control device 200 will be described with reference to the flowchart of Fig. 8. The display processing of Fig. 8 is realized by the CPU 201 expanding a program stored in the non-volatile memory 203 into the memory 202 and executing it. The CPU 201 starts the processing of Fig. 8 in response to, for example, an instruction from a viewer to start viewing.

[0086] In S801, the CPU 201 receives the distribution data distributed in S408 from the digital camera 100 via the communication I / F 210. This distribution data includes the wide-range image and information on each restricted subject (identification information, area, location angle, location angle of view, viewing restriction setting, etc.).

[0087] The CPU 201 performs the processes of S802 to S806 for each restricted subject for which viewing restriction has been set. When the processes of S802 to S806 have been performed for all restricted subjects for which viewing restriction has been set, the process proceeds to S807.

[0088] In S802, the CPU 201 judges whether or not the viewer using the display control device 200 should be restricted from viewing the restricted object to be processed. If the viewer should be restricted, the process proceeds to S804, and if not, the process proceeds to S803. For this judgment, for example, account information of a distribution viewing application associated with the display control device 200 is used. The account information used here is personal information such as age and nationality. For example, in S404 of FIG. 4, the system control unit 50 may detect a person smoking a cigarette as a restricted object. However, viewers who should be restricted from viewing the person smoking a cigarette are limited to viewers of a specific nationality and a specific age group. In S802, the CPU 201 refers to the viewer's account information, and if the viewer does not fall under the specific nationality and the specific age group, the CPU 201 may judge that it is not necessary to restrict viewing of the person smoking a cigarette (restricted object), and proceed to S803.

[0089] In S803, the CPU 201 cancels (invalidates) the parental restriction setting of the restricted subject to be processed, and ends the processes of S802 to S806 for the restricted subject to be processed. Information on the changed parental restriction setting (cancellation, invalidation information) is stored in the memory 202 in association with the restricted subject, and may be reflected in the distribution data received in S801 next time.

[0090] In step S804, the CPU 201 detects that the viewing restriction setting of the restricted object to be processed is set to hidden. If the setting is non-display, the process proceeds to S805, and if not (if the setting is reduced visibility), the process of S802 to S806 for the restricted subject to be processed is terminated.

[0091] In S805, the CPU 201 determines whether or not the display control device 200 is set to change the range (display range) of the wide-range image in response to a change in the attitude of the display control device 200. If such a setting is made, the process proceeds to S806, and if not, the process of S802 to S806 for the restricted subject to be processed is terminated. For example, if the display control device 200 is attached to the VR goggles 230 of FIG. 2(c) and the display range is changed by detecting the attitude of the viewer's head, the process proceeds to S806. If the viewer does not use the VR goggles 230 and the display control device 200 changes the display range in response to a touch operation as shown in FIGS. 3(a) to 3(c), the process of S802 to S806 for the restricted subject to be processed is terminated.

[0092] In S806, the CPU 201 changes the viewing restriction setting of the restricted object to be processed from a hidden setting to a reduced visibility setting. When the display range is changed in response to a change in the viewer's posture, it is considered that the discomfort felt by the viewer due to a part of the angle of view of the wide range image being hidden is relatively greater than in the other cases. The processing of S806 can reduce the discomfort. As described above, the information on the changed viewing restriction setting is stored in the memory 202 in association with the restricted object, and may be reflected in the distribution data received in S801 next time.

[0093] In S807, the CPU 201 performs viewing restriction processing (non-display, mosaic processing, blurring processing, etc.) for restricting viewing of the restricted subject in the wide-range image in accordance with the viewing restriction setting of each restricted subject.

[0094] In S808, the CPU 201 displays the wide-range image after the viewing restriction process on the display 205.

[0095] Fig. 9(a) is a schematic diagram showing a wide-range image before the viewing restriction process of S807 is performed. Here, a person 901 is a subject (subject for which viewing restriction setting is not enabled) set in S401 of Fig. 4. A person 902 is a restricted subject detected in S404, and a view angle 904 is set as the view angle of the person 902 in S601 of Fig. 6, and a setting to hide the person 902 (view angle 904) is enabled in S407. Here, the view angle 904 is in a range of 30°. A person 903 is a restricted subject detected in S404, and a visibility reduction setting for the person 903 is enabled in S407. Here, it is assumed that the viewing restriction settings for the people 902 and 903 are not disabled in S803, and it is determined in S805 that a setting to change the display range of the wide-range image according to a change in posture is not set.

[0096] FIG. 9(b) is a schematic diagram showing a wide-range image after the viewing restriction process of S807 is performed. In S808, for example, the wide-range image of FIG. 9(b) is displayed. Since the angle of view 904 of FIG. 9(a) is hidden according to the non-display setting of the person 902, the wide-range image is displayed in a manner in which both ends of the angle of view 904 are joined together. That is, the viewer views a wide-range image with an overall angle of view of 330°, not a wide-range image with an overall angle of view of 360° as in FIG. 9(a). At this time, a straight line 906 or the like may be placed at a position corresponding to the angle of view 904 to make it possible to identify that the angle of view 904 is hidden. The area 905 is the area of ​​the person 903 of FIG. 9(a). The area 905 is displayed in a state in which mosaic processing has been applied according to the visibility reduction setting of the person 903.

[0097] In step S809, the CPU 201 changes the viewing restriction setting of the restricted object from the hidden setting to the visible setting. It is determined whether a change operation to change to the reduced setting has been performed. If a change operation has been performed, the process proceeds to S810, and if not, the process proceeds to S811. The change operation here is, for example, an operation for displaying a hidden angle of view, and is an operation of touching down and then touching up on the touch panel 206a at a position corresponding to the straight line 906 in FIG. 9(b).

[0098] In S810, the CPU 201 changes the viewing restriction setting of the restricted object from the non-display setting to the reduced visibility setting in response to the change operation detected in S809, and updates the display of the wide-range image. FIG. 9(c) is a schematic diagram of a wide-range image displayed when the viewing restriction setting of the person 902 is changed from the non-display setting to the reduced visibility setting in the situation shown in FIG. 9(b). The angle of view 904 that was not displayed in FIG. 9(b) is displayed, and the area 908 of the person 902 is displayed in a state in which mosaic processing has been applied according to the reduced visibility setting. In order to make it easier for the viewer to distinguish between the angle of view displayed in S810 (the angle of view that was not displayed in S808) and the other angles of view, or for other purposes, a shading process or the like may be applied to the image of the angle of view displayed in S810. In addition, at this time, a message indicating the reason why the restricted object was detected as a restricted object may be displayed near the restricted object with the reduced visibility setting. As described above, the information on the changed viewing restriction setting is stored in the memory 202 in association with the restricted object, and may be reflected in the distribution data received in the next S801.

[0099] In S811, the CPU 201 determines whether or not a change operation has been performed to change the viewing restriction setting of the restricted subject from the reduced visibility setting to the hidden setting. If it is determined that a change operation has been performed, the process proceeds to S812, and if not, the process proceeds to S813. The change operation here is, for example, an operation for making the angle of view displayed in S810 hidden again in response to the change operation in S809, and is an operation of touching down and then touching up on the touch panel 206a at a position corresponding to the angle of view 904 in Fig. 9(c). The viewing restriction setting of a subject whose initial viewing restriction setting was the reduced visibility setting (the subject that was subjected to the mosaic processing in S808) may be changed from the reduced visibility setting to the hidden setting.

[0100] In S812, the CPU 201 changes the viewing restriction setting of the restricted subject from the reduced visibility setting to the hidden setting in response to the change operation detected in S811, and updates the display of the wide-range image. As described above, the information on the changed viewing restriction setting is associated with the restricted subject and stored in the memory 202, and may be reflected in the next distribution data received in S801.

[0101] In S813, the CPU 201 determines whether or not to stop displaying the delivery image. If it is to be stopped, the display process in FIG. 8 ends, and if not, the process proceeds to S801.

[0102] As described above, according to the present embodiment, viewing restriction settings are performed for a specific subject that appears in a wide-range image to be distributed, and viewing of the specific subject is restricted according to the viewing restriction settings. As a result, when a wide-range image is distributed, if a specific subject appears in the wide-range image against the distributor's intention, the viewer can be restricted from viewing the specific subject. As a result, for example, the risk that the distributor will infringe on portrait rights or privacy rights by including a third party in the image, or infringe on copyright by including a copyrighted work in the image, can be reduced.

[0103] Furthermore, according to this embodiment, by hiding the angle of view in which the restricted subject is included, the possibility that the viewer will be able to see the restricted subject can be reduced. For example, when restricting the viewing of the restricted subject by image processing such as mosaic processing or blurring processing, there is a risk that the viewer will be able to determine what the restricted subject is, depending on the extent of the processing. Also, when detecting the restricted subject by pattern matching or the like, the accuracy of the detection is not necessarily high. For example, there is a risk that the restricted subject will be detected as smaller than it actually is, or that if the restricted subject is moving quickly, a position different from its actual position will be detected as the position of the restricted subject. For this reason, the decision in FIG. 6 In the determination process, the angle of view of the restricted subject is hidden when the method of hiding the angle of view is particularly useful (when the various risks described above are high) and when the usefulness of performing image processing that reduces visibility is low.

[0104] Also, by hiding the angle of view where the restricted subject is located, the operability of the viewer can be improved. FIG. 10(a) shows a wide-range image before hiding the angle of view where the restricted subject is located. Here, a group of people 1003 (multiple people) who are restricted subjects and whose angles of view partially overlap each other are located between a person 1001 (a subject for which the viewing restriction setting is not enabled) who is a subject set in S401 and a clock 1002. When mosaic processing or the like is performed on all the people included in the group of people 1003, most of the angle of view 1004 is subjected to mosaic processing or the like. In such a case, when the display range of the wide-range image is changed in response to an instruction from the viewer, it is better for the viewer to have the angle of view 1004 hidden, which provides better operability for the viewer. For example, it is assumed that the clock 1002 in FIGS. 10(a) and 10(b) is displayed on the display 205, and the person 1001 is not displayed. In such a case, in order for the viewer to display the person 1001, it is necessary to change the display range of the wide-range image by a touch operation as shown in FIGS. 3(a) to 3(c). At this time, if the angle of view 1004 is hidden as in FIG. 10(b), the person 1001 can be displayed with a shorter touch-move than when the angle of view 1004 is not hidden (FIG. 10(a)).

[0105] In addition, in the present embodiment, the viewer can change the viewing restriction setting of the restricted object, but the viewer may not be able to change the viewing restriction setting of the restricted object. In such a case, the digital camera 100 (system control unit 50) may perform viewing restriction processing and distribute or store the wide-range image after the viewing restriction processing. For example, the digital camera 100 (system control unit 50) may hide the angle of view in which the restricted object is located and distribute or store the wide-range image with a total angle of view of less than 360°. This can prevent the viewer from viewing the restricted object against the distributor's intention. Therefore, this method is effective when the distributor does not want the viewer to view the restricted object.

[0106] The angle of view to be hidden may be removed from the wide-range image or may be replaced with a dummy image. When using a distribution site where it is difficult or impossible to post an image (still image or video) with an angle of view of less than 360°, the digital camera 100 (system control unit 50) may use a dummy image to post a wide-range image with a total angle of view of 360° to the distribution site. The dummy image may be an image taken in advance (for example, an image with an angle of view to be hidden and without a restricted subject), or may not be such an image (for example, a black image). In addition, a message indicating that the angle of view is hidden or a message indicating the reason for hiding may be displayed in association with the angle of view, so that it is possible to identify that the angle of view is hidden.

[0107] The various controls described above as being performed by the system control unit 50 may be performed by one piece of hardware, or multiple pieces of hardware may share the processing to control the entire device. The same applies to the various controls described above as being performed by the CPU 201.

[0108] In addition, although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-mentioned embodiments merely shows one embodiment of the present invention, and each embodiment can be appropriately combined.

[0109] In the above-described embodiment, the present invention is applied to an imaging device (digital camera) and a display control device. However, the present invention is not limited to this example. The present invention is applicable to any electronic device capable of acquiring the above-mentioned information. For example, the present invention is applicable to personal computers, PDAs, mobile phone terminals, portable image viewers, printers, digital photo frames, music players, game consoles, electronic book readers, etc. The present invention is also applicable to video players, display devices (including projection devices), tablet terminals, smartphones, AI speakers, home appliances, in-vehicle devices, etc.

[0110] The present invention is also applicable not only to the imaging device itself, but also to a control device that communicates with an imaging device (including a network camera) via wired or wireless communication and remotely controls the imaging device. Examples of devices that remotely control an imaging device include a smartphone, a tablet PC, and a desktop PC. The imaging device can be remotely controlled by notifying the control device of commands that cause the imaging device to perform various operations and settings based on operations or processes performed on the control device. Also, a live view image captured by the imaging device may be received via wired or wireless communication and displayed on the control device.

[0111] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0112] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) An acquisition means for acquiring an image; A detection means for detecting a specific object from the image; a setting means for setting a range including the specific subject detected by the detection means, out of a range in a predetermined direction of the image, so as to be hidden; 1. An electronic device comprising: (Configuration 2) The setting means selectively executes a process of setting so as to make the range including the specific subject invisible and a process of setting so as to perform image processing for lowering the visibility of the specific subject. 2. The electronic device according to configuration 1. (Configuration 3) The image processing is a mosaic processing. 3. The electronic device according to configuration 2. (Configuration 4) The image processing is a blurring process. 3. The electronic device according to configuration 2. (Configuration 5) The setting means sets the range including the specific subject to be hidden when the specific subject is included in the range that has already been set to be hidden. 5. The electronic device according to any one of configurations 1 to 4. (Configuration 6) The setting means sets so as to hide an area including the specific subject when a distance between the specific subject and an object designated by a user is longer than a threshold value. 6. The electronic device according to any one of configurations 1 to 5. (Configuration 7) The setting means, when a ratio of a size of an area of ​​the specific subject to a size of an area corresponding to a range including the specific subject that is set to be hidden is larger than a threshold value, , set the area including the specific subject to be hidden. 7. The electronic device according to any one of configurations 1 to 6. (Configuration 8) The setting means sets so as to hide the range including the specific subject when the moving speed of the specific subject is slower than a threshold value. 8. The electronic device according to any one of configurations 1 to 7. (Configuration 9) The setting means sets so as to hide the range including the specific subject when a ratio of the number of viewers of the specific subject to the total number of viewers is smaller than a threshold value. 9. The electronic device according to any one of configurations 1 to 8. (Configuration 10) The setting means sets the range including the specific subject to be hidden when there is no range already set to be hidden. 10. The electronic device according to any one of configurations 1 to 9. (Configuration 11) The setting means sets the range including the specific subject to be hidden when a total size of the ranges already set to be hidden is smaller than a threshold value. 11. The electronic device according to any one of configurations 1 to 10. (Configuration 12) The setting means sets the range including the specific subject to be hidden when a distance between the range already set to be hidden and the specific subject is shorter than a threshold value. 12. The electronic device according to any one of configurations 1 to 11. (Configuration 13) The setting means does not perform processing on the subject designated by the user. 13. The electronic device according to any one of configurations 1 to 12. (Configuration 14) The setting means does not process the specific subject in a specific range within the range of the predetermined direction of the image. 14. The electronic device according to any one of configurations 1 to 13. (Configuration 15) A receiving means for receiving an image from the electronic device according to any one of configurations 1 to 14; a control means for controlling the image received by the receiving means to be displayed on a display unit; a change means for canceling the setting made by the electronic device for the specific subject in accordance with the account information of a user; A display control device comprising: (Configuration 16) A receiving means for receiving an image from the electronic device according to any one of configurations 1 to 14; A control means for controlling a display unit to display a partial range of the image; a change means for changing a setting for hiding a range including the specific subject to a setting for performing image processing for reducing visibility of the specific subject when a setting is set for changing a range to be displayed in the image in response to a change in the attitude of the display unit; A display control device comprising: (Configuration 17) A receiving means for receiving an image from the electronic device according to any one of configurations 1 to 14; a control means for controlling a display unit to display the image received by the receiving means; a change means for changing a setting for hiding an area including a specific object in response to a user operation to a setting for performing image processing for lowering the visibility of the specific object; have A display control device comprising: (Configuration 18) The change means changes a setting for performing image processing for reducing the visibility of the specific subject to a setting for not displaying an area including the specific subject in response to a user operation. 18. The display control device according to any one of configurations 15 to 17. (Configuration 19) When the control means hides the range including the specific subject, the control means controls the range including the specific subject to be hidden so that the range is not hidden in a manner that makes it identifiable that the range is not hidden. 19. The display control device according to any one of configurations 15 to 18. (method) An acquisition step of acquiring an image; a detection step of detecting a specific object from the image; a setting step of setting a range including the specific subject detected by the detection step, out of a range in a predetermined direction of the image, so as to be hidden; 13. A method for controlling an electronic device comprising: (program) A program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 14. (medium) 15. A computer-readable recording medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 14. [Explanation of symbols]

[0113] 100: Digital camera 50: System control unit 200: Display control device 201: CPU 210: Communication I / F

Claims

1. Means for acquiring images, A detection means for detecting a specific subject from the aforementioned image, A setting means that selectively performs a process to hide the range of the image in a predetermined direction that includes the specific subject detected by the detection means, and a process to perform image processing that reduces the visibility of the specific subject. It has, If the area of ​​the aforementioned image that includes the specific subject is hidden, the image will be displayed by connecting the ends of the area that includes the specific subject. An electronic device characterized by the following features.

2. The aforementioned image processing is mosaic processing. The electronic device according to feature 1.

3. The aforementioned image processing is a blurring process. The electronic device according to feature 1.

4. The setting means sets the range containing the specific subject to be hidden if that range is already set to be hidden. The electronic device according to feature 1.

5. The setting means configures the range including the specific subject to be hidden when the distance between the specific subject and the subject specified by the user is greater than a threshold. The electronic device according to feature 1.

6. The setting means sets the range containing the specific subject to be hidden if the ratio of the size of the area containing the specific subject to the size of the area containing the specific subject is greater than a threshold. The electronic device according to feature 1.

7. The setting means configures the range including the specific subject to be hidden when the movement speed of the specific subject is slower than a threshold. The electronic device according to feature 1.

8. The setting means configures the range containing the specific subject to be hidden when the ratio of the number of viewers of the specific subject to the total number of viewers is less than a threshold. The electronic device according to feature 1.

9. The setting means sets the range containing the specific subject to be hidden if there is no range already set to be hidden. The electronic device according to feature 1.

10. The setting means sets the range containing the specific subject to be hidden if the total size of the range already set to be hidden is smaller than a threshold. The electronic device according to feature 1.

11. The setting means sets the range containing the specific subject to be hidden if the distance between the range already set to be hidden and the specific subject is shorter than a threshold. The electronic device according to feature 1.

12. The aforementioned setting means does not perform any processing on subjects specified by the user. The electronic device according to feature 1.

13. The setting means does not process the specific subject that is within a specific range of the predetermined direction range of the image. The electronic device according to feature 1.

14. Receiving means for receiving an image from the electronic device described in claim 1, A control means for controlling the display unit to display the image received by the receiving means, A means for changing settings made on a specific subject by the electronic device, according to the user's account information. A display control device characterized by having the following features.

15. Receiving means for receiving an image from the electronic device described in claim 1, A control means for controlling the display unit to show a portion of the aforementioned image, When the display range of the aforementioned image is set to change according to the change in the orientation of the display unit, a changing means is provided to change the setting that hides the range including the specific subject to a setting that performs image processing to reduce the visibility of the specific subject. A display control device characterized by having the following features.

16. Receiving means for receiving an image from the electronic device described in claim 1, Control means to display the image received by the receiving means on the display unit, A means for changing the setting that hides an area containing a specific subject in response to user operation, to a setting that performs image processing to reduce the visibility of the said specific subject. A display control device characterized by having the following features.

17. The modification means changes the setting that performs image processing to reduce the visibility of the specific subject in response to user operation to a setting that hides the area including the specific subject. The display control device according to claim 15.

18. The control means controls the hiding of the area including the specific subject in a manner that makes it identifiable that the area is being hidden. The display control device according to claim 15.

19. Image acquisition step, A detection step of detecting a specific subject from the aforementioned image, A setting step that selectively executes a process to hide the range of the image in a predetermined direction that includes the specific subject detected by the detection step, and a process to perform image processing that reduces the visibility of the specific subject. It has, If the area of ​​the aforementioned image that includes the specific subject is hidden, the image will be displayed by connecting the ends of the area that includes the specific subject. A method for controlling electronic equipment characterized by the following features.

20. A program for causing a computer to function as one of the means of an electronic device according to any one of claims 1 to 13.

21. A computer-readable storage medium storing a program for causing a computer to function as one of the means of an electronic device according to any one of claims 1 to 13.

Images