Display control device, display control method, storage medium, and program product

Abstract

A display control device, a display control method, a storage medium, and a program product are provided. The display control device inputs an input image and receives an operation of zooming in / out. The display control device, during reception of the operation of zooming in, superimposes a frame for indicating a second region corresponding to a zoom-in ratio on the input image corresponding to a first region displayed by a display means, and displays the input image with the frame superimposed on the display means. The display control device, during reception of the operation of zooming out, superimposes the input image corresponding to a third region corresponding to a zoom-out ratio on a background image corresponding to the first region, and displays the background image with the input image superimposed on the display means.

Description

Technical Field

[0001] This disclosure relates to display control devices, display control methods, and storage media. Background Technology

[0002] In recent years, it has become common to equip cameras, such as camcorders, with wireless communication or wired connectivity. Furthermore, applications for remotely controlling cameras via wireless or wired networks (camera control applications) are available for information terminals such as smartphones. Camera control applications provide functions for displaying camera video, controlling video recording, assisting with video recording, displaying camera information, and changing camera settings. For example, video control functions include PTZ (pan, tilt, and zoom) control. Assisting with video recording functions include overlaying markers onto the video corresponding to the camera's video output.

[0003] Smartphone applications rely on touchscreen operation. Touchscreen operations include, for example, pinching. A pinch operation involves closing or opening two fingers on the touchscreen. Closing two fingers on a touchscreen is generally called pinch-in, and opening two fingers is generally called pinch-out. Some camera control applications follow pinch operations on the camera's video feed and control the camera's zoom level.

[0004] In camera control applications, since the camera is controlled via a network, there is a time lag from the start of the operation until the control result is received from the camera. For this reason, when controlling the camera's zoom level using a pinch gesture in a camera control application, there are operability issues because the zoomed-out video is displayed with a delay relative to the pinch gesture.

[0005] Traditionally, as a technique for improving the operability of camera zoom control, there are methods for changing the camera's zoom ratio after adjusting the viewing angle. For example, Japanese Patent Application Publication No. 2016-082556 discloses a technique in which a region on the camera's video is specified using start and end coordinates, and the camera's zoom ratio is controlled based on the positional relationship between the start and end coordinates.

[0006] The function of displaying markers on video (images) captured by a camera is primarily used to confirm the viewing angle of the image. However, the technology disclosed in Japanese Patent Application Publication No. 2016-082556 does not consider the display of markers while the viewing angle of the image is being adjusted. For this reason, there is room for improvement in the technology used to adjust the viewing angle of an image. Summary of the Invention

[0007] This disclosure relates to improved techniques for adjusting the perspective of an image.

[0008] A display control device according to an embodiment of the present disclosure includes: an input unit for inputting an input image; an operation unit for receiving a zoom-in / zoom-out operation; and a display control unit for, during the operation unit receiving a zoom-in operation, overlaying a frame indicating a second region corresponding to a magnification factor onto the input image corresponding to a first region displayed by the display unit, and displaying the input image with the overlaid frame on the display unit; and for, during the operation unit receiving a zoom-out operation, overlaying the input image corresponding to a third region corresponding to a zoom-out factor onto a background image corresponding to the first region, and displaying the background image with the overlaid input image on the display unit.

[0009] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is given by way of example. Attached Figure Description

[0010] Figure 1 This is a configuration diagram of the system according to the first embodiment.

[0011] Figure 2 This is a block diagram illustrating the configuration of the camera 100 according to the first embodiment.

[0012] Figure 3 This is a block diagram illustrating the configuration of a mobile phone 200 according to a first embodiment.

[0013] Figure 4A and Figure 4B This is an example of the screen UI of a camera control application according to the first embodiment.

[0014] Figures 5A to 5D This is an example of a screen UI related to the display of markers in a camera control application according to the first embodiment.

[0015] Figure 6A and Figure 6B This is an example of a screen UI related to PTZ operation of a camera control application according to the first embodiment.

[0016] Figures 7A to 7C Examples of images captured by the camera 100 according to the first embodiment.

[0017] Figure 8 This is a diagram showing the direction and displacement input by PTZ operation of the camera control application according to the first embodiment.

[0018] Figure 9A and Figure 9B This is a diagram illustrating the pinching operation according to the first embodiment.

[0019] Figures 10A to 10C This is a diagram illustrating the zoom operation via pinching according to the first embodiment.

[0020] Figures 11A to 11C This is a diagram illustrating the zoom operation by pinching according to the first embodiment.

[0021] Figures 12A to 12H This is a diagram illustrating the angle adjustment and zoom operation via pinching according to the first embodiment.

[0022] Figures 13A to 13H This diagram illustrates the angle adjustment and zoom operation via pinching according to the first embodiment.

[0023] Figure 14 This is a flowchart of the zoom angle adjustment process according to the first embodiment.

[0024] Figure 15 This is a diagram illustrating the perspective adjustment and PTZ operation via pinching according to the first embodiment.

[0025] Figure 16 This is a diagram illustrating the perspective adjustment and PTZ operation via pinching according to the first embodiment.

[0026] Figure 17 This is a flowchart of the zoom angle adjustment process according to the second embodiment.

[0027] Figure 18A and Figure 18B This is a display example of the view frame according to the second embodiment.

[0028] Figure 19A and Figure 19B This is a diagram showing the adjustment of the viewing angle by pinching according to the second embodiment.

[0029] Figure 20 This is a flowchart of the zoom angle adjustment process according to the third embodiment.

[0030] Figure 21 This is a flowchart of the zoom angle adjustment process according to the fourth embodiment.

[0031] Figure 22A and Figure 22B This is a diagram showing the adjustment of the viewing angle by pinching according to the fourth embodiment.

[0032] Figure 23A and Figure 23B This is a diagram showing the adjustment of the perspective by pinching according to the fourth embodiment.

[0033] Figure 24 This is a flowchart of the zoom angle adjustment process according to the fifth embodiment.

[0034] Figure 25 This is a diagram illustrating the angle adjustment and zoom operation via pinching according to the fifth embodiment.

[0035] Figure 26 This is a diagram illustrating the angle adjustment and zoom operation via pinching according to the fifth embodiment. Detailed Implementation

[0036] The following description will use the accompanying drawings to illustrate the mode for implementing this disclosure. It should be noted that the configurations used to implement this disclosure are not limited to those described in the embodiments. Within the scope where similar effects can be obtained, portions of the configurations described in the embodiments may be omitted or replaced with equivalents. <First Embodiment> <System Configuration>

[0037] The first embodiment of this disclosure is provided for illustration.

[0038] Figure 1 This is a system configuration diagram of a system according to a first embodiment of the present invention. System 10 is configured such that camera 100 and mobile phone 200 are connected via network 112. Although network 112 uses Wi-Fi (registered trademark) in this embodiment, network 112 is not limited thereto. Network 112 can be replaced by, for example, Bluetooth (registered trademark) or proprietary wireless communication methods. Furthermore, network 112 can be wired communication instead of wireless communication. That is, network 112 can communicate regardless of the communication method (such as RS-232C, RS-422A, USB (Universal Serial Bus), and Ethernet (registered trademark)). However, network 112 needs to be a communication method capable of sending and receiving video and setting values ​​from the camera. <Camera Configuration>

[0039] Figure 2This is a block diagram illustrating the configuration of a camera 100 according to a first embodiment. The camera 100 is an example of a data processing device. It should be noted that although an example of a data processing device has been provided herein with respect to the camera 100, the data processing device according to this disclosure is not limited thereto. For example, the data processing device according to this disclosure may be an information processing device such as a so-called tablet device and a personal computer. The camera 100 is an example of a recording device.

[0040] The camera 100 has a control unit 101. The control unit 101 controls the various units of the camera 100 according to input signals and the program described below. It should be noted that instead of the control unit 101 controlling the entire device, multiple hardware components can control the entire device by sharing the processing.

[0041] The camera 100 includes an image-capturing unit 102. The image-capturing unit 102 converts the light of the subject imaged by the lens included in the image-capturing unit 102 into electrical signals, performs noise reduction processing, and outputs digital data as image data. After the image data captured by the image-capturing unit 102 is stored in a buffer memory, predetermined calculations are performed at the control unit 101, and the image data is recorded on the recording medium 110. Furthermore, the image-capturing unit 102 has pan, tilt, and zoom mechanisms, and performs pan, tilt, and zoom (telephoto control / wide-angle control) operations under the control of the control unit 101.

[0042] The camera 100 has a non-volatile memory 103. The non-volatile memory 103 is an electrically erasable and recordable non-volatile memory that stores programs, etc., executed by the control unit 101 as described below.

[0043] The camera 100 has a working memory 104. The working memory 104 is used as a buffer memory for temporarily storing image data captured by the imaging unit 102, an image display memory for the display unit 106, and a working area for the control unit 101, etc.

[0044] The camera 100 includes an operation unit 105. The operation unit 105 receives commands from a user for the camera 100. The operation unit 105 includes, for example, operation components such as a power button for the user to indicate ON / OFF of the camera 100's power, a release switch for indicating video recording, and a playback button for indicating playback of image data. Additionally, a touch panel formed on the display unit 106, described below, is also included in the operation unit 105. It should be noted that the release switch included in the operation unit 105 has SW1 and SW2. When the release switch is in a so-called half-pressed state, SW1 becomes ON. In this way, commands for video recording preparation (such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (flash pre-lighting) processing, etc.) are received. Conversely, when the release switch is in a so-called full-pressed state, SW2 becomes ON. In this way, commands for recording video are received. It should be noted that the operation unit 105 is not limited to the above-described contents, and the operation unit 105 only needs to operate the camera 100. In addition, the operation unit 105 does not necessarily need to be built into the camera 100, and for example, it can be operated from an external device.

[0045] The camera 100 has a display unit 106. The display unit 106 displays the viewfinder image during video recording, displays captured image data, and displays characters for interactive operation. It should be noted that the display unit 106 does not necessarily need to be built into the camera 100. The camera 100 can be connected to an internal or external display unit 106, and the camera 100 only needs to have at least display control functions for controlling the display of the display unit 106.

[0046] Camera 100 has a recording medium 110. Recording medium 110 can record image data output from imaging unit 102. Recording medium 110 can be configured to be attached to and detached from camera 100, or it can be built into camera 100. That is, camera 100 only needs to have a unit for at least accessing recording medium 110.

[0047] The camera 100 has a connection unit 111. The connection unit 111 is an interface for connecting to an external device. In this embodiment, the camera 100 can exchange data with an external device via the connection unit 111. It should be noted that in this embodiment, the connection unit 111 includes an interface for communicating with an external device via a wireless LAN. The control unit 101 realizes wireless communication with the external device by controlling the connection unit 111. It should be noted that the communication method is not limited to a wireless LAN.

[0048] It should be noted that the camera 100 in this embodiment can operate as a client device in a wireless LAN communication infrastructure mode. When the camera 100 operates as a client device, it can join the network formed by a nearby access point (hereinafter referred to as AP) by connecting to it. Additionally, although the camera 100 in this embodiment is an AP type, it can also operate as a simplified AP (hereinafter referred to as a simplified AP) with more limited functionality. It should be noted that the AP in this embodiment is an example of a relay device. When the camera 100 operates as a simplified AP, it forms its own network. Peripheral devices of the camera 100 recognize it as an AP, and can join the network formed by the camera 100. As described above, it is assumed that the program for operating the camera 100 is stored in the non-volatile memory 103. It should be noted that although the camera 100 in this embodiment is an AP type, it is a simplified AP without gateway functionality for transmitting data received from client devices to Internet service providers, etc. Therefore, even if camera 100 receives data from other devices that have joined a network formed by camera 100 itself, camera 100 cannot transmit that data to a network such as the Internet. Alternatively, connection unit 111 can be an interface for wired communication rather than wireless communication. That is, regardless of the communication method, connection unit 111 can be an interface for wired communication (such as RS-232C, RS-422A, USB, Ethernet, etc.).

[0049] The above is a description of camera 100. Next, a description will be provided regarding mobile phone 200 as an example of an external device. <Configuration of Display Control Devices>

[0050] Figure 3 This is a block diagram illustrating the configuration of a mobile phone 200 according to a first embodiment. The mobile phone 200 is an example of a display control device. It should be noted that although a description of a mobile phone is provided herein as an example of a display control device, the display control device is not limited thereto. For example, the display control device may be an information processing device, such as a digital camera with wireless capabilities, a portable media player, a so-called tablet device, a personal computer, and a smartphone, etc.

[0051] Mobile phone 200 has a control unit 201. The control unit 201 controls the various units of mobile phone 200 according to input signals and the procedures described below. It should be noted that instead of the control unit 201 controlling the entire device, multiple hardware components can control the entire device by sharing the processing.

[0052] The mobile phone 200 has a camera unit 202. The camera unit 202 converts the light of the subject imaged by the lens included in the camera unit 202 into electrical signals, performs noise reduction processing, and outputs digital data as image data. After the image data captured by the camera unit 202 is stored in a buffer memory, the control unit 201 performs predetermined calculations, and the image data is recorded in the recording medium 210.

[0053] Mobile phone 200 has non-volatile memory 203. Non-volatile memory 203 is electrically erasable and recordable non-volatile memory, and the OS (operating system), which is the basic software executed by control unit 201, as well as various programs, are stored in non-volatile memory 203. It is assumed that the program for communicating with camera 100 is also maintained in non-volatile memory 203 and is installed as a camera control application. It should be noted that the processing of mobile phone 200 in this embodiment is achieved by reading the program provided by the camera control application. It should be noted that it is assumed that the camera control application has programs for using the basic functions of the OS installed in mobile phone 200 (e.g., wireless LAN function, Bluetooth function, and functions for calling other applications, etc.). In addition, the camera control application has a remote video recording function, which allows video recording by remotely operating camera 100 from mobile phone 200 when viewing a live view image obtained from camera 100 on mobile phone 200. Furthermore, the camera control application has the function of remotely viewing image data recorded on the recording medium mounted on the camera 100, and the function of remotely viewing image data. The camera control application executed by the control unit 201 is an example of an input unit that inputs a live view image obtained from the camera 100 as the input image.

[0054] It should be noted that the OS of the mobile phone 200 may have a program for implementing the processing in this embodiment.

[0055] The mobile phone 200 has a working memory 204. The working memory 204 is used as a buffer memory for temporarily storing image data generated by the camera unit 202, an image display memory for the display unit 206, and a working area for the control unit 201, etc.

[0056] Mobile phone 200 has an operation unit 205. The operation unit 205 is used to receive instructions from a user for mobile phone 200. The operation unit 205 includes, for example, operation components such as a power button for the user to indicate whether the power of mobile phone 200 is turned on / off, and a touch panel formed on display unit 206.

[0057] Mobile phone 200 has a display unit 206. The display unit 206 displays image data and characters for interactive operation, etc. It should be noted that the display unit 206 does not necessarily need to be built into the mobile phone 200. The mobile phone 200 can be connected to the display unit 206, and the mobile phone 200 only needs to have at least display control functions for controlling the display of the display unit 206. The display control functions performed by the control unit 201 are an example of a display control unit.

[0058] Mobile phone 200 has a recording medium 210. The recording medium 210 can record image data output from camera unit 202 and image data received from data processing device. The recording medium 210 can be configured to be attached to and detached from mobile phone 200, or it can be built into mobile phone 200. That is, mobile phone 200 only needs to have a unit for at least accessing the recording medium 210.

[0059] Mobile phone 200 has a connection unit 211. The connection unit 211 is an interface for connecting to an external device. In this embodiment, mobile phone 200 can exchange data with an external device via the connection unit 211. It should be noted that in this embodiment, the connection unit 211 includes an interface for communicating with an external device via a wireless LAN. Control unit 201 implements wireless communication with the external device by controlling the connection unit 211. It should be noted that in this embodiment, camera 100 can operate at least as a client device in a wireless LAN communication infrastructure mode and can join a network formed by surrounding access points (APs). Additionally, camera 100 can operate as a simplified AP, and mobile phone 200 can join the simplified AP of camera 100.

[0060] Mobile phone 200 has a public network connection unit 212. The public network connection unit 212 is an interface used for public wireless communication. Mobile phone 200 can make calls and data communications with other devices via the public network connection unit 212. During a call, control unit 201 inputs and outputs audio signals via microphone 213 and speaker 214. In this embodiment, it is assumed that the public network connection unit 212 includes an interface for communication using 3G (third-generation mobile communication system). It should be noted that the communication method is not limited to 3G, and other communication methods such as LTE, WiMAX, ADSL, FTTH, and so-called 4G (fourth-generation mobile communication system) can be used. LTE is an abbreviation for Long Term Evolution. WiMAX is an abbreviation for Global Microwave Access Interoperability. ADSL is an abbreviation for Asymmetric Digital Subscriber Line. FTTH is an abbreviation for Fiber to the Home. Furthermore, connection unit 211 and public network connection unit 212 do not necessarily need to be configured as separate hardware, and for example, connection unit 211 and public network connection unit 212 can also share a single antenna. Connection unit 211 and public network connection unit 212 are examples of communication units used for communicating with camera equipment.

[0061] The above is a description of the mobile phone 200. In this context, although the following description may present the mobile phone 200 as the subject of processing, in reality, the control unit 201 reads the program stored in the non-volatile memory 203 and implements various types of processing. Similarly, in the section where the camera 100 is described as the subject of processing, the control unit 101 reads the program stored in the non-volatile memory 103 and implements various types of processing. <Description of the screen of the camera control application on mobile phone 200>

[0062] Figure 4A and Figure 4B , Figures 5A to 5D as well as Figure 6A and Figure 6B This is an example of a camera control application screen displayed on the display unit 206 of the mobile phone 200. In this context, the camera control application connects to multiple cameras simultaneously and enables operation of each camera. Furthermore, the camera control application enables the receiving of video being captured by the camera and its current settings via communication with the camera, and displays the video and these settings on the camera control application screen. Additionally, the camera control application is an example of a control unit that, after zooming in / out at the operating device has ended, controls the camera device via the communication unit to change the zoom ratio according to the zoom level.

[0063] Figure 4A This is an example of a camera control application screen. Figure 4A In the example of camera control application screen 400, the video being captured and the camera's current settings are received from the camera via a communication connection, and the video and these settings are displayed on the screen.

[0064] The application settings area 410 is located within the camera control application screen 400. The application settings area 410 contains settings buttons for changing various settings of the camera control application. Figure 4A In the example of the camera control application screen 400, the camera connection setting button 411, the full-screen display button 412, and the marker setting button 413 are arranged in the application setting area 410. When the user presses the camera connection setting button 411, a transition occurs to the screen for setting the camera connection.

[0065] When the user presses the full-screen display button 412, such as Figure 4B As shown, the camera control application screen 400 displays the camera's video, which is shown in the main video area 450, on the full screen. When the camera's video is displayed on the full screen, as... Figure 4B As shown, the camera control application screen 400 displays the application settings area 410 by overlaying it onto the camera's video. It should be noted that, as... Figure 4B As shown, with the camera's video displayed in full screen, the operation screen display button 414 and the marker setting button 413 are arranged in the application settings area 410. When the user touches the operation screen display button 414, the camera control application screen 400 changes to... Figure 4A The screen shown is shown below. When the user touches the marked settings button 413, the camera control application screen 400 changes to... Figures 5A to 5D The image shown.

[0066] like Figure 4A As shown, a main video area 450 and a sub-video area 460 are arranged within the camera control application screen 400. The main video area 450 and the sub-video area 460 are areas used to display video received from the cameras. In this example, the camera control application is connected to multiple cameras simultaneously, and the application arranges and displays the videos being captured by each camera side-by-side in the sub-video area 460. Additionally, the video from the camera to be controlled by the camera control application is displayed in the main video area 450.

[0067] like Figure 4AAs shown, the recording information area 470 is arranged in the camera control application screen 400. The recording information area 470 is used to display the recording information and recording button of the camera to be controlled. For example, the recording information includes the remaining media time of the camera to be controlled and the timecode of the camera. Furthermore, when the recording button is pressed, the camera control application starts or stops recording the video being captured by the camera to be controlled.

[0068] like Figure 4A As shown, the operation area 480 is arranged within the camera control application screen 400. The operation area 480 is an area where operation buttons for operating the camera to be controlled are arranged. These operation buttons include, for example, buttons for operating the camera's exposure, shutter speed, zoom, and focus. When the user presses an operation button, the camera control application screen 400 performs an operation or displays an operation screen. For example, a PTZ operation button 481 is arranged within the operation area 480 to receive instructions for displaying a screen for performing PTZ operation on the camera to be controlled. Additionally, the user can scroll through the operation area 480 to display operation buttons located outside this area.

[0069] Figures 5A to 5D An example of a camera control application screen related to the display of markers is shown. The camera is controlled by the user by touching the markers. Figure 4A The application settings area 410 uses the marker settings button 413 to set the markers. Figures 5A to 5D The image shows a transition in the camera control application screen 400. Figures 5A to 5D In the example of the camera control application screen 400, a back button 431, a marker display toggle button 432, and a marker detail settings button 433 are arranged in the application settings area 410. When the user presses the back button 431, the camera control application screen 400 transitions to... Figure 4A The image shown.

[0070] When the user presses the marker display toggle button 432, the camera control application screen 400 switches between valid and invalid display of markers superimposed on the main video area 450. Figure 5A The marker display toggle button 432 indicates that the marker display is invalid. In this case, as... Figure 5A As shown, the video from the camera to be controlled is displayed in the main video area 450 without overlay display markers. The camera control application is an example of an overlay unit superimposed on the input image to specify the frames of subjects included in the input image.

[0071] Figure 5B , Figure 5C and Figure 5DThe marker display toggle button 432 indicates that the marker display is active. In this case, as... Figure 5B , Figure 5C and Figure 5D As shown, the video from the camera to be controlled is displayed overlaid with marked states in the main video area 450. Figure 5B The main video area 450 displays center marker 510 and grid marker 540. Figure 5C In the main video area 450, a 4:3 aspect ratio marker 560 displays a mask with 50% opacity. Figure 5D In the main video area 450, the 4:3 appearance marker 560 displays a mask with 100% opacity.

[0072] When the user touches the mark detail settings button 433, such as Figure 5B , Figure 5C and Figure 5D As shown, the camera control application screen 400 displays a marker detail settings screen 500. The marker detail settings screen 500 is an example of a setting unit for setting the display format of markers as overlay components. The marker detail settings screen 500 includes a close button 501 and interfaces for setting the display format of various markers (such as center marker setting 511, grid marker setting 541, and appearance marker setting 551). The marker detail settings screen 500 also includes interfaces for setting the display format of each marker, such as center marker type setting 521, horizontal marker setting 531, and marker aspect ratio setting 561. Furthermore, the marker detail settings screen 500 includes interfaces for setting the display format of each marker, such as safety zone marker setting 571, safety zone area setting 581, and marker opacity setting 591. When the user presses the close button 501, the camera control application screen 400 hides the marker detail settings screen 500.

[0073] like Figure 5B , Figure 5C and Figure 5D As shown, when the marker display toggle button 432 is active, the camera control application screen 400 displays markers in the main video area 450 according to the settings of the marker detail settings screen 500.

[0074] exist Figure 5B In the marker detail settings screen 500, the center marker setting 511 is set to blue, and the grid marker setting 541 is set to yellow. For this reason, in Figure 5B The main video area 450 displays a blue center marker 510 and a yellow grid marker 540.

[0075] exist Figure 5CIn the marker detail settings screen 500, the appearance marker setting 551 is set to a mask of 50%, and the marker aspect ratio setting 561 is set to 4:3. For this reason, in Figure 5C In the main video area 450, a 50% opacity mask is used to display the 4:3 appearance marker 560.

[0076] exist Figure 5D In the marker detail settings screen 500, the appearance marker setting 551 is set to mask 100%, and the marker aspect ratio setting 561 is set to 4:3. For this reason, in Figure 5D In the main video area 450, a 100% opacity mask is used to display the 4:3 appearance marker 560.

[0077] Figure 6A and Figure 6B This is an example of a camera control application screen related to PTZ operation. Figure 6A Camera control application screen 400 and Figure 6B In the camera control application screen 400, as will be described in detail below, the operating state associated with the direction limit button 623 is different. This is achieved by the user touching the button located on... Figure 4A The PTZ operation button 481 in the operation area 480 generates a signal to... Figure 6A and Figure 6B The camera control application screen 400 shown is a transition.

[0078] exist Figure 6A and Figure 6B The camera control application screen 400 displays the PTZ operation screen 600. The PTZ operation screen 600 includes a power off button 601, a zoom operation button 611, a direction button 621, a joystick 622, and a direction limit button 623. When the user presses the power off button 601, the camera control application screen 400 hides the PTZ operation screen 600. The zoom operation button 611 is an example of an operation unit for receiving zoom-in / zoom-out operations.

[0079] When the user presses the zoom operation button 611, the camera control application changes the zoom ratio of the camera 100, which is the camera to be controlled. Specifically, the camera control application controls the zoom ratio of the camera 100 to magnify when the zoom operation button 611 is pressed in the upward direction and to zoom out when the zoom operation button 611 is pressed in the downward direction. Furthermore, the camera control application displays the video received from the camera 100 after the zoom ratio change in the main video area 450 and the sub-video area 460 of the camera control application screen 400. For example, by zooming in using the camera 100, the viewing angle of the video to be captured changes from... Figure 7AChange to Figure 7B Additionally, for example, by using camera 100 to zoom in, the perspective of the video to be captured can be changed from... Figure 7A Change to Figure 7C It should be noted that if the operation of changing the zoom ratio of camera 100 is possible, the operation is not limited to using the zoom operation button 611. While the zoom operation button 611 is being pressed in the camera control application, the mobile phone 200 sends the change in zoom ratio to camera 100 at fixed intervals. The change in zoom ratio can be a predetermined value or a value set by the user.

[0080] When the user presses directional button 621, the camera control application changes the pan / tilt position of camera 100. When directional button 621 is pressed up, the camera control application adjusts the tilt of camera 100 upwards. When directional button 621 is pressed down, the camera control application adjusts the tilt of camera 100 downwards. When directional button 621 is pressed left, the camera control application adjusts the pan of camera 100 to the left. When directional button 621 is pressed right, the camera control application adjusts the pan of camera 100 to the right. When directional button 621 is pressed upper right, the camera control application adjusts the pan of camera 100 to the right and adjusts the tilt of camera 100 upwards. When directional button 621 is pressed lower right, the camera control application adjusts the pan of camera 100 to the right and adjusts the tilt of camera 100 downwards. When the directional button 621 is pressed and points to the lower left, the camera control application adjusts the pan of the camera 100 to the left and the tilt of the camera 100 to the lower right. When the directional button 621 is pressed and points to the upper left, the camera control application adjusts the pan of the camera 100 to the left and the tilt of the camera 100 to the upper right. While the directional button 621 is being pressed in the camera control application, the mobile phone 200 sends the pan and tilt amounts to the camera 100 at fixed intervals. The pan and tilt amounts can be predetermined values ​​or values ​​set by the user.

[0081] The joystick 622 is an interface for inputting direction and displacement relative to a reference position via sliding operation, where the center is the reference position. Using... Figure 6A The joystick 622 is operable in 360 degrees. In this case, such as Figure 6A As shown, the background of the joystick 622 clearly indicates that the operable direction is 360 degrees by coloring all directions within 360 degrees. Additionally, utilizing... Figure 6BThe joystick 622 can be operated in four directions: up, down, left, and right. In this case, such as... Figure 6B As shown, the background of the joystick 622 clearly indicates the four operable directions (up, down, left, and right) by coloring the background to indicate these directions. The user can switch the operable directions using the joystick 622 by pressing the direction restriction button 623. When the operable direction using the joystick 622 is 360 degrees, the camera control application screen 400 is displayed. Figure 6A The direction restriction button 623 is shown. When the directions operable using the joystick 622 are up, down, left, and right, the camera control application screen 400 is displayed. Figure 6B The direction restriction button 623 is shown.

[0082] When the user performs a sliding operation on the joystick 622, such as Figure 8 As shown, the camera control application determines the horizontal displacement dh and vertical displacement dv based on the input direction θ and displacement d. Direction θ is the angle of the joystick 622 with the reference position centered, and has a value range equal to or greater than 0 degrees and less than 360 degrees (equal to or greater than 0 rad (radians) and less than 2π rad). It should be noted that when the operable directions using the joystick 622 are up, down, left, and right, direction θ is limited to 0 degrees, 90 degrees, 180 degrees, and 270 degrees (0 rad, π / 2 rad, π rad, and 3π / 2 rad). The horizontal displacement dh is calculated using d × cosθ. Furthermore, the vertical displacement dv is calculated using d × sinθ.

[0083] When the horizontal displacement dh is positive, the camera control application adjusts the pan of camera 100 to the right at a speed corresponding to the absolute value of the horizontal displacement dh. When the horizontal displacement dh is negative, the camera control application adjusts the pan of camera 100 to the left at a speed corresponding to the absolute value of the horizontal displacement dh. When the vertical displacement dv is positive, the camera control application adjusts the pitch of camera 100 upward at a speed corresponding to the absolute value of the vertical displacement dv. When the vertical displacement dv is negative, the camera control application adjusts the pitch of camera 100 downward at a speed corresponding to the absolute value of the vertical displacement dv.

[0084] It should be noted that the calculation method is not limited to this if the horizontal and vertical displacements of the joystick 622 can be obtained from a reference position. During a sliding operation of the joystick 622 in a camera control application, the mobile phone 200 sends pan and pitch movements to the camera 100 at fixed intervals. The pan movement is a value obtained by multiplying the horizontal displacement by a predetermined coefficient. The pitch movement is a value obtained by multiplying the vertical displacement by a predetermined coefficient. In this context, the predetermined coefficient can be a predetermined value or a value set by the user. It should be noted that the calculation method is not limited to this if the pan and pitch movements can be determined based on the operation of the joystick 622. <Instructions for zooming via pinch gestures>

[0085] Reference Figures 9A to 13H An example of zoom operation via pinch gesture in the camera control application of mobile phone 200 is provided. This pinch gesture is an example of an operation unit for receiving zoom-in / zoom-out operations.

[0086] Figure 9A and Figure 9B This is a diagram used to illustrate the pinching action. In Figure 9A and Figure 9B In the diagram, contact points A and B are the points on the touch panel 701 that are touched by a finger. Additionally, in... Figure 9A and Figure 9B In the figure, AB indicates the distance between contact point A and contact point B on the touch panel 701. Figure 9A The distance AB is longer than Figure 9B The distance AB. Pinch is (from...) Figure 9A Transform to Figure 9B The operation involves moving the finger on the touch panel 701 by pinching it together. Pinch is (to move the finger from...) Figure 9B Transform to Figure 9A (In a manner that allows movement of the fingers on the touch panel 701 by spreading them out)

[0087] When the zoom level of the camera 100 is changed via a pinch operation on the mobile phone 200, the camera control application performs the processing described below. The camera control application is based on... Figure 9A The distance AB and Figure 9B The zoom ratio is determined by the ratio of distance AB, and the camera control application sends the determined zoom ratio to camera 100 via network 112. During the pinch operation, making... Figure 9B The distance AB relative to Figure 9A When the distance AB becomes 1 / 2, the camera control application controls camera 100, causing the zoom ratio to become 1 / 2. During the pinch operation, making... Figure 9AThe distance AB relative to Figure 9B When the distance AB is doubled, the camera control application controls camera 100 to double the zoom ratio. Zoom ratio is an example of magnification / reduction ratio, indicating magnification when the zoom ratio is greater than 1 and reduction ratio when the zoom ratio is less than 1.

[0088] Reference Figures 10A to 10C and Figures 13A to 13H Specific examples of zooming via pinch gestures are provided. Figures 10A to 10C and Figures 13A to 13H Is Figures 4A to 6B This is an example of a pinch operation displayed in the main video area 450 of the camera control application screen 400. The main video area 450 is displayed on the display unit 206 of the mobile phone 200, which has a touch panel, and touch operation is possible.

[0089] Reference Figures 10A to 10C Specific examples are provided regarding situations where the zoom ratio of camera 100 changes in response to a pinch operation. For example... Figure 10A As shown, the video from camera 100 is displayed in the main video area 450 of the camera control application screen 400. The camera control application follows pinch operations on the main video area 450 and changes the zoom level of camera 100. Figure 10C This shows the distance between the fingers in contact via a pinching operation relative to... Figure 10B An example that becomes 1 / 2. Therefore, Figure 10C The zoom ratio of the video of the camera 100 shown is relative to... Figure 10B The zoom ratio of the video from the camera 100 shown is 1 / 2.

[0090] Reference Figures 11A to 11C Specific examples are provided regarding situations where the zoom ratio of camera 100 changes in response to a pinch operation. For example... Figure 11A As shown, the video from camera 100 is displayed in the main video area 450 of the camera control application screen 400. The camera control application follows pinch operations on the main video area 450 and changes the zoom level of camera 100. Figure 11C This shows the distance between the fingers in contact via a pinching operation relative to... Figure 11B The example becomes twice as large. Therefore, Figure 11C The zoom ratio of the video of the camera 100 shown is relative to... Figure 11B The zoom level of the video from the camera 100 shown is doubled.

[0091] Will be through reference Figures 12A to 12H A specific example is provided regarding the case where the viewing angle of the camera 100 after the zoom ratio is changed is adjusted by pinching. Figure 12B , Figure 12C , Figure 12F and Figure 12G The view frame 801 is a frame used to indicate the view of the video from the camera 100 after the zoom ratio has changed. Figure 12E , Figure 12F , Figure 12G and Figure 12H Marker 802 is an overlay element displayed on the video of camera 100 to confirm the viewing angle. Figure 12E , Figure 12F , Figure 12G and Figure 12H In the example, a grid marker is shown as marker 802. It should be noted that marker 802 only needs to be an element that is overlaid on the video accordingly, and as... Figure 5B , Figure 5C and Figure 5D The markings shown may be other markings such as center markers and appearance markers, or they may be autofocus frames. Marker 802 is an example of an overlay component displayed on the input image. Display unit 206 is an example of a display device capable of displaying the input image and the overlay component displayed on the input image in the main video area 450, which serves as the first area.

[0092] Figure 12A This is an example of displaying the video of camera 100 in the main video area 450 of camera control application screen 400. Figure 12E Is Figure 12A The example shown is an overlay display of marker 802 in the indicated state. Figure 12E As shown, mark 802 is superimposed on the video of camera 100.

[0093] Figure 12B Is Figure 12A Examples of pinching actions starting in the video. For example... Figure 12B As shown, by initiating the pinch operation, a view frame 801 is displayed on the video corresponding to the video of the camera 100. Figure 12F Is Figure 12B The example shown is an overlay of marker 802 in the displayed state.

[0094] Figure 12C From Figure 12B The example shown illustrates a pinch operation performed in the indicated state. The camera control application determines the zoom level of the camera 100 based on the pinch operation on the main video area 450, and as shown... Figure 12C As shown, the video is displayed in the center of the main video area 450 by reducing the video of the camera 100 based on the determined zoom ratio. Figure 12CIn the image, the color of the area where no video is displayed (the background area) is expected to be different from the color of marker 802, making marker 802 easily identifiable visually. Figure 12G Is Figure 12C The example shown is an overlay display of marker 802 in the indicated state. Figure 12G In the image, viewpoint frame 801 indicates the viewpoint of the video from camera 100 after the zoom ratio has changed. For this reason, as... Figure 12G As shown, mark 802 did not shrink by the pinching operation.

[0095] Figure 12C It shows relative to Figure 12B An example where the distance between the fingers in contact during a pinching operation is reduced to 1 / 2. Therefore, Figure 12C The zoom level in the middle becomes... Figure 12B Half the zoom ratio in the image. For this reason, Figure 12C The video size (vertical and horizontal) becomes Figure 12B Half of the video. The camera control application changes the camera's zoom level by 100 when the pinch operation ends. Figure 12C When the pinch operation ends in the specified state, the zoom ratio of camera 100 changes to 1 / 2, and the video captured by camera 100 becomes... Figure 12D The perspective shown. Figure 12H Is Figure 12D The example shown is an overlay display of marker 802 in the indicated state. Figure 12G and Figure 12H As shown, marker 802 is displayed at approximately the same position relative to the subject in the video.

[0096] Will be through reference Figures 13A to 13H A specific example is provided regarding the situation where the field of view of the camera 100 after the zoom ratio is changed is adjusted by pinching.

[0097] Figure 13B , Figure 13C , Figure 13F and Figure 13G The view frame 801 is a frame used to indicate the view of the video from the camera 100 after the zoom ratio has changed. Figure 13E , Figure 13F , Figure 13G and Figure 13H Marker 802 is an overlay element displayed on the video of camera 100 to confirm the viewing angle. Figure 13E , Figure 13F , Figure 13G and Figure 13HIn the example, a grid marker is shown as marker 802. It should be noted that marker 802 only needs to be an element that is overlaid on the video accordingly, and as... Figure 5B , Figure 5C or Figure 5D The markings shown can be other markings such as center markers and appearance markers, or they can be autofocus frames.

[0098] Figure 13A This is an example of displaying the video of camera 100 in the main video area 450 of camera control application screen 400. Figure 13E Is Figure 13A The example shown is an overlay display of marker 802 in the indicated state. Figure 13E As shown, mark 802 is superimposed on the video of camera 100.

[0099] Figure 13B Is Figure 13A Examples of pinching actions starting in the video. For example... Figure 13B As shown, by initiating the pinch operation, a view frame 801 is displayed on the video corresponding to the video of the camera 100. Figure 13F Is Figure 13B The example shown is an overlay of marker 802 in the displayed state.

[0100] Figure 13C From Figure 13B The example shown illustrates a pinch operation performed in the indicated state. The camera control application determines the zoom level of the camera 100 based on the pinch operation on the main video area 450, and as... Figure 13C As shown, based on the determined zoom ratio, the camera control application reduces the viewing angle frame 801 and overlays the viewing angle frame 801 in the center of the video of the camera 100. Figure 13G Is Figure 13C The example shown is an overlay display of marker 802 in the indicated state. Figure 13G In the image, viewpoint frame 801 indicates the viewpoint of the video from camera 100 after the zoom ratio has changed. For this reason, as... Figure 13G As shown, marker 802 is correspondingly scaled down with view frame 801.

[0101] Figure 13C It shows relative to Figure 13B An example of doubling the distance between the fingers in contact through a pinching operation. Therefore, Figure 13C The zoom level in the middle becomes... Figure 13B It has twice the zoom ratio of [the standard zoom lens]. For this reason, Figure 13C The size (vertical and horizontal) of the view frame 801 relative to Figure 13BThe field of view 801 changes to the reciprocal of 2 (1 / 2). The camera control application changes the zoom level of camera 100 when the pinch operation ends. Figure 13C When the pinch operation ends in the specified state, the zoom level of camera 100 doubles, and the video captured by camera 100 becomes... Figure 13D The perspective shown. Figure 13H Is Figure 13D The example shown is an overlay display of marker 802 in the indicated state. Figure 13G and Figure 13H As shown, marker 802 is displayed at approximately the same position relative to the subject in the video. <Explanation of the process for adjusting the zoom angle via pinch operation>

[0102] Figure 14 This is a flowchart of the zoom angle adjustment process in mobile phone 200 via pinch operation. Figure 14 The flowchart illustrates how the control unit 201 loads the program recorded in the non-volatile memory 203 into the working memory 204 and executes the program. (See attached diagram.) Figure 1 Starting with, regarding Figure 14 The process will be explained in detail.

[0103] In step S101, the control unit 201 determines whether a function such as [missing information] is detected at the operation unit 205. Figure 9A and Figure 9B The control unit 201 determines whether a pinch operation has started by detecting a pinch operation, etc. If the control unit 201 determines that a pinch operation has started by detecting a pinch operation, it executes step S102. If the control unit 201 determines that a pinch operation has not started by not detecting a pinch operation, the process ends. Figure 14 The process.

[0104] In step S102, the control unit 201 aligns the viewing frame 801 with the video area displayed on the display unit 206 (in... Figures 4A to 6B In the example, the main video area (450) is displayed accordingly. In such... Figure 12A When video is displayed in the video area as shown, the control unit 201... Figure 12B The view frame shown is 801. In, as... Figure 12E When video is displayed in the video area as shown, the control unit 201... Figure 12F The view frame shown is 801. In the example shown... Figure 13A When video is displayed in the video area as shown, the control unit 201... Figure 13B The view frame shown is 801. In, as... Figure 13E When video is displayed in the video area as shown, the control unit 201... Figure 13FThe view frame 801 is shown. Note that view frame 801 can be displayed before the pinch operation begins.

[0105] In step S103, the control unit 201 detects the amount of the pinching operation in the operation unit 205. Figure 9A and Figure 9B In the case shown, the control unit 201 calculates the change in distance AB between the contact points of the fingers from the start of the pinch operation as the operation quantity. This is achieved by the pinch operation... Figures 9A to 9B With the change in distance AB and the distance becoming half, the amount of manipulation in the pinch operation becomes half. Furthermore, the amount of manipulation caused by the pinch operation... Figures 9B to 9A With changes in distance AB and the doubling of distance AB, the amount of pinching operation is doubled.

[0106] In step S104, the control unit 201 calculates the zoom ratio of the camera 100 based on the amount of the pinch operation detected in step S103. Specifically, when the amount of the pinch operation is 1 / 2, the control unit 201 sets the zoom ratio to 1 / 2, and when the amount of the pinch operation is twice, the control unit 201 sets the zoom ratio to twice.

[0107] In step S105, the control unit 201 determines whether the pinching operation detected by the operation unit 205 is a pinching together or a pinching apart. If the control unit 201 determines that the pinching operation is a pinching together, the processing in step S106 is executed. If the control unit 201 determines that the pinching operation is not a pinching together (that is, a pinching apart), the processing in step S107 is executed. A pinching together operation is (to...) Figure 9A Change to Figure 9B (The method) involves pinching two fingers together on the touch panel. Additionally, the pinch-to-split gesture is (from...) Figure 9B Change to Figure 9A The gesture operation involves spreading two fingers on the touch panel. Furthermore, as explained in steps S103 and S104, the zoom ratio increases or decreases as the amount of the pinch operation increases or decreases. For this reason, the control unit 201 determines that the operation is a pinch when the zoom ratio is less than 1, and determines that the operation is a pinch when the zoom ratio is greater than 1.

[0108] In step S106, the control unit 201 changes the size of the video displayed in the video area of ​​the camera 100 based on the zoom ratio calculated in step S104, and the control unit 201 centers the video in the video area. Specifically, if the zoom ratio calculated in step S104 is 1 / 2, the control unit 201 reduces the size (vertical and horizontal) of the video displayed in the video area of ​​the camera 100 to half. By setting the zoom ratio to 1 / 2, the size of the video displayed in the video area is reduced from... Figure 12B Change to Figure 12C With marker 802 displayed in the video area, by setting the zoom ratio to 1 / 2, the size of the video displayed in the video area increases from... Figure 12F Change to Figure 12G .like Figure 12G As shown, when marker 802 is displayed in the video area, the size of marker 802 does not change even if the size of the video changes.

[0109] In step S107, the control unit 201 changes the size of the viewing frame 801 based on the zoom ratio calculated in step S104, and the control unit 201 positions the viewing frame 801 in the center of the video area. Specifically, if the zoom ratio calculated in step S104 is double, the control unit 201 shrinks the viewing frame 801, making its vertical and horizontal sizes each the reciprocal (1 / 2) of the zoom ratio. By doubling the zoom ratio, the size of the viewing frame 801 displayed in the video area changes from... Figure 13B Change to Figure 13C .

[0110] In step S108, the control unit 201 determines whether to display marker 802 in accordance with the video. If the control unit 201 determines that marker 802 is displayed, the process in step S109 is executed. If the control unit 201 determines that marker 802 is not displayed, the process in step S110 is executed.

[0111] In step S109, the control unit 201 changes the size of the marker 802 accordingly to the view frame 801, the size of which was changed in step S107. Figure 13F As shown, in the case of display mark 802, as Figure 13G As shown, marker 802 and view frame 801 are correspondingly scaled down.

[0112] In step S110, the control unit 201 determines whether the pinching operation in the operation unit 205 has ended. If the control unit 201 determines that the pinching operation has ended, the processing in step S111 is executed. If the control unit 201 determines that the pinching operation has not ended, the processing continues by returning to step S103.

[0113] In step S111, the control unit 201 hides the view frame 801 displayed in step S102.

[0114] In step S112, if the video size is changed in step S106, the control unit 201 returns the video to its size before the pinch operation, and the control unit 201 displays the video on the display unit 206. Specifically, in such cases... Figure 12C When the video is displayed at a reduced size, as shown Figure 12A As shown, the video size is returned to the state before the pinch operation. Furthermore, in step S112, if the size of marker 802 is changed in step S109, the control unit 201 returns marker 802 to the state before the pinch operation, and the control unit 201 displays marker 802 in the video area. Specifically, in... Figure 13G In the case of the reduced mark 802 shown, as Figure 13E As shown, this returns the size of marker 802 to its state before the pinch operation.

[0115] In step S113, the control unit 201 sends the zoom ratio calculated in step S104 to the camera 100 and changes the zoom ratio of the camera 100. Furthermore, the control unit 201 receives the video captured by the camera 100 after the zoom ratio change and displays the video on the display unit 206. When the zoom ratio of the camera 100 is changed by a pinch operation, the video captured by the camera 100 is as follows: Figures 12A to 12D The change is shown. When the zoom level of camera 100 is changed by pinching while marker 802 is displayed in the video area, the video captured by camera 100 is as follows: Figures 12E to 12H The change is shown. When the zoom ratio of camera 100 is changed by pinching, the video captured by camera 100 is as follows. Figures 13A to 13D The change is shown. When the zoom level of camera 100 is changed by a pinch operation while marker 802 is displayed in the video area, the video captured by camera 100 is as follows: Figures 13E to 13H The changes are shown.

[0116] Despite Figure 14 The flowchart provides an example of adjusting the video perspective of a zoomed camera using a pinch gesture, but this disclosure is not limited thereto. For example, in this disclosure, one can adjust the perspective of a zoomed camera by touching... Figure 6A Zoom operation button 611 and Figure 6B The zoom operation button 611 is used to adjust the viewing angle of the camera video after zooming.

[0117] As explained above, in the display control device of the first embodiment, since the viewing angle of the zoomed camera video is adjusted by pinch operation, the zoom control via pinch operation appears to have good tracking accuracy. Furthermore, in the display control device of the first embodiment, since the markers are displayed corresponding to the video viewing angle, the markers can be displayed at appropriate positions even when the video viewing angle is being adjusted. Additionally, in the display control device of the first embodiment, by changing the color of the background image displayed when the video is zoomed out to a color different from the marker color, the reduction in the visibility of the overlaid markers can be suppressed. That is, it is desirable that the color of the background image is a color not included in the markers that serve as overlay components.

[0118] Although the first embodiment illustrates an example of adjusting the camera's video perspective after zooming via a pinch operation, the camera's video perspective can be adjusted after panning-tilt-zooming. In the case of a pinch operation, for example, as... Figure 15 As shown, the camera video is reduced at a position based on the pinch position, and the camera's pan-tilt-zoom can be controlled to reduce the video size at that position. In this case, as explained in the first embodiment, the marker is not reduced. It should be noted that the position of the reduced video size can be changed while performing the pinch operation. Additionally, in the case of a pinch-to-divide operation, for example, as... Figure 16 As shown, the view frame is reduced at a position based on the pinch-off location, and the camera's pan-tilt-zoom can be controlled to magnify the camera's video within the view frame. In this case, as described in the first embodiment, the marker and view frame are correspondingly reduced. It should be noted that the position of the reduced view frame can be changed while performing the pinch-off operation. <Second Embodiment>

[0119] A second embodiment of this disclosure will be described. In this second embodiment, the case where the viewing angle is narrowed by a pinch operation will be described. Since the camera and display control device, which are configuration elements in this embodiment, are respectively similar to... Figures 1 to 3 The camera and display control equipment are included, therefore descriptions are omitted. <Explanation of the process for adjusting the zoom angle via pinch operation>

[0120] Figure 17 This is a flowchart of the zoom angle adjustment process via pinch operation in a mobile phone 200 according to a second embodiment of the present disclosure. Figure 17The flowchart is implemented by the control unit 201 loading the program recorded in the non-volatile memory 203 into the working memory 204 and executing the program. Figure 17 In the steps of the flowchart, regarding the performance with Figure 14 The steps in the flowchart are the same as the processing steps, by assigning the same... Figure 14 The same reference numerals are used in the accompanying drawings to omit the description.

[0121] In step S105, the control unit 201 determines whether the pinching operation detected by the operation unit 205 is a pinching together or a pinching apart. If the control unit 201 determines that the pinching operation is a pinching together, the processing in step S201 is executed. If the control unit 201 determines that the pinching operation is not a pinching together (that is, a pinching apart), the processing in step S205 is executed.

[0122] In step S201, the control unit 201 changes the viewing frame 801 displayed in step S102 to a viewing frame used to indicate pinching. Specifically, as shown... Figure 18A As shown, an inward arrow 811 is displayed near view frame 801. The combination of arrow 811 and view frame 801 is an example of a view frame used to indicate pinch. A view frame used to indicate pinch only needs to be distinguished from a view frame used to indicate split, and is not limited to, for example... Figure 18A The display format, etc.

[0123] In step S202, the control unit 201 changes the size of the viewing frame 801 based on the zoom ratio calculated in step S104, and the control unit 201 positions the viewing frame 801 in the center of the video area. Specifically, if the zoom ratio calculated in step S104 is 1 / 2, the control unit 201 shrinks the viewing frame 801, making its size (vertical and horizontal) halved. By setting the zoom ratio to 1 / 2, the size of the viewing frame 801 displayed in the video area changes from... Figure 12B Change to Figure 19A .

[0124] In step S203, the control unit 201 determines whether to display marker 802 in accordance with the video. If the control unit 201 determines that marker 802 is displayed, the process in step S204 is executed. If the control unit 201 determines that marker 802 is not displayed, the process in step S110 is executed.

[0125] In step S204, the control unit 201 changes the size of the marker 802 based on the zoom ratio calculated in step S104, and the control unit 201 positions the marker 802 in the center of the video area. Specifically, if the zoom ratio calculated in step S104 is 1 / 2, the control unit 201 enlarges the marker 802 so that the size of the marker 802 (vertical and horizontal) becomes the reciprocal (twice) of the zoom ratio. By setting the zoom ratio to 1 / 2, the size of the marker 802 displayed in the video area changes from... Figure 12F Change to Figure 19B In other words, during the shrinking operation, as the shrinkage rate decreases, the mark 802, which is a superimposed component, is magnified and displayed.

[0126] In step S205, the control unit 201 changes the viewing frame 801 displayed in step S102 to a viewing frame used to indicate pinching. Specifically, as shown... Figure 18B As shown, an outward arrow 812 is displayed near view frame 801. The combination of arrow 812 and view frame 801 is an example of a view frame used to indicate pinching. A view frame used to indicate pinching only needs to be distinguished from a view frame used to indicate pinching, and is not limited to, for example... Figure 18B The display format, etc.

[0127] In step S206, if the size of the mark 802 is changed in step S109 or step S204, the control unit 201 returns the mark 802 to its state before the pinch operation and displays the mark 802 on the display unit 206. Figure 13G In the case of the reduced mark 802 shown, as Figure 13E As shown, this returns marker 802 to its state before the pinch operation. In... Figure 19B In the case of the magnified mark 802 shown, as Figure 12E As shown, this returns marker 802 to its state before the pinch operation.

[0128] As explained above, in the second embodiment, even when the viewing angle of the zoomed-out camera's video is adjusted without zooming out the video display, markers can be displayed accordingly to the viewing angle of the zoomed-out camera's video. <Third Embodiment>

[0129] A third embodiment of this disclosure will be provided for description. In this third embodiment, a situation will be described where the marker is temporarily hidden while the camera is adjusting the viewing angle of the video after zooming. Since the camera in this embodiment is similar to... Figures 1 to 3 The camera shown, and the display control device of this embodiment, are similar to... Figures 1 to 3 The display control device shown is therefore omitted from the description. <Explanation of the process for adjusting the zoom angle via pinch operation>

[0130] Figure 20 This is a flowchart of the zoom angle adjustment process via pinch operation in a mobile phone 200 according to a third embodiment of the present disclosure. Figure 20 The flowchart is implemented by the control unit 201 loading the program recorded in the non-volatile memory 203 into the working memory 204 and executing the program. Figure 20 In the steps of the flowchart, for performing with Figure 14 The corresponding steps in the flowchart are the same as the processing steps, and are assigned the same as... Figure 14 The same reference numerals are used in the accompanying drawings, and their descriptions are omitted.

[0131] In step S301, the control unit 201 determines whether to display marker 802 in accordance with the video. If the control unit 201 determines that marker 802 is displayed, the processing in step S302 is executed. If the control unit 201 determines that marker 802 is not displayed, the processing in step S110 is executed.

[0132] In step S302, the control unit 201 temporarily hides the marker 802. (The sentence appears to be incomplete and requires further context.) Figure 12E As shown, if mark 802 is displayed before the pinch operation, such as Figure 12C As shown, marker 802 is temporarily hidden during the kneading operation.

[0133] In step S303, the control unit 201 determines whether to display marker 802 in accordance with the video. If the control unit 201 determines that marker 802 is displayed, the process in step S304 is executed. If the control unit 201 determines that marker 802 is not displayed, the process in step S110 is executed.

[0134] In step S304, the control unit 201 temporarily hides the marker 802. (The sentence appears to be incomplete and requires further context.) Figure 13E As shown, if mark 802 is displayed before the pinch operation, such as Figure 13C As shown, marker 802 is temporarily hidden during the pinching operation.

[0135] In step S305, if the video size is changed in step S106, the control unit 201 returns the video to its size before the pinch operation, and the control unit 201 displays the video on the display unit 206. Specifically, in such cases... Figure 12C When the video is displayed at a reduced size, as shown Figure 12AAs shown, the video size is restored to its state before the pinch operation. Furthermore, in step S305, if the marker was temporarily hidden in step S302 or S304, the control unit 201 redisplays the marker in the video area. (As shown in...) Figure 12C In the case of temporarily hiding marker 802, as shown Figure 12E As shown, mark 802 is redisplayed. In... Figure 13C In the case of temporarily hiding marker 802, as shown Figure 13E As shown, mark 802 is redisplayed.

[0136] As explained above, in the third embodiment, by temporarily hiding the marker while the camera is adjusting the viewing angle of the video after zooming, information superimposed on the video is suppressed, and the visibility of the video can be ensured. <Fourth Embodiment>

[0137] A fourth embodiment of this disclosure will be described. In this fourth embodiment, a case will be described where a predetermined marker is set to be displayed while the field of view of the video from the camera is being adjusted after zooming. Since the camera, which is a constituent element of this embodiment, is similar to... Figures 1 to 3 The camera in the image, and since the display control device, which is a constituent element of this embodiment, is similar to... Figures 1 to 3 The display control device in the text is omitted from the description. <Explanation of the process for adjusting the zoom angle via pinch operation>

[0138] Figure 21 This is a flowchart of the zoom angle adjustment process via pinch operation in a mobile phone 200 according to the fourth embodiment of this disclosure. Figure 21 The flowchart is implemented by the control unit 201 loading the program recorded in the non-volatile memory 203 into the working memory 204 and executing the program. Figure 21 In the steps of the flowchart, for performing with Figure 14 The corresponding steps in the flowchart are the same as the processing steps, and are assigned the same as... Figure 14 The same reference numerals are used in the accompanying drawings, and their descriptions are omitted.

[0139] In step S401, the control unit 201 displays the video area (in the display unit 206) on the display unit 206. Figure 4A and Figure 4B as well as Figure 6A and Figure 6B In the example, a predefined marker is displayed in the main video area (450). For example... Figure 22A and Figure 23A As shown, if no mark is displayed before the pinch operation, the control unit 201 displays a predetermined mark 803 in the video area immediately after the pinch operation begins. Figure 22B and Figure 23B As shown, when marker 802 is displayed before the pinch operation, the control unit 201 displays a predetermined marker 803 instead of marker 802 in the video area. Figure 22A and Figure 22B as well as Figure 23A and Figure 23B In the example, marker 802 is a grid marker, and marker 803 is a safety zone marker. It should be noted that markers 802 and 803 can be other types of markers besides grid markers or safety zone markers. Furthermore, markers 802 and 803 can be markers of different types, or they can be markers of the same type.

[0140] In step S402, if the video size is changed in step S106, the control unit 201 returns the video to its size before the pinch operation and displays the video on the display unit 206. Figure 22A and Figure 22B As shown, when the displayed video shrinks during a pinch operation, the video size is returned to its state before the pinch operation. Furthermore, in step S402, the control unit 201 returns the marker 802 displayed in the video area to its state before the pinch operation. If the marker was not displayed before the pinch operation, as shown... Figure 22A and Figure 23A As shown, the control unit 201 hides the marker 803. If marker 802 is displayed before the pinch operation, as... Figure 22B and Figure 23B As shown, the control unit 201 hides marker 803 and displays marker 802 in accordance with the video.

[0141] It should be noted that, Figure 21 In the flowchart, although an example of displaying a predetermined mark during the pinch operation is provided, the mark can be a user-specified mark.

[0142] As explained above, in the fourth embodiment, since a predetermined marker is displayed while the viewing angle of the video from the zoomed-out camera is being adjusted, marker display suitable for adjusting the viewing angle of the video can be performed. <Fifth Embodiment>

[0143] The fifth embodiment of this disclosure will be described. In this fifth embodiment, a subject frame will be displayed instead of a marker during the adjustment of the viewing angle of the video from a zoomed-in camera. Instead of displaying a marker corresponding to the viewing angle of the video, a subject frame is displayed corresponding to the subject included in the video. The subject frame may be, for example, a face detection frame, a pupil detection frame, or an automatic tracking frame in the case of automatic person tracking. Since the camera in this embodiment is similar to... Figures 1 to 3The camera in the image, and because the display control device in this embodiment is similar to... Figures 1 to 3 The display control device in the text is omitted from the description. <Explanation of the process for adjusting the zoom angle via pinch operation>

[0144] Figure 24 This is a flowchart of the zoom angle adjustment process via pinch operation in a mobile phone 200 according to the fifth embodiment of this disclosure. Figure 24 The flowchart provides an example of displaying a face detection box as a subject box. Figure 24 The flowchart is implemented by the control unit 201 loading the program recorded in the non-volatile memory 203 into the working memory 204 and executing the program. Figure 24 In the steps of the flowchart, for performing with Figure 14 The corresponding steps in the flowchart are the same as the processing steps, and are assigned the same as... Figure 14 The same reference numerals are used in the accompanying drawings, and their descriptions are omitted.

[0145] In step S501, the control unit 201 determines whether the video area displayed on the display unit 206 is within the specified range. Figure 4A and Figure 4B as well as Figure 6A and Figure 6B In the example, a subject frame is displayed in the main video area 450. If the control unit 201 determines that the subject frame is displayed, the process in step S502 is executed. If the control unit 201 determines that the subject frame is not displayed, the process in step S110 is executed. The subject frame is a frame used to specify a subject, such as a face, detected from the video of the camera 100. Although the method used to detect the subject is, for example, YOLO (You Only See Once), a subject detection technology using deep learning, the method only needs to be a method capable of detecting the subject.

[0146] In step S502, the control unit 201 changes the size of the subject frame accordingly to the video whose size was changed in step S106. In such cases... Figure 25 When the video displayed in the video area is reduced, the control unit 201 will also reduce the size of the subject frame 821 accordingly to the video.

[0147] In step S110, the control unit 201 determines whether the pinching operation in the operation unit 205 has ended. If the control unit 201 determines that the pinching operation has ended, the processing in step S111 is executed. If the control unit 201 determines that the pinching operation has not ended, the processing continues by returning to step S103.

[0148] When it is determined in step S110 that the pinch operation has ended, in step S113, the control unit 201 sends the zoom ratio calculated in step S104 to the camera 100, and the control unit 201 changes the zoom ratio of the camera 100. In step S503, the control unit 201 receives the video from the camera 100 after the zoom ratio has been changed, and the control unit 201 re-detects the subject based on the received video. Furthermore, as... Figure 25 and Figure 26 As shown, the control unit 201 displays the subject frame for specifying the detected subject on the video of the camera 100 displayed in the video area.

[0149] As explained above, in the fifth embodiment, the subject frame can be appropriately displayed even when the viewing angle of the video from the zoomed camera is adjusted. <Other Embodiments>

[0150] Although the present disclosure has been described in detail above based on preferred embodiments, the present disclosure is not limited to the specific embodiments, and various forms are included in the present disclosure without departing from the spirit of the disclosure. A portion of the above embodiments may be suitably combined.

[0151] In addition, this disclosure also includes situations where a software program for implementing the functions of the above embodiments is supplied directly from a recording medium or via wired / wireless communication to a system or device having a computer capable of executing the program, and the program is executed.

[0152] Therefore, in order to implement the functional processing of this disclosure by a computer, the program code supplied to and installed in the computer itself also implements this disclosure. That is, the computer program used to implement the functional processing of this disclosure is itself included in this disclosure. In this case, if the program has program functionality, the form of the program is not limited, and it can be, for example, object code, a program executed by an interpreter, and script data supplied to the OS, etc.

[0153] As a recording medium for supplying the program, for example, magnetic recording media such as hard disks and magnetic tapes, optical / magneto-optical storage media, or non-volatile semiconductor memory can be used. Additionally, as a method for supplying the program, it is conceivable to store the computer program forming this disclosure on a server on a computer network, and to have a connected client computer download and execute the computer program.

[0154] This disclosure can also be implemented by supplying a program for implementing one or more functions of the above embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. Alternatively, this disclosure can also be implemented by circuitry (e.g., an ASIC) for implementing one or more functions.

[0155] As described above, although preferred embodiments of the present disclosure have been provided, the present disclosure is not limited to these embodiments, and various modifications and changes are possible within the scope of the spirit of the present disclosure.

[0156] According to this disclosure, techniques for adjusting the perspective of an image can be improved.

[0157] Other embodiments The embodiments of the present invention can also be implemented by the following method: providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0158] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

[0159] This application claims the benefit of Japanese Patent Application No. 2025-051224, filed on March 26, 2025, the entire contents of which are incorporated herein by reference.

Claims

1. A display control device, comprising: An input component is used to input an input image; Operating components are used to receive zoom-in / zoom-out commands; as well as The display control unit is configured to, during the operation of the operation unit receiving a magnification operation, overlay a box indicating a second region corresponding to the magnification onto the input image corresponding to a first region displayed by the display unit, and display the input image with the overlaid box on the display unit; and to, during the operation of the operation unit receiving a reduction operation, overlay the input image corresponding to a third region corresponding to the reduction ratio onto a background image corresponding to the first region, and display the background image with the overlaid input image on the display unit.

2. The display control device according to claim 1, in, The color of the background image is a color not included in the overlay component superimposed on the first region of the input image.

3. The display control device according to claim 2, in, The display control unit displays the overlay member and the second region on the display unit respectively during the operation unit receiving a zoom-in operation, and displays the overlay member and the first region on the display unit respectively during the operation unit receiving a zoom-out operation.

4. A display control device, comprising: The input component is used to input the input image; Operating components are used to receive zoom-in / zoom-out commands; as well as The display control unit is configured to, during the operation of the operation unit receiving a magnification operation, overlay a box indicating a second region corresponding to a magnification ratio onto the input image corresponding to a first region displayed by the display unit, and display the input image with the overlaid box on the display unit; and to, during the operation of the operation unit receiving a reduction operation, overlay a box indicating a third region corresponding to a reduction ratio onto the input image corresponding to the first region, and display the input image with the overlaid box on the display unit.

5. The display control device according to claim 4, in, During the operation of the operating component receiving magnification, the display control component displays the overlay member and the second area correspondingly in the first area of ​​the input image on the display component, and During the operation of the operating component, as the reduction ratio becomes smaller, the superimposed component is magnified and displayed on the display component.

6. The display control device according to claim 1 or 4, further comprising: The settings component is used to configure the display format of overlay components, and The display control component displays the overlay member on the display component in the first area of ​​the input image based on the settings of the setting component.

7. The display control device according to claim 6, in, The display control unit hides the overlay member while the operating unit is receiving a zoom-in / zoom-out operation.

8. The display control device according to claim 1 or 4, in, The overlay element superimposed on the first region of the input image is a marker.

9. The display control device according to claim 1 or 4, further comprising: An overlay component is used to overlay a frame on the input image to specify the subject included in the input image.

10. The display control device according to claim 1 or 4, further comprising: Communication components, used for communicating with camera equipment; as well as A control unit, configured to control the camera device via the communication after the zoom-in / zoom-out operation is complete, such that the zoom ratio changes according to the zoom-out / zoom-out ratio, and The input image is an image received from the camera device via the communication.

11. A display control method, comprising: Input the image; Receive zoom-in / zoom-out commands; as well as During the magnification process, a frame indicating a second region corresponding to the magnification is superimposed on the input image corresponding to the first region displayed by the display unit, and the input image with the superimposed frame is displayed on the display unit. During the process of receiving a zoom-out operation, the input image corresponding to the third region corresponding to the zoom-out ratio is superimposed on the background image corresponding to the first region, and the background image with the input image superimposed is displayed on the display unit.

12. A display control method, comprising: Input the image; Receive zoom-in / zoom-out commands; as well as During the magnification process, a frame indicating a second region corresponding to the magnification is superimposed on the input image corresponding to the first region displayed by the display unit, and the input image with the superimposed frame is displayed on the display unit. During the process of receiving a zoom-out operation, a box indicating a third region corresponding to the zoom-out ratio is superimposed on the input image corresponding to the first region, and the input image with the superimposed box is displayed on the display component.

13. A computer-readable storage medium storing a program that causes a computer to perform the display control method according to claim 11 or 12.

14. A computer program product comprising a program that causes a computer to perform the display control method according to claim 11 or 12.

Images