Electronic apparatus
The electronic device addresses the challenge of converting circumferential fisheye images to equirectangular images by using metadata to highlight conversion target areas, facilitating the production of complete and accurate equirectangular images.
Patent Information
- Application Number
- JP2023072965
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-27
- Publication Date
- 2024-11-08
AI Technical Summary
Existing methods fail to efficiently convert circumferential fisheye images to equirectangular images, particularly for VR formats, resulting in incomplete or undesired equirectangular images due to the wider angle of view of circumferential fisheye images.
An electronic device acquires image data and metadata of circumferential fisheye images, using metadata to control the display of target areas for equirectangular transformation, highlighting or emphasizing these areas to facilitate easy identification and conversion.
Enables the easy acquisition of desired equirectangular images by clearly indicating the conversion target areas within circumferential fisheye images, ensuring complete and accurate conversion.
Smart Images

Figure 2024158095000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to electronic devices, and more particularly to a technique for displaying circular fisheye images. [Background technology]
[0002] There are display devices that display circular fisheye images as they are (without converting the format).Patent Document 1 discloses a technique for generating an arranged image in which a plurality of circular fisheye images are arranged. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-123172 A Summary of the Invention [Problem to be solved by the invention]
[0004] The circular fisheye image may be converted into an equirectangular image (an image in equirectangular projection). At this time, the angle of view of the circular fisheye image may be wider than that of the equirectangular image (the area converted into the equirectangular image). In other words, a part of the area of the circular fisheye image may be converted into an equirectangular image. For example, in order to obtain an equirectangular image corresponding to a VR (Virtual Reality) format such as VR180, an area corresponding to an angle of view of 180 degrees in the circular fisheye image corresponding to an angle of view wider than 180 degrees may be converted into an equirectangular image. In such a case, a desired equirectangular image (for example, a desired subject is not captured) may not be obtained.
[0005] An object of the present invention is to make it possible to easily obtain a desired equirectangular image. [Means for solving the problem]
[0006] The electronic device of the present invention has a first acquisition means for acquiring image data of a circular fisheye image, a second acquisition means for acquiring metadata of the circular fisheye image, and a control means for controlling the display of an image based on the image data, wherein the control means controls the display of a target area of equirectangular transformation in the circular fisheye image in an identifiable manner based on the metadata. Effect of the Invention
[0007] According to the present invention, a desired equirectangular image can be easily obtained. [Brief description of the drawings]
[0008] [Figure 1] FIG. 2 is a block diagram showing a configuration of a display device. [Diagram 2] FIG. 1 is a schematic diagram showing a configuration of a fisheye lens. [Diagram 3] FIG. 2 is a schematic diagram showing an image including a circular fisheye region. [Figure 4] 13 is a flowchart of a display process. [Diagram 5] 4 is a flowchart of a display image generating process according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram showing a display image according to the first embodiment. [Figure 7] 10 is a flowchart of a display image generating process according to the second embodiment. [Figure 8] FIG. 11 is a schematic diagram showing a display image according to the second embodiment. [Figure 9] 13 is a flowchart of a display image generating process according to the third embodiment. [Figure 10] 13 is a flowchart of a partial image generation process according to the third embodiment. [Figure 11] FIG. 11 is a schematic diagram showing a display image according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Fig. 1 is a block diagram showing the configuration of a display device 100 as an example of an electronic device according to this embodiment. Note that the present invention is not limited to display devices, and can be applied to various electronic devices such as digital cameras, personal computers, smartphones, and tablet terminals.
[0010] The control unit 101 is, for example, a central processing unit (CPU), and controls the entire display device 100. The read only memory (ROM) 102 stores various data (for example, programs and parameters) that do not require modification. The random access memory (RAM) 103 temporarily stores various data (for example, programs and parameters).
[0011] The storage device 104 stores various data. For example, the storage device 104 is a solid state drive (SSD), a hard disk drive (HDD), or a flash storage that is fixed (built-in) to the display device 100. The storage device 104 may be an optical disk, a magnetic card, an optical card, an IC card, or a memory card that is removable from the display device 100.
[0012] The operation unit 105 includes an operation member such as a button or a touch panel, and accepts an operation by a user (user operation). The operation unit 105 may be a receiving unit that receives, from an operation device such as a remote controller, an operation signal corresponding to a user operation on the operation device, by wire or wirelessly. The display unit 106 displays an image based on various image data such as image data stored in advance in the display device 100, image data generated by the display device 100, and image data shared by the display device 100. The communication unit 107 is a communication interface for connecting to and communicating with an external device by wire or wirelessly. The system bus 108 is connected to each unit of the display device 100 to enable communication within the display device 100.
[0013] FIG. 2 is a schematic diagram showing the configuration of a fisheye lens 200 according to this embodiment. The fisheye lens 200 is a lens used for capturing a circular fisheye image. FIG. 2 is a diagram showing the fisheye lens 200 as viewed from above (perpendicular to the optical axis). The fisheye lens 200 has a lens unit 201 and a support unit 202. The lens unit 201 is supported by the support unit 202 (support). Angle 203 between line segments AC and CB is the angle of view of the imaging range of the fisheye lens 200 (the range captured using the fisheye lens 200), and is, for example, greater than 180 degrees. Angle 204 between line segments DC and CE is the angle of view of the target area of equirectangular conversion (the range of the imaging range to be converted into an equirectangular image (image of equirectangular projection), the conversion target area), and is, for example, 180 degrees.
[0014] 3(A) and 3(B) are schematic diagrams showing images 300 and 310 represented by image data (image data of a circular fisheye image) according to this embodiment. The image 300 in FIG. 3(A) includes one circular fisheye region 301 (region of a circular fisheye image). The circular fisheye region 301 (circular fisheye image) corresponds to, for example, the angle 203 in FIG. 2. A part of the circular fisheye region 301, a region 302, is a region to be transformed, and corresponds to, for example, the angle 204 in FIG. 2. In the circular fisheye region 301, a region outside the region to be transformed 302 (region indicated by diagonal lines) is a region that is captured but is not included in the equirectangular image. A position 303 in the circular fisheye region 301 (in the circular fisheye image) is an optical axis corresponding position that corresponds to the optical axis (optical axis center position) of the fisheye lens (for example, the fisheye lens 200) used to capture the circular fisheye region 301. An arrow 304 indicates the radius of the circular fisheye region 301. An arrow 305 indicates the radius of the region 302 to be transformed.
[0015] The image 310 in FIG. 3B includes two circular fisheye regions 311 and 316. By using multiple fisheye lenses, image data of an image including multiple circular fisheye regions can be obtained. A part 312 of the circular fisheye region 311 is a region to be transformed. A region outside the transformation target region 312 in the circular fisheye region 311 (a region indicated by diagonal lines) is a region that is captured but not included in the equirectangular image. A position 313 in the circular fisheye region 311 is an optical axis corresponding position that corresponds to the optical axis of the fisheye lens used to capture the circular fisheye region 311. An arrow 314 indicates the radius of the circular fisheye region 311. An arrow 315 indicates the radius of the transformation target region 312. A part 317 of the circular fisheye region 316 is a region to be transformed. In the circular fisheye region 316, the area outside the transformation target region 317 (the area indicated by diagonal lines) is an area that is captured but not included in the equirectangular image. A position 318 within the circular fisheye region 316 is an optical axis corresponding position that corresponds to the optical axis of the fisheye lens used to capture the circular fisheye region 316. An arrow 319 indicates the radius of the circular fisheye region 316. An arrow 320 indicates the radius of the transformation target region 317.
[0016] Fig. 4 is a flowchart of the display process performed by the display device 100. The display process in Fig. 4 is realized by the control unit 101 expanding a program stored in the ROM 102 into the RAM 103 and executing it. For example, when the display device 100 is started, the control unit 101 starts the display process in Fig. 4. In the display process in Fig. 4, an image based on image data of a circular fisheye image is displayed.
[0017] In S401, the control unit 101 acquires image data of a circular fisheye image from the storage device 104 or from an external device via the communication unit 107. The acquired image data may be any of a still image, a video, a RAW still image, and a RAW video.
[0018] In S402, the control unit 101 acquires metadata of the image data acquired in S401 (metadata of the circular fisheye image) from the storage device 104 or from an external device via the communication unit 107.
[0019] In S403, the control unit 101 determines whether or not to display the image represented by the image data acquired in S401 as is (without performing image processing such as the processing of S410 described later). For example, the control unit 101 determines whether or not a setting for not performing image processing (a setting for displaying the image represented by the acquired image data as is) is valid. The control unit 101 may determine whether or not the user has given an instruction for not performing image processing (an instruction for displaying the image represented by the acquired image data as is). If the image represented by the acquired image data is to be displayed as is, the process proceeds to S412, and if not, the process proceeds to S404. According to the processing of S403, whether or not to display the image represented by the acquired image data as is can be switched in response to an instruction from the user. This improves convenience.
[0020] In S404, the control unit 101 determines whether the metadata acquired in S402 includes lens metadata required for the process of S410. If the metadata includes lens metadata, the process proceeds to S405, and if not, the process proceeds to S406. The lens metadata includes, for example, at least one of the optical axis corresponding position of the circular fisheye region (circular fisheye image), the radius of the circular fisheye region, the field angle of the circular fisheye region, and the field angle of the conversion target region. The lens metadata may be design information (design value) determined for the type of lens, or individual information (individual value) such as a measurement result corresponding to a combination of a lens and an imaging device. Since individual information is more reliable than design information, it is preferable to acquire individual information.
[0021] In S405, the control unit 101 extracts (acquires) lens metadata from the metadata acquired in S402, and stores it in the RAM 103. If the region is included, multiple lens metadata corresponding to the multiple circular fisheye regions are extracted.
[0022] Some imaging devices can capture circular fisheye images by mounting a fisheye lens, but cannot record lens metadata because the firmware of the imaging device is old. Taking such imaging devices into consideration, lens metadata (design information) of one or more lenses may be stored in advance in a storage unit (e.g., storage device 104). Then, general metadata such as lens identification information (e.g., lens name) that identifies the fisheye lens used to capture the circular fisheye image may be obtained, and lens metadata corresponding to the used fisheye lens may be read from storage device 104 based on the obtained identification information. In S406 to S409, lens metadata is obtained in this manner.
[0023] In S406, the control unit 101 determines whether or not lens identification information is included in the metadata acquired in S402. If lens identification information is included in the metadata, the process proceeds to S407, and if not, the process proceeds to S412.
[0024] In S407, the control unit 101 extracts (acquires) the lens identification information from the metadata acquired in S402, and stores it in the RAM 103.
[0025] In S408, the control unit 101 determines whether or not the lens identification information acquired in S407 corresponds to a known fisheye lens (a fisheye lens whose lens metadata is stored in the storage device 104). If the lens identification information corresponds to a known fisheye lens, the process proceeds to S409, and if not, the process proceeds to S412.
[0026] In S409, the control unit 101 reads lens metadata of the fisheye lens corresponding to the lens identification information acquired in S407 from the storage device 104. If the image acquired in S401 contains multiple circular fisheye regions, multiple lens metadata corresponding to the multiple circular fisheye regions respectively are read.
[0027] In S410, the control unit 101 generates display image data based on the lens metadata acquired in S405 or S409 (display image generation process). Details of the display image generation process will be described later with reference to FIGS.
[0028] In S411, the control unit 101 displays an image on the display unit 106 according to the display image data generated in S410.
[0029] In S412, the control unit 101 displays an image on the display unit 106 according to the image data acquired in S401.
[0030] (Embodiment 1) Fig. 5 is a flowchart of the display image generation process (S410 in Fig. 4) according to embodiment 1. The display image generation process in Fig. 5 is realized by the control unit 101 loading a program stored in the ROM 102 into the RAM 103 and executing it.
[0031] In S501, the control unit 101 secures in the RAM 103 a display image buffer having the same size as the image data acquired in S401 of FIG.
[0032] In S502, the control unit 101 calculates the distance from the optical axis corresponding position to the edge (periphery pixel) of the conversion target region for each circular fisheye region of the image data acquired in S401, that is, the radius of the conversion target region. For example, the radius indicated by the arrow 304 in FIG. 3A (the radius of the circular fisheye region 301) 2 (angle of view of the circular fisheye region 301) is defined as θr, and angle 204 (angle of view of the region to be converted 302) is defined as θc. The radius indicated by the arrow 305 (radius of the region to be converted 302) is defined as Rc. The radius Rc can be calculated using the following Equation 1. Rc = Rr × (θc / θr) (Equation 1)
[0033] 5, the processes of S504 to S508 are performed for each pixel of the image data acquired in S401. In S503, the control unit 101 determines whether or not the processes of S504 to S508 have been performed for all pixels of the image data acquired in S401. If the processes of S504 to S508 have been performed for all pixels, the display image generation process ends, and if not, the process proceeds to S504.
[0034] In S504, the control unit 101 calculates the distance from the pixel to be processed to the optical axis corresponding position. For example, the coordinates (horizontal position, vertical position) of the pixel to be processed are (Xt, Yt), and the coordinates of the optical axis corresponding position 303 in FIG. 3A are (Xc, Yc). Then, the distance from the pixel to be processed to the optical axis corresponding position 303 is Rt. The distance Rt can be calculated using the following Equation 2. Rt = ((Xt - Xc) 2 +(Yt-Yc) 2 ) 1 / 2 ...(Formula 2)
[0035] If the image acquired in S401 contains multiple circular fisheye regions, the control unit 101 may calculate multiple distances from the pixel to be processed to multiple optical axis corresponding positions corresponding to the multiple circular fisheye regions, and adopt (select) the minimum value of these as the distance Rt. Hereinafter, the circular fisheye region corresponding to the distance Rt (the circular fisheye region to be processed) will be referred to as the corresponding fisheye region.
[0036] In S505, the control unit 101 determines whether the distance Rt calculated in S504 is equal to or less than the radius Rc calculated in S502 (the radius of the conversion target area in the corresponding fisheye area). That is, the control unit 101 determines whether the pixel to be processed is included in the conversion target area in the corresponding fisheye area. If the distance Rt is equal to or less than the radius Rc, that is, if the pixel to be processed is included in the conversion target area, the process proceeds to S506, and if not, the process proceeds to S507.
[0037] In S506, the control unit 101 stores the pixel value of the pixel to be processed (the pixel value of the image data acquired in S401) in the display image buffer as the pixel value of the pixel of the display image data corresponding to the pixel to be processed.
[0038] In S507, the control unit 101 stores the pixel value of a predetermined background color (for example, black) in the display image buffer as the pixel value of the pixel of the display image data corresponding to the pixel to be processed.
[0039] In S508, the control unit 101 updates the processing target to the next pixel. For example, the pixel in the upper left corner of the image is first selected as the processing target. Then, the process of selecting the pixel immediately to the right of the current processing target as the new processing target is repeated. If the current processing target is the pixel at the right end of the image, the pixel at the left end one row below the current processing target is selected as the processing target. Then, the process of selecting the pixel immediately to the right of the current processing target as the new processing target is repeated. If the current processing target is the pixel at the lower right corner of the image, the pixel at the upper left corner of the image is selected again as the processing target, and the display image generation process is terminated.
[0040] By carrying out the processes of S504 to S508 for all pixels of the image data acquired in S401, display image data is generated in the display image buffer.
[0041] FIG. 6 is a schematic diagram showing a display image 600 (an image represented by display image data) according to the first embodiment. An area 601 of the display image 600 is a conversion target area, which is a valid area that represents the subject. The area outside the conversion target area 601 (the area shown by hatching) is an area filled with a predetermined background color, which is an invalid area that does not represent the subject.
[0042] As described above, according to the first embodiment, the conversion target area in the circular fisheye image is displayed in an identifiable manner based on the metadata of the circular fisheye image. Specifically, the circular fisheye image is displayed with the area outside the conversion target area filled with a predetermined color. This allows the user to easily grasp the conversion target area and perform shooting, and ultimately to easily obtain a desired equirectangular image. Note that when image data of an image including areas of multiple circular fisheye images is acquired, the conversion target areas of each of the multiple circular fisheye images are displayed in an identifiable manner.
[0043] (Embodiment 2) Fig. 7 is a flowchart of the display image generation process (S410 in Fig. 4) according to embodiment 2. The display image generation process in Fig. 7 is realized by the control unit 101 loading a program stored in the ROM 102 into the RAM 103 and executing it.
[0044] 5, in S701, the control unit 101 secures in the RAM 103 a display image buffer having the same size as the image data acquired in S401 in FIG.
[0045] In S702, similarly to S502, the control unit 101 calculates the distance from the optical axis corresponding position to the edge (periphery pixel) of the conversion target region, that is, the radius Rc of the conversion target region, for each circular fisheye region of the image data acquired in S401.
[0046] 7, the processes of S704 to S708 are performed for each pixel of the image data acquired in S401. In S703, similar to S503, the control unit 101 determines whether or not the processes of S704 to S708 have been performed for all pixels of the image data acquired in S401. If the processes of S704 to S708 have been performed for all pixels, the display image generation process ends, and if not, the process proceeds to S704.
[0047] In S704, similarly to S504, the control unit 101 calculates the distance Rt from the pixel to be processed to the optical axis corresponding position.
[0048] In S705, the control unit 101 determines whether the value obtained by subtracting the radius Rc (the radius of the conversion target region in the corresponding fisheye region) calculated in S702 from the distance Rt calculated in S704 is greater than 0 and equal to or less than the threshold Th. That is, the control unit 101 determines whether the pixel to be processed is included in the contour of the conversion target region. If the value obtained by subtracting the radius Rc from the distance Rt is greater than 0 and equal to or less than the threshold Th, that is, if the pixel to be processed is included in the contour, the process proceeds to S706, and if not, the process proceeds to S707.
[0049] In S706, the control unit 101 stores the pixel value of a predetermined contour color (for example, red) in the display image buffer as the pixel value of the pixel of the display image data corresponding to the pixel to be processed.
[0050] In S707, similarly to S506, the control unit 101 stores the pixel value of the pixel to be processed (the pixel value of the image data acquired in S401) in the display image buffer as the pixel value of the pixel of the display image data corresponding to the pixel to be processed.
[0051] In S708, similarly to S508, the control unit 101 updates the processing target to the next pixel.
[0052] The processes in steps S704 to S708 are carried out for all pixels of the image data acquired in step S401. Thus, display image data is generated in the display image buffer.
[0053] 8 is a schematic diagram showing a display image 800 (an image represented by display image data) according to the second embodiment. In the display image 800, the shaded area is an area that is not captured, and is an invalid area that does not represent a subject. The shaded area is an area that is captured but is not included in the equirectangular image, and is a valid area that represents the subject. The area 801 is a conversion target area, and is a valid area that represents the subject. The shaded area is an area outside the conversion target area 801 in the circular fisheye area, and the shaded area is an area outside the circular fisheye area. In the display image 800, a contour line 802 of the conversion target area 801 is emphasized.
[0054] As described above, according to the second embodiment, the circular fisheye image is displayed with the contour of the conversion target area emphasized based on the metadata of the circular fisheye image. This allows the user to easily grasp the conversion target area and take a picture, and thus to easily obtain the desired equirectangular image.
[0055] (Embodiment 3) Fig. 9 is a flowchart of the display image generation process (S410 in Fig. 4) according to embodiment 3. The display image generation process in Fig. 9 is realized by the control unit 101 loading a program stored in the ROM 102 into the RAM 103 and executing it.
[0056] In the display image generation process of Fig. 9, the processes of S902 to S906 are performed for each circular fisheye region of the image data acquired in S401 of Fig. 4. In S901, the control unit 101 determines whether or not the processes of S902 to S906 have been performed for all circular fisheye regions of the image data acquired in S401. If the processes of S902 to S906 have been performed for all circular fisheye regions, the process proceeds to S907, and if not, the process proceeds to S902.
[0057] In S902, similarly to S502, the control unit 101 calculates the distance from the optical axis corresponding position of the circular fisheye region to be processed to the edge (periphery pixels) of the conversion target region, that is, the radius Rc of the conversion target region.
[0058] In S903, the control unit 101 calculates a rectangular area (circumscribed rectangular area) that circumscribes the conversion target area in the circular fisheye area to be processed. For example, the coordinates of the optical axis corresponding position 303 in FIG. 3A are (Xc, Yc), and the radius indicated by the arrow 305 (the radius of the conversion target area 302) is Rc. The area whose upper left corner pixel coordinates are (Xc-Rc, Yc-Rc), whose width is (Rc×2), and whose height is (Rc×2) can be calculated as the circumscribed rectangular area of the conversion target area 302.
[0059] In S904, the control unit 101 secures in the RAM 103 a buffer having an image size of (Rc×2) width and (Rc×2) height as a partial image buffer for the circular fisheye region to be processed.
[0060] In S905, the control unit 101 generates partial image data for the circular fisheye region to be processed (partial image generation process). As will be described in detail later, the image (partial image) represented by the partial image data is an image of a rectangular region (circumscribed rectangular region) that circumscribes the region to be converted in the circular fisheye region to be processed. The details of the partial image generation process will be described later with reference to FIG. 10.
[0061] In S906, the control unit 101 updates the processing target to the next circular fisheye region.
[0062] In S907, the control unit 101 stores all the generated partial image data in the RAM 103. A display image buffer for storing the partial image is secured. Assume that the width and height of the partial image are each (Rc×2). If there is one circular fisheye region, a buffer with an image size of (Rc×2) width and (Rc×2) height is secured as the display image buffer. In the third embodiment, if the image acquired in S401 contains multiple circular fisheye regions, multiple partial images corresponding to the multiple circular fisheye regions are arranged in the left-right direction (horizontal direction, width direction) without any gaps. Therefore, if the image acquired in S401 contains two circular fisheye regions, a buffer with an image size of ((Rc×2)×2) width and (Rc×2) height is secured as the display image buffer.
[0063] In S908, the control unit 101 stores all the generated partial image data in the display image buffer, whereby display image data including all the partial image data is generated in the display image buffer.
[0064] Fig. 10 is a flowchart of the partial image generation process (S905 in Fig. 9) according to embodiment 3. The partial image generation process in Fig. 10 is realized by the control unit 101 loading a program stored in the ROM 102 into the RAM 103 and executing it.
[0065] In the partial image generation process of Fig. 10, the processes of S1002 to S1006 are performed for each pixel of the circumscribing rectangular area calculated (determined) in S903 of Fig. 9. In S1001, the control unit 101 determines whether or not the processes of S1002 to S1006 have been performed for all pixels of the circumscribing rectangular area calculated (determined) in S903. If the processes of S1002 to S1006 have been performed for all pixels, the display image generation process ends, and if not, the process proceeds to S1002.
[0066] In S1002, similarly to S504, the control unit 101 calculates the distance Rt from the pixel to be processed to the optical axis corresponding position (the optical axis corresponding position corresponding to the corresponding fisheye region to be processed).
[0067] In S1003, similarly to S505, the control unit 101 determines whether the distance Rt calculated in S1002 is equal to or less than the radius Rc calculated in S902 (the radius of the conversion target area in the corresponding fisheye area to be processed). That is, the control unit 101 determines whether the pixel to be processed is included in the conversion target area in the corresponding fisheye area to be processed. If the distance Rt is equal to or less than the radius Rc, that is, if the pixel to be processed is included in the conversion target area, the process proceeds to S1004, and if not, the process proceeds to S1005.
[0068] In S1004, the control unit 101 stores the pixel value of the pixel to be processed (the pixel value of the image data acquired in S401) in the partial image buffer as the pixel value of the pixel of the partial image data corresponding to the pixel to be processed.
[0069] In S1005, the control unit 101 stores the pixel value of a predetermined background color (for example, black) in the partial image buffer as the pixel value of the pixel of the partial image data corresponding to the pixel to be processed.
[0070] In S1006, the control unit 101 updates the processing target to the next pixel. For example, first, the pixel at the upper left corner of the circumscribing rectangular area is selected as the processing target. Then, the process of selecting the pixel immediately to the right of the current processing target as the new processing target is repeated. If the current processing target is the pixel at the right end of the circumscribing rectangular area, the pixel at the left end one row below the current processing target is selected as the processing target. Then, the process of selecting the pixel immediately to the right of the current processing target as the new processing target is repeated. If the current processing target is the pixel at the lower right corner of the circumscribing rectangular area, the pixel at the upper left corner of the circumscribing rectangular area is selected again as the processing target, and the partial image generation process is terminated.
[0071] By performing the processes of S1002 to S1006 for all pixels of the circumscribing rectangular area, partial image data is generated in the partial image buffer.
[0072] FIG. 11 is a schematic diagram showing a display image 1100 (an image represented by display image data) according to the third embodiment. The display image 1100 is a display image in the case where the image acquired in S401 includes two circular fisheye regions. In the display image 1100, each of an area 1110 and an area 1120 is a partial image area (a partial image area, a circumscribing rectangular area of a conversion target area). In the partial image area 1110, an area 1111 is a conversion target area, and is a valid area representing a subject. In the partial image area 1110, an area (a shaded area) outside the conversion target area 1111 is an area filled with a predetermined background color, and is an invalid area not representing a subject. Similarly, in the partial image area 1120, an area 1121 is a conversion target area, and is a valid area representing a subject. In the partial image area 1120, the area outside the conversion target area 1121 (the area shown by hatching) is an area filled with a predetermined background color, and is an invalid area that does not represent the subject. The partial image area 1110 and the partial image area 1120 are side by side with no gaps between them.
[0073] The arrangement direction of the partial image regions in the display image is not limited to the left-right direction, and may be, for example, the up-down direction (vertical direction, height direction). The arrangement of the partial image regions in the display image may be the same as the arrangement of the circular fisheye regions in the acquired image. There may be gaps between the partial image regions. The number of circular fisheye regions is not particularly limited, and the number of partial image regions is also not particularly limited. If the acquired image contains only one circular fisheye region, an image such as only partial image region 1110 or partial image region 1120 is obtained as the display image.
[0074] As described above, according to the third embodiment, the area outside the conversion target area is filled with a predetermined color, as in the first embodiment. This allows the user to easily grasp the conversion target area and take a picture, and thus to easily obtain a desired equirectangular image. Note that the method for making the conversion target area identifiable is not particularly limited, and the contour line of the conversion target area may be emphasized, as in the second embodiment.
[0075] Furthermore, according to the third embodiment, since the circumscribing rectangular area of the conversion target area is extracted and displayed, the ratio of the conversion target area to the displayed image can be increased. As a result, fine adjustments such as placing a desired subject at a desired position within the conversion target area can be easily performed, and a desired equirectangular image can be obtained more easily.
[0076] The above-described embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above-described embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.
[0077] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0078] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) A first acquisition means for acquiring image data of a circular fisheye image; A second acquisition means for acquiring metadata of the circular fisheye image; A control means for controlling to display an image based on the image data; having The control means controls to identifiably display a target area of the equirectangular transformation in the circular fisheye image based on the metadata. 1. An electronic device comprising: (Configuration 2) The control means controls the display of the circular fisheye image in a state where an area outside the target area is filled with a predetermined color. 2. The electronic device according to configuration 1. (Configuration 3) The control means controls the display of the circular fisheye image in a state where the contour line of the target area is emphasized. 2. The electronic device according to configuration 1. (Configuration 4) The metadata includes at least one of a position in the circular fisheye image corresponding to an optical axis of a lens used to capture the circular fisheye image, a radius of the circular fisheye image, a width of an angle of view of the circular fisheye image, and a width of an angle of view of the target area. 4. The electronic device according to any one of configurations 1 to 3. (Configuration 5) The metadata is design information. 5. The electronic device according to configuration 4. (Configuration 6) The metadata is individual information 5. The electronic device according to configuration 4. (Configuration 7) design information including at least one of a position in the circumferential fisheye image corresponding to an optical axis of the lens, a radius of the circumferential fisheye image, a width of an angle of view of the circumferential fisheye image, and a width of an angle of view of the target area is stored in advance in a storage unit for one or more lenses; the metadata is identification information of a lens used to capture the circular fisheye image acquired by the first acquisition means, The control means read, from the storage unit, design information corresponding to the lens used to capture the circular fisheye image acquired by the first acquisition means, from among the one or more pieces of design information stored in the storage unit based on the identification information; Based on the read design information, the target area is controlled to be displayed in an identifiable manner. 4. The electronic device according to any one of configurations 1 to 3. (Configuration 8) The control means controls the circumscribing rectangular area of the target area so as to display the target area in a state in which the target area can be identified. 8. The electronic device according to any one of configurations 1 to 7. (Configuration 9) The control means controls the display of circumscribing rectangular areas of a plurality of target areas corresponding to a plurality of circular fisheye images in such a manner that the plurality of target areas are identifiable and the plurality of circumscribing rectangular areas corresponding to the plurality of target areas are arranged without gaps. 9. The electronic device according to configuration 8. (Configuration 10) The control means switches between whether or not to identifiably display the target area in response to an instruction from a user. 10. The electronic device according to any one of configurations 1 to 9. (Configuration 11) The first acquisition means acquires image data of an image including a region of a plurality of circular fisheye images, The control means controls the display so that the target area of each of the plurality of circular fisheye images is identifiable. 11. The electronic device according to any one of configurations 1 to 10. (method) A first acquisition step of acquiring image data of a circular fisheye image; a second acquisition step of acquiring metadata of the circular fisheye image; a control step of controlling to display an image based on the image data; having In the control step, a control is performed so that a target area of the equirectangular transformation in the circular fisheye image is identifiably displayed based on the metadata. 23. A method for controlling an electronic device comprising: (program) 12. A program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 11. (medium) 12. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 11. [Explanation of symbols]
[0079] 100: Display device 101: Control unit
Claims
1. A first acquisition means for acquiring image data of a circular fisheye image; A second acquisition means for acquiring metadata of the circular fisheye image; A control means for controlling to display an image based on the image data; having The control means controls to identifiably display a target area of the equirectangular transformation in the circular fisheye image based on the metadata.
1. An electronic device comprising:
2. The control means controls the display of the circular fisheye image in a state where an area outside the target area is filled with a predetermined color.
2. The electronic device according to claim 1 .
3. The control means controls the display of the circular fisheye image in a state where the contour line of the target area is emphasized.
2. The electronic device according to claim 1 .
4. The metadata includes at least one of a position in the circular fisheye image corresponding to an optical axis of a lens used to capture the circular fisheye image, a radius of the circular fisheye image, a width of an angle of view of the circular fisheye image, and a width of an angle of view of the target area.
2. The electronic device according to claim 1 .
5. The metadata is design information.
5. The electronic device according to claim 4.
6. The metadata is individual information 5. The electronic device according to claim 4.
7. design information including at least one of a position in the circumferential fisheye image corresponding to an optical axis of the lens, a radius of the circumferential fisheye image, a width of an angle of view of the circumferential fisheye image, and a width of an angle of view of the target area is stored in advance in a storage unit for one or more lenses; the metadata is identification information of a lens used to capture the circular fisheye image acquired by the first acquisition unit, The control means read, from the storage unit, design information corresponding to the lens used to capture the circumferential fisheye image acquired by the first acquisition means, from among the one or more pieces of design information stored in the storage unit based on the identification information; Based on the read design information, the target area is controlled to be displayed in an identifiable manner.
2. The electronic device according to claim 1 .
8. The control means controls the circumscribing rectangular area of the target area so as to display the target area in a state in which the target area can be identified.
2. The electronic device according to claim 1 .
9. The control means controls the display of circumscribing rectangular areas of a plurality of target areas corresponding to a plurality of circular fisheye images in such a manner that the plurality of target areas are identifiable and the plurality of circumscribing rectangular areas corresponding to the plurality of target areas are arranged without gaps.
9. The electronic device according to claim 8.
10. The control means switches between whether or not to identifiably display the target area in response to an instruction from a user.
2. The electronic device according to claim 1 .
11. The first acquisition means acquires image data of an image including a region of a plurality of circular fisheye images, The control means controls the display so that the target area of each of the plurality of circular fisheye images is identifiable.
2. The electronic device according to claim 1 .
12. A first acquisition step of acquiring image data of a circular fisheye image; a second acquisition step of acquiring metadata of the circular fisheye image; a control step of controlling to display an image based on the image data; having In the control step, a control is performed so that a target area of the equirectangular transformation in the circular fisheye image is identifiably displayed based on the metadata.
23. A method for controlling an electronic device comprising:
13. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 11.
14. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 11.
Citation Information
Patent Citations
Imaging system, developing system, imaging method, and program
JP2020123172A
