Information processing device, information processing method, and program

Abstract

To provide an information processing device that can identify the correspondence of image areas before and after converting coordinate.SOLUTION: The information processing device includes conversion means for converting first area information indicating a first area on a first image into second area information indicating a second area into which the first area has been converted by the coordinate transformation through which a second image is obtained by coordinate transformation to a coordinate system different from that of the first image, addition means for adding identification information indicating that each of the plurality of pieces of the second area information obtained by the transformation of one piece of the first area information corresponds to the same first area, and output means for outputting an image file containing the second image, the second area information, and the identification information.SELECTED DRAWING: Figure 12

Description

Technical Field

[0005] , , 360-degree images equipped with ,

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, with the spread of 360-degree cameras, the use of 360-degree images has been expanding. Also, 360-degree images may be coordinate-transformed into an equirectangular format or a cube map format. For example, Patent Document 1 discloses a technique of coordinate-transforming an image before transformation into an image after transformation represented in a different coordinate system, and coordinate-transforming a position within the image before transformation specified by a position specifying means into a position after transformation represented in a different coordinate system.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide an information processing apparatus capable of identifying the correspondence of image regions before and after coordinate transformation.

Means for Solving the Problems

[0006] [Figure 1] A block diagram showing the hardware configuration of an information processing device according to one embodiment. [Figure 2] A diagram showing the format of a HEIF file. [Figure 3] A diagram showing the format of RegionItemBox. [Figure 4] A diagram showing the format of PrimaryItemBox. [Figure 5] A diagram illustrating the format of an ItemInfoBox. [Figure 6] A diagram showing the format of ItemInfoEntryBox. [Figure 7] A diagram showing the format of ItemLocationBox. [Figure 8] A diagram showing the format of UserDescriptionProperty. [Figure 9] A diagram showing an example of creating a 360-degree image file. [Figure 10] A diagram illustrating an example of region information in 360-degree image data. [Figure 11] A block diagram showing the functional configuration of an information processing device. [Figure 12] A flowchart illustrating the process of outputting HEIF files. [Modes for carrying out the invention]

[0007] Embodiments of the present invention will be described in detail below with reference to the attached drawings. Note that the following embodiments are not limiting to the present invention, and not all combinations of features described in the embodiments are essential to the solutions of the present invention. The configuration of the embodiments may be modified or changed as appropriate depending on the specifications and various conditions (usage conditions, usage environment, etc.) of the system and apparatus to which the present invention is applied. The technical scope of the present invention is defined by the claims, and not by the following individual embodiments.

[0008] <Hardware configuration of the information processing device> Figure 1 shows the hardware configuration of an information processing device according to one embodiment. The information processing device 100 shown in Figure 1 is equipped with a camera function capable of 360-degree shooting and is an information processing device capable of capturing panoramic images (also called 360-degree images). The information processing device 100 includes a system bus 101, a CPU 102 (central processing unit), RAM (Random Access Memory) 103, and ROM (Read Only Memory) 104. The information processing device 100 also includes an imaging processing unit 105, an encoding processing unit 106, a display unit 107, a user interface unit 108, a communication control unit 109, and a storage media unit 110.

[0009] The system bus 101 is connected to the CPU 102, RAM 103 which is a temporary storage device for the execution of system programs and application programs, and ROM 104 which is a non-volatile storage device where these programs are stored. System programs and application programs are loaded from ROM 104 into RAM 103 and executed by the CPU 101.

[0010] Also, an imaging processing unit 105, an encoding processing unit 106, a display unit 107, a user interface unit 108, a communication control unit 109, and a storage media unit 110 are connected to the system bus 101. The system bus 101 transfers data between these blocks. The RAM 103 is also used as an output buffer for the captured image data and as a data buffer for the creation process of the captured image file and the output image file. The ROM 104 is also used as a storage destination for the user setting data of the information processing device 100.

[0011] The imaging processing unit 105 includes a lens group 111, a CCD (Charge Coupled Device) 112, a CCD control unit 113, and an image processing unit 114. The lens group 111 is composed of a plurality of lenses for optically projecting the subject image onto the CCD 112.

[0012] The CCD control unit 113 includes a timing generator for supplying a transfer clock signal and a shutter signal to the CCD 112, and also includes a circuit for noise removal and gain processing of the output signal of the CCD 112. Further, the CCD control unit 113 includes an A / D conversion circuit for converting an analog signal into a digital signal.

[0013] The image processing unit 114 performs image processing such as gamma conversion, color space conversion, white balance adjustment, and exposure correction on the digital signal output from the CCD control unit 113, and outputs it to the RAM 103 as digital image signal data that can be encoded by the encoding processing unit 106.

[0014] The encoding processing unit 106 is responsible for video coders in videos and still images such as H.265 (HEVC), H.264 (AVC), and JPEG. That is, the encoding processing unit 106 performs encoding and decoding processes on the data of still images and videos handled by the information processing device 100.

[0015] The display unit 107 has a screen for displaying applications of the information processing apparatus 100, and is, for example, a liquid crystal display device. Further, the display unit 107 may be provided with a screen touch sensor to enable a user to operate an application by means of a GUI. The user interface unit 108 is a physical operation interface having buttons, switches, etc. for operating the information processing apparatus 100.

[0016] The communication control unit 109 is a network interface that connects to a network and transmits and receives transmission frames. For example, it is responsible for PHY and MAC (media control processing) in Ethernet (registered trademark) of a wired LAN. In the present embodiment, as an example, the information processing apparatus 100 is a device that can be connected to a wireless LAN. The communication control unit 109 includes a controller that executes wireless LAN control such as IEEE802.11a / b / g / n / ac, an RF circuit, and an antenna.

[0017] The storage media unit

[110] is a non-volatile memory device having a large storage capacity such as a hard disk device, an SD card, and a CompactFlash (registered trademark) card. The storage media unit 110 is used as a storage location for storing data of still images and moving images captured by the information processing apparatus 100 as, for example, HEIF files. HEIF is an abbreviation for High Efficiency Image File.

[0018] <Structure of Image File> Next, the file structure of the image file created and stored by the information processing apparatus 100 will be described while referring to FIGS. 2 to 8. The format of the image file in the present embodiment is HEIF. FIG. 2 is a diagram showing the format of the HEIF file 300. The HEIF file 300 has a FileTypeBox ('ftyp') 301, a MetadataBox ('meta') 302, and a MediaDataBox ('mdat') 311.

[0019] The brand name is stored in 'ftyp'301. The brand name in ftyp'301 is used by the HEIF file reader to understand the HEIF file specifications. In this embodiment, as an example, the brand name 'mif1' or the compatible brand name 'heic' is stored in 'ftyp'301.

[0020] 'meta'302 is the area where metadata is stored, and 'mdat'311 is the area where encoded image data is stored. In this embodiment, image region information data is also stored in 'mdat'311. 'meta'302 contains HandlerReferenceBox('hdlr')303, PrimaryItemBox('pitm')304, and ItemLocationBox('iloc')305.

[0021] Furthermore, 'meta'302 contains ItemInfoBox('iinf')306, ItemReferenceBox('iref')307, and ItemPropertiesBox('iprp')308. 'hdlr'303 stores the handler type for understanding the structure of 'meta'302. In this embodiment, as an example, the handler type name 'pict' is stored in 'hdlr'303.

[0022] 'pitm'304 stores the ID of the representative image item among the image items stored in the HEIF file 300. As shown in Figure 4, the format of 'pitm'304 is a simple format that specifies an item ID 401 that represents any image item in the HEIF file 300.

[0023] The 'iloc'305 contains information indicating the storage location of data for each item, such as an image, within the HEIF file 300. The 'iloc'305 has the format shown in Figure 7. The 'iloc'305 also contains item_count706, which indicates the number of items.

[0024] 'iloc'305 is a parameter set that identifies the file location where the item data exists, and includes item_ID701, item data location information702, number of item data locations703, offset position704, and data length705. The parameter set is included in 'iloc'305 only for the number of items indicated by item_count706.

[0025] In the 'iloc'305 parameter set, the position of the item data corresponding to item_ID701 is represented by position information 702 as a byte offset from the beginning of the file. The 'iinf'306 contains the item ID and item type of each item in the HEIF file 300. 'iinf'306 includes entry_count501 and ItemInfoEntry502, as shown in the format in Figure 5. entry_count501 indicates the number of items, and the ItemInfoEntry502 data array contains the corresponding number of items.

[0026] ItemInfoEntry502 has the format shown in Figure 6 and includes item_ID601, item_type602, and item_name603 as parameters. item_ID601 indicates the item ID, item_type602 indicates the item type, and item_name603 indicates the item name. For example, if it is an HEVC image item, 'hvc1' will be stored in item_type602.

[0027] The 'iref'307 contains information about the relationships between items. For example, 'iref'307 is used to describe the relationship between an image item of a captured image and its corresponding information item (such as Exif data). Another example of 'iref'307 is its use to describe the relationships between image items in a derived image item, which is composed of multiple image items.

[0028] 'iprp'308 contains ItemPropertyContainerBox('ipco')309 and ItemPropertyAssociationBox('ipma')310. 'ipco'309 stores various attribute information (property) data in a list format. 'ipma'310 stores the association between attribute information and items.

[0029] 'ipco'309 stores information such as the width and height of an image item in pixels, and data for the HEVC parameter set required to decode an HEVC image item. 'ipco'309 also includes a UserDescriptionProperty('udes') that can store arbitrary descriptive information.

[0030] As shown in Figure 8, 'udes'800 includes lang801, name802, description803, and tags805. lang801 is a language tag string that indicates language information in accordance with RFC5646. The contents of name802, description803, and tags805 are written using the language specified in lang801.

[0031] name802 indicates the name of the item associated with 'udes'800. description803 provides a description of the item. tags804 lists user-defined tag information associated with the item, separated by commas. In this embodiment, 'udes'800 is used to annotate the asset area, which is a sub-region of the image and is described later, with name, description, and tag information.

[0032] Note that 'ipma'310 may contain descriptions that associate multiple items with a single attribute piece of information. For example, 'ipma'310 can associate one 'udes'800 with multiple domain information items.

[0033] The 'mdat' 311 contains the encoded data 312 and region information data 313 of the image items in the HEIF file 300. If the HEIF file 300 contains multiple image items, there will be multiple encoded data 312 entries. Similarly, if the HEIF file 300 contains multiple region information entries, there will be multiple region information data 313 entries.

[0034] As mentioned above, the information from 'iloc'305, which identifies the file location where the item's data exists, is used to associate item_ID701 with the encoded data 312 and region information data 313 within 'mdat'311. As a result, the ID and data are also associated with image items.

[0035] The region information data 313 stores region information in the format of RegionItemBox('rgan')200 shown in Figure 3. 'rgan'200 has a format that allows specifying multiple regions. 'rgan'200 has reference_width201, reference_height202, and region_count203.

[0036] Furthermore, 'rgan'200 has geometry_type204, asset_id205, and parameter sets 206~211. reference_width201 and reference_height202 indicate the width and height of the 2D coordinate space referenced by each region information specified in 'rgan'200. Typically, region information indicates a sub-region of an image stored in the HEIF file 300, so the above width and height values ​​indicate the width and height of the frame in the image item within the HEIF file 300.

[0037] region_count203 indicates the number of region information items specified by rgan'200. Then, geometry_type204, asset_id205, and parameter sets 206-211 are repeated the number of times corresponding to the number of region information items. `geometory_type204` is a parameter that selects the type of geographical shape of the region. The value of this parameter can be one of the parameters 206 to 211.

[0038] asset_id205 is the asset ID assigned to the region information of the image data. Region information with the same asset ID value is treated as a single asset region when displaying the image. Each parameter set 206-211 indicates the position and size for each type of region shape. For a single region, only one parameter set from 206-211, indicated by geometry_type204, contains substantial information.

[0039] The first parameter set 206 indicates that the value of geometry_type is "0", meaning the region shape is a point. The second parameter set, 207, indicates that the value of geometry_type is "1", meaning the region shape is rectangular. The third parameter set, 208, indicates that the value of geometry_type is "2", meaning the region shape is elliptical.

[0040] The fourth parameter set 209 represents a region represented by the coordinates of multiple points, where the points are connected by straight lines in a specific order, and the coordinates of each point are shown. If the value of geometry_type in the fourth parameter set 209 is "3", it indicates that the region shape is a polygon connecting the first and last points. If the value of geometry_type in the fourth parameter set 209 is "6", it indicates that the region shape is a polyline that does not connect the first and last points.

[0041] The fifth parameter set 210 indicates that the value of geometory_type is "4", referring to a rectangular area within another image item as the area shape. Specifically, the mask image item is referred to, and the luminance value at each pixel within the rectangular range in the referred mask image item is used as the mask value to represent the shape of the area in detail.

[0042] The sixth parameter set 211 indicates that the value of geometory_type is "5", and the area shape is represented by a mask defined within the sixth parameter set 211. Specifically, the sixth parameter set 211 includes the position coordinates, width, height, and mask value data array of the mask. For the range of the rectangular area defined by the position coordinates, width, and height, the shape of the area is represented in pixel units by the mask value indicated by the mask value data array. ‘rgan’200 can specify multiple area information and can also specify multiple areas with different shapes.

[0043] <Example of creating a HEIF file> Next, an example of creating a HEIF file containing 360-degree image data and area information within that 360-degree image data will be described using FIGS. 9 and 10. FIG. 9 is a diagram showing an example from the shooting of a 360-degree image to the creation of a HEIF file. FIG. 10 is a diagram showing an example from the specification of an asset area to the storage of area information.

[0044] FIG. 9(A) shows a schematic diagram representing the state of shooting a 360-degree image in the shooting space 900 by the shooting processing unit 105 of the information processing apparatus 100. In the example shown in FIG. 9(A), the front direction 901 of the shooting processing unit 105 at the time of shooting faces the right side of the figure. In contrast, the left side of the figure is the back direction of the shooting processing unit l05.

[0045] [[ID= twenty-one]] Figure 9(B) shows the fisheye images 903 and 904 obtained by the imaging processing unit 105. The imaging processing unit 105 obtains two fisheye images: a fisheye image 903 facing the front and a fisheye image 904 facing the back. The data for these fisheye images 903 and 904 is generated by the imaging processing unit 105 and the encoding processing unit 106.

[0046] The information processing device 100 performs a stitching (joining) process on the two fisheye images 903 and 904 to create a single equirectangular 360-degree image 905, as shown in Figure 9(C). The created 360-degree image 905 is stored as image data in a HEIF format file and saved in the storage media unit 110.

[0047] The 360-degree image 905 may be a single image in equirectangular format, or it may be six images in cubemap format. In other words, the information processing device 100 may output an image file containing image data of the 360-degree image 905 represented in equirectangular format. Alternatively, the information processing device 100 may output an image file containing image data of the 360-degree image 905 represented in cubemap format. The following explanation will use the equirectangular format 360-degree image 905 as an example.

[0048] The saved 360-degree image 905 is transcoded into image data for full-spherical display and displayed on the display unit 107. The stitching process, the creation of equirectangular image data, and the transcoding process are performed by software processing in the encoding processing unit 106 and the CPU 102.

[0049] Figure 10(A) shows the display image 1000, in which the 360-degree image 905 is projected onto the entire sphere, as a sphere. The user of the information processing device 100 can use the screen of the display unit 107 as an observation window to view the full sphere display image 1000, which extends 180 degrees vertically and 360 degrees horizontally, from a viewpoint of the center point 1001.

[0050] The dotted arrow indicates the front direction 1002, which is 0 degrees vertically and 0 degrees horizontally. The solid arrow indicates the viewpoint direction 1003, which is any direction the user is looking. A spherical area within a rectangular range 1004 centered on the viewpoint direction 1003 is displayed on the display unit 107. Figure 10(B) shows a display screen 1005 showing the sphere within the rectangular area 1004 shown in Figure 10(A).

[0051] Figure 10(C) shows a viewpoint direction 1006 different from the viewpoint direction 1003 shown in Figure 10(A), and a rectangular area 1007 in that viewpoint direction 1006. Figure 10(D) shows a display screen 1005 showing the sphere of the rectangular area 1007 shown in Figure 10(C).

[0052] In this embodiment, a user can operate the information processing device 100 to designate any portion of the displayed image 1000 as an asset area, and can also add annotations to the designated asset area. In the example shown in Figure 10, the user designates asset area 1009 on the display screen 1005 in Figure 10(B) and adds the annotation "ABC". Also in the example shown in Figure 10, the user designates asset area 1010 on the display screen 1008 in Figure 10(D) and adds the annotation "XYZ".

[0053] The user-specified asset regions 1009 and 1010 are converted into regions on the 360-degree image 905 and saved in the HEIF file 300. Figure 10(E) shows the equirectangular 360-degree image 905 containing regions 1012, 1013, and 1014 into which asset regions 1009 and 1010 have been converted.

[0054] The asset area 1009 specified by the user on the display screen 1005 in Figure 10(B) is transformed into the shape of area 1012 on the 360-degree image 905. In addition, the asset area 1010 specified by the user on the display screen 1008 in Figure 10(D) is divided and transformed into two areas 1013 and 1014 on the 360-degree image 905.

[0055] The regions 1012, 1013, and 1014, which are converted from asset regions 1009 and 1010, are saved as region information for RegionItemBox200 in the HEIF file 300 mentioned above.

[0056] The user-assigned annotation "ABC" is stored in HEIF file 300 as annotation information associated with region 1012. Similarly, the user-assigned annotation "XYZ" is stored in HEIF file 300 as annotation information associated with both regions 1013 and 1014. The annotation information is stored in 'udes'800 of 'ipco'309, which is part of the metadata stored in 'meta'302. Furthermore, asset areas 1009 and 1010 are not limited to those specified by the user; they may also be automatically generated, for example, through object recognition of the displayed image or machine learning processing.

[0057] <Functional Configuration of Information Processing Devices> Figure 11 is a block diagram showing the functional configuration of the information processing device 100. The information processing device 100 includes an imaging unit 122, an HEIF file storage unit 123, an HEIF file storage unit 124, a coordinate transformation unit 125, a display unit 126, and a region designation unit 127.

[0058] The imaging unit 122 performs imaging and obtains a 360-degree image 905 in equirectangular format. The HEIF file storage unit 123 stores the image data of the 360-degree image 905 into an HEIF file 300 and outputs it to the HEIF file storage unit 124. The HEIF file storage unit 124 stores the HEIF file 300.

[0059] The coordinate transformation unit 125 reads the HEIF file 300 from the HEIF file storage unit 124 and performs a coordinate transformation on the 360-degree image 905 to obtain a panoramic display image 1000. The display unit 126 displays the panoramic display image 1000. The area designation unit 127 designates an asset area on the panoramic display image 1000 according to the user's operation. The area designation unit 127 also adds annotation information to the asset area according to the user's operation.

[0060] The coordinate transformation unit 125 transforms the coordinates of the specified asset area to obtain the area on the 360-degree image 905. The HEIF file storage unit 123 assigns an asset ID to the area information indicating the area on the image in the 360-degree image 905 and stores it in the HEIF file 300. The HEIF file storage unit 123 also associates the annotation information attached to the asset area with the area information and stores it in the HEIF file 300. The HEIF file 300 containing the area information and annotation information is output from the HEIF file storage unit 123 to the HEIF file storage unit 124 and stored there.

[0061] The configuration shown in Figure 11 is just one example; multiple functional blocks may constitute a single functional block, or any functional block may be divided into blocks that perform multiple functions. Furthermore, at least one of the functional blocks may be implemented as hardware. When implemented in hardware, for example, a specific compiler can be used to automatically generate dedicated circuits on the FPGA from the program required to realize each step. FPGA stands for Field Programmable Gate Array. Alternatively, a Gate Array circuit may be formed in a similar manner to an FPGA and implemented as hardware. It may also be implemented using an ASIC (Application Specific Integrated Circuit).

[0062] <Processing procedure of information processing device> Next, we will describe the processing procedure in the information processing device 100 that implements the HEIF file creation example described above. Figure 12 is a flowchart showing the processing procedure in the information processing device 100. The processing procedure shown in the flowchart of Figure 12 starts from S1101, for example, when the user instructs the display of the 360-degree display image 1000.

[0063] In S1102, the image data of the equirectangular 360-degree image 905 is transcoded to generate image data for the display image 1000 that is projected onto the entire sphere. The image data of the 360-degree image 905 is, for example, image data read from a HEIF file 300 stored in the storage media unit 110.

[0064] In the case of image data of a newly created 360-degree image 905 captured by the imaging processing unit 105, in this embodiment, the HEIF file 300 is first stored in the storage media unit 110, and then read from the HEIF file 300 in S1102. However, in other embodiments, image data of the display image 1000 may be generated directly from the newly created 360-degree image 905 in S1102.

[0065] When image data for 360-degree display is created in S1102, screen display processing is executed in S1103. In the screen display processing in S1103, the 360-degree display image 1000 is displayed on the display unit 107 in a viewing direction corresponding to the user's operation on the information processing device 100. In addition, the viewing direction is changed in the screen display processing according to the user's operation on the information processing device 100.

[0066] Furthermore, in S1103, the user can specify an asset area by performing an area specification operation on the display image 1000 displayed on the display unit 107, or add annotation text to the asset area by performing a string input operation. In S1104, it is checked whether the asset area specified by the user in S1103 exists. If the asset area does not exist in S1104, the process proceeds to S1110 and terminates.

[0067] If the S1104 check confirms the existence of an asset area, the processes from S1105 to S1108 are repeatedly executed for each asset area. In S1106, the user-specified asset area is converted from area information indicating the asset area in the projected display image 1000 to area information indicating the area on the 360-degree image 905, which is the source of the transcode. The converted area information is created as the data for 'rgan'200 in the HEIF file 300.

[0068] The transformation of region information in S1106 is a process equivalent to a coordinate transformation from the displayed image 1000 to the 360-degree image 905. In other words, in S1106, a conversion is performed from the coordinate position and shape of the asset area, which is a partial region on the full-sphere display image 1000, to the position and shape of the region in the coordinate system of the 360-degree image 905 from which the transcoding was performed.

[0069] In other words, in S1106, the coordinate transformation unit 125 transforms the first region information, which indicates the asset region on the display image 1000, into second region information, which indicates the second region, which is the asset region transformed by a coordinate transformation to a coordinate system different from that of the display image 1000.

[0070] In processing in S1106, the region shape does not necessarily have to be converted to the same region shape. The region shape is selected from the shape types indicated by the values ​​"1" to "6" of geometry_type204. For a more accurate conversion of the asset region range, it is desirable to use the shape indicated by the geometry_type value "5", that is, the shape defined using a mask data array.

[0071] The asset region may be transformed into multiple regions on the 360-degree image 905. If the asset region is divided into multiple discontinuous regions on the 360-degree image 905 as a result of the coordinate transformation, the asset region will automatically be transformed into multiple regions.

[0072] If an asset area is converted into multiple areas, the 'rgan'200 data created in S1106 will contain information for multiple areas. Furthermore, the asset_id205 of 'rgan'200 will store, for example, the same asset ID for each of the multiple areas converted from the asset area. Note that the value of asset_id205 is not necessarily limited to the same value; any value indicating that it corresponds to the same asset area, such as having a common set of bits, is acceptable. In other words, in S1106, the HEIF file storage unit 123 adds identification information to each of the multiple second area information obtained by converting one of the first area information to indicate that it corresponds to the same asset area.

[0073] In S1107, data 'udes'800, which indicates the annotation information attached to the asset area, is created and associated with 'rgan'200, which was created in S1106. In other words, in S1107, the annotation information attached to one of the above first area information is associated with multiple above second area information obtained by converting the said first area information.

[0074] After the iterative processing in S1105 to S1108 is completed, the process proceeds to S1109, where the HEIF metadata, with 'udes'800 associated with the created 'rgan'200, is stored in the HEIF file 300 and saved to the storage media unit 110. In other words, in S1109, the HEIF file storage unit 123 outputs an image file containing the 360-degree image 905, which is the display image 1000 transformed by the above coordinate transformation, the second region information, and the identification information. After saving the HEIF file 300, the process proceeds to S1110 and is completed.

[0075] In another embodiment, in which the image data of the display image 1000 is generated directly from the newly created 360-degree image 905 in S1102, a new HEIF file 300 containing HEIF metadata may be created and saved in S1109. In step S1106 above, the same asset ID is assigned to the multiple areas that have been converted from the asset area, making it possible to identify that these multiple areas correspond to a single asset area in subsequent displays and other operations. In the above embodiment, an HEIF file 300 is output as an image file, but the present invention can also be applied to outputting image files in formats other than HEIF.

[0076] (Other embodiments)

[0077] The present invention can take the form of, for example, a system, apparatus, method, program, or recording medium (storage medium). Specifically, it may be applied to a system consisting of multiple devices (for example, a host computer, interface devices, a web application, etc.), or to an apparatus consisting of a single device.

[0078] Furthermore, the present invention can also be realized by supplying a program (computer program) that implements one or more of the functions of the embodiments described above to a system or device via a network or recording medium (storage medium). One or more processors in the computer of that system or device read and execute the program. In this case, the program (program code) read from the recording medium itself implements the function of the embodiment. Moreover, the recording medium on which the program is recorded can constitute the present invention.

[0079] Furthermore, the functionality of the embodiment may be realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the program's instructions, thereby realizing the functionality of the embodiment described above.

[0080] Furthermore, after the program read from the recording medium is written to the memory of a function expansion card inserted into the computer or a function expansion unit connected to the computer, the CPU or other components of the function expansion card or function expansion unit may perform some or all of the actual processing based on the instructions of the program, thereby realizing the functions of the embodiment described above. When the present invention is applied to the above-mentioned recording medium, the recording medium will store a program corresponding to the flowchart described above. [Explanation of symbols]

[0081] 100... Information processing unit, 101... System bus, 102... CPU, 103... RAM, 104...ROM, 105...Imaging processing unit, 106...Encoding processing unit, 107...Display unit, 108...User interface unit, 109...Communication control unit, 110...Storage media unit, 122...Shooting unit, 123...HEIF file storage unit, 124...HEIF file memory unit, 125... Coordinate transformation unit, 126... Display unit, 127... Area selection unit, 300 HEIF file

Claims

1. A conversion means that converts first region information, which indicates a first region on a first image that is a 360-degree image, into second region information, which indicates a second region obtained by converting the first region through a coordinate transformation to a 360-degree image having a different coordinate system from the first image, An adding means for adding identification information to each of the multiple second region pieces obtained by converting one of the first region pieces, indicating that it corresponds to the same first region, Output means for outputting an image file containing the second image obtained by transforming the first image using the coordinate transformation, the second region information, and the identification information, An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, further comprising association means for associating annotation information attached to one of the first region pieces of information with a plurality of second region pieces of information obtained by converting the first region piece of information.

3. The information processing apparatus according to claim 1 or 2, characterized in that the output means outputs an image file containing image data in which the second image is represented in equirectangular format as the image file.

4. The information processing apparatus according to claim 1 or 2, characterized in that the output means outputs an image file containing image data in which the second image is represented in cubemap format as the image file.

5. The information processing apparatus according to any one of claims 1 to 4, characterized in that the output means outputs a HEIF (High Efficiency Image File) file as the image file.

6. A transformation step of transforming first region information, which indicates a first region on a first image that is a 360-degree image, into second region information, which indicates a second region obtained by transforming the first region by a coordinate transformation to a 360-degree image having a different coordinate system from the first image, An addition step of adding identification information to each of the multiple second region pieces obtained by converting one of the first region pieces, indicating that it corresponds to the same first region; An output step of outputting an image file containing the second image obtained by transforming the first image using the coordinate transformation, the second region information, and the identification information, An information processing method characterized by having the following features.

7. The information processing method according to claim 6, further comprising an association step of associating annotation information attached to one of the first region pieces of information with a plurality of second region pieces of information obtained by converting the first region piece of information.

8. The information processing method according to claim 6 or 7, characterized in that the output step outputs an image file containing image data in which the second image is represented in equirectangular format as the image file.

9. The information processing method according to claim 6 or 7, characterized in that the output step outputs an image file containing image data in which the second image is represented in cubemap format as the image file.

10. The information processing method according to any one of claims 6 to 9, characterized in that the output step outputs a HEIF (High Efficiency Image File) file as the image file.

11. A program for causing a computer to function as one of the means of an information processing apparatus according to any one of claims 1 to 5.

Images