Information processing system, information processing method, and program

The system improves editing freedom in volumetric image content by detecting unconstrained frames and editing object movements in these frames to create seamless transitions, addressing the limitations of existing technologies in loop playback.

JP2026181949APending Publication Date: 2026-11-19SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2025077790
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-08
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing technologies for generating and editing volumetric images with loop playback have limited editing freedom and result in noticeable visual incongruities at scene transitions due to constrained movement in predetermined axis directions.

Method used

An information processing system and method that detects unconstrained frames in volumetric data and edits the movement of objects in these frames to create different movements, allowing for arbitrary changes in position, rotation, and other attributes at times when the changes are difficult to perceive visually.

Benefits of technology

Enhances the degree of freedom in editing volumetric image content by enabling seamless transitions and arbitrary editing of object movements, reducing visual incongruities during loop playback.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026181949000001_ABST
    Figure 2026181949000001_ABST
Patent Text Reader

Abstract

This improves the editing flexibility of content by looping scenes composed of volumetric images. [Solution] A scene that is looped and played a predetermined number of times is composed of multiple frames generated from volumetric data. Frames in which the objects represented by the volumetric data are not constrained to move in a predetermined axis direction are detected as unconstrained frames. The frames are edited so that the movement represented by the objects in at least the unconstrained frames is a different movement from the aforementioned movement. This can be applied to content using volumetric images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program, and more particularly, to an information processing apparatus, an information processing method, and a program that are configured to improve the degree of freedom in editing content constituted by loop-playing a scene made of volumetric images. Background Art

[0002] Techniques for generating volumetric (free-viewpoint, virtual viewpoint) images by combining images captured using a large number of cameras have been proposed.

[0003] Volumetric images sometimes involve a heavy processing load related to generation, and there are certain limitations to generating content consisting of long scenes longer than a predetermined duration, so a large amount of content that repeats loop playback of short scenes has been generated.

[0004] When a short-duration scene is played back in a loop, if there is a large difference between the leading portion and the trailing portion of the scene that form the joint, the visual change is significant, and a sense of incongruity will be given during viewing every time the joint scene is repeated.

[0005] Accordingly, first, two similar scenes with different timings are searched for from content consisting of scenes of a predetermined duration, and a short-duration scene composed of the leading portion and the trailing portion of the searched scenes is cut out as a scene for one loop. Then, a technique has been proposed that suppresses the sense of incongruity at the joint by generating content in which the cut-out scene for one loop is played back in a loop (see Non-Patent Document 1). Prior Art Documents Patent Documents

[0006] Non-Patent Document 1 Motion Graphs for Unstructured Textured Meshes (https: / / hhoppe.com / proj / motiongraph / ) [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] However, when editing content generated using the technology described in Non-Patent Document 1, editing is required on a scene-by-scene basis, meaning that the only edits actually possible were those that changed the number of loops.

[0008] This disclosure is made in light of these circumstances and, in particular, aims to improve the degree of freedom in editing content that consists of loop playback of scenes composed of volumetric images (virtual viewpoint images, free viewpoint images). [Means for solving the problem]

[0009] An information processing device and program, as one aspect of the present disclosure, is an information processing system and program that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, comprising: a frame detection unit that detects frames in which the movement of an object represented by the volumetric data is not constrained in a predetermined axis direction as unconstrained frames; and an editing unit that edits the movement represented by the object in at least the unconstrained frames of the multiple frames so that it becomes a different movement from the aforementioned movement.

[0010] One aspect of the present disclosure is an information processing method for an information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, the information processing method comprising: frame detection processing to detect frames in which the movement of an object represented by the volumetric data is not constrained in a predetermined axis direction as unconstrained frames; and editing processing to edit the movement represented by the object in at least the unconstrained frames among the multiple frames so that it becomes a different movement from the aforementioned movement.

[0011] In one aspect of this disclosure, a scene to be looped and played a predetermined number of times is composed of multiple frames generated from volumetric data, and frames in which the object represented by the volumetric data is not constrained to move in a predetermined axis direction are detected as unconstrained frames, and the multiple frames are edited such that the movement represented by the object in at least the unconstrained frames becomes a different movement from the aforementioned movement. [Brief explanation of the drawing]

[0012] [Figure 1] This diagram illustrates content that uses volumetric images to loop short scenes. [Figure 2] This diagram illustrates content editing using volumetric images. [Figure 3] This is a diagram illustrating the overview of this disclosure. [Figure 4] This figure illustrates an example configuration of an information processing device to which the technology disclosed herein is applied. [Figure 5] Figure 4 is a diagram illustrating an example configuration of the first embodiment of the volumetric data editing unit. [Figure 6] This diagram illustrates an example of editing the horizontal position of an object in volumetric data. [Figure 7] This diagram illustrates an example of editing the horizontal position of an object in volumetric data. [Figure 8] This is a diagram illustrating an example of editing the horizontal position of an object in volumetric data. [Figure 9] This is a diagram illustrating an example of editing the vertical position of an object in volumetric data. [Figure 10] This is a diagram illustrating an example of editing the rotation of an object in volumetric data. [Figure 11] This is a flowchart illustrating volumetric data editing processing by the volumetric data editing unit in Figure 5. [Figure 12] This is a diagram illustrating editing of volumetric data using a constraint axis. [Figure 13] This is a diagram illustrating a configuration example of the second embodiment of the volumetric data editing unit in Figure 12. [Figure 14] This is a flowchart illustrating volumetric data editing processing by the volumetric data editing unit in Figure 5. [Figure 15] This shows a configuration example of a computer. [Description of Embodiments]

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in the present specification and the drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals to omit redundant description.

[0014] Hereinafter, modes for implementing the technology of the present disclosure will be described. The description will be given in the following order. 1. Overview of the Present Disclosure 2. First Embodiment 3. Second Embodiment 4. Description of Computer to Which the Present Technology is Applied

[0015] <<1. Overview of the Present Disclosure>> <Generation of Virtual Viewpoint Image> This disclosure, in particular, improves the degree of freedom in editing content that consists of loop playback of scenes composed of volumetric images (virtual viewpoint images, free viewpoint images).

[0016] To explain the details of this disclosure, we will first provide an overview of it.

[0017] For content using volumetric images (virtual viewpoint images, free viewpoint images), a lot of content is being generated that loops short scenes shorter than a predetermined duration.

[0018] Content that repeatedly plays short scenes is, for example, like the one shown in Figure 1.

[0019] Figure 1 shows an example of content structure in which a short scene, in which a volumetric image of a human-shaped object jumps from right to left in the figure, is played on a loop.

[0020] More specifically, Figure 1 shows humanoid objects Sn1-1 to Sn8-1, located from the right side of the figure to near the center, jumping while changing their position and posture chronologically frame by frame. The first loop portion of a short scene is shown as the first loop, L1.

[0021] Furthermore, moving from the center to the left in the diagram, the second loop, L2, shows objects Sn1-2 to Sn8-2 jumping while changing their position and orientation chronologically frame by frame. Note that the symbols attached to the objects, starting with "-", represent the loop number, so objects Sn1-1 to Sn8-1 and objects Sn1-2 to Sn8-2 are the same. In the following, unless there is a need to distinguish between the two, they will simply be referred to as objects Sn1 to Sn8, and other configurations will be referred to similarly.

[0022] The content for these two loops, represented by the first loop L1 and the second loop L2, is generated by extracting the content between two scenes: one with identical (or similar) objects Sn1-1 and Sn2-1 at different timings, and the other with objects Sn9-1 and Sn10-1, from a predetermined length of content.

[0023] The scenes of objects Sn1-1 and Sn2-1, which form the beginning of the extracted sequence, and the scenes of objects Sn9-1 and Sn10-1, which form the end of the sequence, are either identical or similar. Of these, the scenes of objects Sn9-1 and Sn10-1 are extracted, and the scene consisting of objects Sn1 to Sn8 is set as one loop of the sequence.

[0024] With this configuration, at the transition LP, after object Sn8-1, instead of the scenes of the originally existing objects Sn9-1 and Sn10-1, the beginning of the next loop consisting of similar objects Sn1-2 and Sn2-2 is connected. As a result, the visual change at the transition is less noticeable, and the sense of incongruity is suppressed.

[0025] <Editing content using volumetric images> Here, as shown in Figure 2, consider content where, for example, one scene consists of a predetermined duration during which a human-shaped object jumps 1.5 meters, and this scene is repeated for three loops.

[0026] In this case, objects St1-1 to St8-1 are set on a frame-by-frame basis. As a result, when content playback begins, objects St1-1 to St8-1 are presented sequentially frame by frame, making it appear as if the objects are continuously changing their position and orientation as they move. Consequently, in Figure 2, the overall impression is that a human-shaped object jumps from the starting point P0 to the left in the figure and lands at landing point P1, which is 1.5m away.

[0027] Similarly, in the second loop, objects St1-2 to St8-2 are presented in order frame by frame, so that the overall impression is that the human-shaped object jumps from the previous landing point P1 to the left in the diagram and lands at landing point P2, which is 1.5m further away and 3m from the starting point P0.

[0028] Furthermore, in the third loop, objects St1-3 to St8-3 are presented sequentially frame by frame, so that the overall impression is that the human-shaped object jumps from the previous landing point P2 to the left in the diagram and lands at landing point P3, which is 1.5m further away and 4.5m from the starting point P0.

[0029] When editing content as shown in Figure 2, you can adjust the distance objects jump by adjusting the number of loops for the scene that makes up the smallest size of the content.

[0030] In other words, when a single loop playback occurs, content is played that depicts an object jumping from the starting point P0 to the landing point P1, which is 1.5m away.

[0031] When the loop playback is performed twice, the content played will depict the object jumping from its previous landing point P1 to landing point P2, which is 1.5m further away and 3m from the starting point P0.

[0032] When the loop playback is performed three times, content is played that depicts the object jumping to landing point P3, which is 1.5m away from the previous landing point P2 and 4.5m away from the starting point P0.

[0033] However, as shown in Figure 2, the relationship between the position where an object can be placed at the starting point P0 and the position of the landing point is always 1.5m apart. Therefore, for example, it is not possible to edit the object so that the landing point is 4m from the starting point P0, as shown by the dotted line in the figure.

[0034] Therefore, in this disclosure, as shown in Figure 3, the distance an object jumps is edited in a way that does not become noticeable to the user viewing the content, thereby enabling editing that allows the object to land at any desired distance.

[0035] In other words, the upper part of Figure 3 is the original content, similar to the content explained in Figure 2, and in the first loop, objects St1-1 to St6-1 represent landing at point P1, which is 1.5m away from the starting point P0. Then, in the second loop, objects St1-2 to St6-2 represent landing at point P2, which is another 1.5m away from the previous landing point P1 and 3m away from the starting point P0.

[0036] On the other hand, the lower part of Figure 3 shows the changes in an object using content generated by editing the content in the upper part of Figure 3 as a result of this disclosure. Specifically, in the first loop, objects St1'-1 to St6'-1 represent landing at landing point P1', which is 2m away from the starting point P0, and in the second loop, objects St1'-2 to St6'-2 represent landing at landing point P2', which is 2m further away from the previous landing point P1' and 4m away from the starting point P0.

[0037] Furthermore, of the objects St1'-1 to St6'-1 and St1'-2 to St6'-2, all except objects St5'-1 and St5'-1 are identical to St1-1 to St6-1 and St1-2 to St6-2 in the upper part of Figure 3.

[0038] In other words, in the lower part of Figure 3, the distance traveled by objects St5'-1 and St5'-2, which are in a state where they are off the floor due to jumping, has been edited to be longer than the distance traveled by the original objects St5'-1 and St5'-2, so that the distance traveled by the objects in one loop is 2m.

[0039] For example, if an object is in contact with the floor, changing the object's movement distance relative to the original will result in an unnatural appearance, as the object appears to slide across the floor even though its positional relationship to the floor remains unchanged.

[0040] In contrast, when an object and the floor are separated, even if the object's movement distance is changed relative to the original, the object and the floor are not in contact, so the change in their relative positions is less likely to be perceived as an unnatural movement visually.

[0041] Therefore, by editing the movement distance to change at a time that is difficult to perceive visually, that is, when the object leaves the floor surface, it becomes possible to arbitrarily change and edit the movement distance of the object.

[0042] As a result, in the content shown in the lower part of Figure 3, during the first loop, content is played that depicts the object jumping from the starting point P0 to a landing point P1', which is 2m away. Next, during the second loop, content is played that depicts the object jumping from the previous landing point P1' to a landing point P2', which is 2m away. In other words, it becomes possible to edit the content so that, during the second loop, the object appears to land at a landing point P2', which is 4m away from the starting point P0.

[0043] Similarly, by editing the object's movement distance at a time when it is difficult to visually perceive, it becomes possible to edit it so that the landing point is at any desired distance.

[0044] Furthermore, for timings that are difficult to perceive visually, in addition to editing the horizontal movement amount when the object leaves the floor, you could also edit the vertical movement amount, for example, or edit the rotation amount when the object is rotating.

[0045] As a result, according to this disclosure, by editing the amount of change (movement and rotation) of an object at a time when it is difficult to perceive visually, it becomes possible to edit it to any position or rotation angle, thereby improving the degree of freedom of editing.

[0046] <<2. First Embodiment>> <Example of hardware configuration for information processing equipment> Next, with reference to Figure 4, an example of the hardware configuration of an information processing device for editing content that loops the volumetric images (virtual viewpoint images, free viewpoint images) of this disclosure will be described.

[0047] The information processing device 101 shown in Figure 4 consists of a processing circuit 111, an input unit 112, an output unit 113, a storage unit 114, a communication unit 115, a drive 116, and a removable media 117, which are connected to each other via a bus 118, enabling the transmission and reception of data and programs.

[0048] The processing circuit 111 consists of a processor and memory, and controls the overall operation of the information processing device 101. The processing circuit 111 also includes a volumetric data editing unit 131.

[0049] Further details regarding the Volumetric Data Editing Department 131 will be described later, referring to Figure 5.

[0050] The input unit 112 consists of input devices such as a keyboard, mouse, and touch panel for inputting various types of information, and supplies the input information and corresponding signals to the processing circuit 111.

[0051] The output unit 113 is controlled by the processing circuit 111 and includes, for example, a display unit (not shown) which displays various processing results and input contents, and an audio output unit (not shown) which includes a speaker which generates music and various warning sounds.

[0052] The storage unit 114 consists of an HDD (Hard Disk Drive), an SSD (Solid State Drive), or semiconductor memory, and is controlled by the processing circuit 111 to write or read various data and programs. The storage unit 114 also stores volumetric data 151 consisting of loop-playable content of a predetermined length that is edited by the volumetric data editing unit 131 of the processing circuit 111.

[0053] The communication unit 115 is controlled by the processing circuit 111 and enables communication via wired or wireless means, such as LAN (Local Area Network) or Bluetooth (registered trademark), to send and receive various data and programs with other information processing devices (not shown) via a network (not shown) as needed.

[0054] The drive 116 reads and writes data to removable media 117 such as magnetic disks (including flexible disks), optical disks (including CD-ROMs (Compact Disc-Read Only Memory) and DVDs (Digital Versatile Discs)), magneto-optical disks (including MDs (Mini Discs)), or semiconductor memory.

[0055] <Example configuration of the first embodiment of the volumetric data editing unit> Next, referring to Figure 5, we will describe an example configuration of the volumetric data editing unit 131, which is a function realized by the information processing device in Figure 4.

[0056] The volumetric data editing unit 31 edits objects to any position or rotation angle at times when they are difficult for the user to visually perceive, such as when the object leaves the floor or when the object is rotating.

[0057] More specifically, the volumetric data editing unit 131 includes a position editing unit 201, a rotation editing unit 202, and an editing result output unit 203.

[0058] The position editing unit 201 reads the original volumetric data 151 and edits the position where the original object moves so that it becomes an arbitrary target position (any other position different from the original position) that is entered by the user through the operation of the input unit 112.

[0059] More specifically, the position editing unit 201 includes a non-contact frame detection unit 211, a position movement amount calculation unit 212, and a position movement editing processing unit 213.

[0060] The non-contact frame detection unit 211 reads the original volumetric data 151, detects frames in which the original object is in a non-contact state with an inanimate object such as the floor surface, and supplies the information of the detected non-contact frames to the positional movement amount calculation unit 212.

[0061] The position movement amount calculation unit 212 calculates the amount of movement required to move an object to a target position, and further calculates the position movement amount per frame according to the number of frames in which the object is in a non-contact state, and supplies these to the position movement editing processing unit 213.

[0062] The position movement editing processing unit 213 edits the position of an object by shifting it by the amount of position movement per frame, based on the amount of position movement per frame supplied by the position movement amount calculation unit 212, so that the object is in a non-contact state with an inanimate object such as the floor surface for each frame in which the object is in non-contact state, and outputs the editing result to the editing result output unit 203.

[0063] The rotation editing unit 202 reads the original volumetric data 151 and edits the rotation angle of the original object so that it becomes an arbitrary target rotation angle (any other rotation angle different from the original rotation angle) input by the user through the operation of the input unit 112.

[0064] For details on the specific processing performed by Location Editing Unit 201, please refer to Figures 6 to 8, which will be explained later.

[0065] Furthermore, the rotation editing unit 202 includes a rotation frame detection unit 231, a rotation amount calculation unit 232, and a rotation editing processing unit 233.

[0066] The rotation frame detection unit 231 reads the original volumetric data 151, detects a frame in which the original object is in a rotational state around a predetermined axis, and supplies the information of the detected rotation frame to the rotation amount calculation unit 232.

[0067] The rotation amount calculation unit 232 calculates a rotation amount to adjust the rotation angle relative to the rotation of the original object so that it reaches the target rotation angle, and further calculates a rotation amount per frame according to the number of frames in which the object is in a rotating state, and supplies it to the rotation editing processing unit 233.

[0068] The rotation editing processing unit 233, based on the rotation amount per frame supplied by the rotation amount calculation unit 232, edits the rotation of the original object by the rotation amount per frame for each frame in which the object is in a rotated state, and outputs the editing result to the editing result output unit 203.

[0069] For details on the specific processing performed by the Rotation Editorial Department 202, please refer to Figure 9 and see the following explanation.

[0070] The editing result output unit 203 outputs the editing results of the position editing unit 201 and the rotation editing unit 202 and stores them in the storage unit 114.

[0071] <Specific procedures of the location editing department> For example, consider the case where the original volumetric data 151 is content as shown in the upper left of Figure 6.

[0072] In other words, the content in Figure 6 has the same configuration as in Figure 3, and in the first loop scene, objects St1 to St6, which are set for each frame, are shown jumping from the starting point P0 and landing at landing point P1, which is 1.5m away. In the second loop, objects St11 to St16 are shown jumping again from the previous landing point P1 and landing at landing point P2, which is 1.5m away and 3m away from the starting point P0.

[0073] The change in the position of the original content object in the upper left of Figure 6 can be represented graphically as shown in the upper right of Figure 6. In the upper right of Figure 6, the vertical axis represents the distance from the starting point P0, and the horizontal axis represents the frame (and the object within it).

[0074] In other words, in the frames for objects St1 to St4, the position is 0 because it is at the starting point P0 and in contact with the floor. In the frame for object St5 (the frame between objects St4 and St6), the position is no longer in contact with the floor, and the change in position from the starting point P0 to the landing point P1, which is 1.5m away, is represented.

[0075] Furthermore, in the frames from object St6 to St14, the position is at the previous landing point P1, and since it is in contact with the floor, it is 1.5m away. In the frame of object St15 (the frame between objects St14 and St16), it is no longer in contact with the floor, and the position is 1.5m further from the previous landing point P1, that is, the change in position from the starting point P0 to the landing point P2, which is 3m away, is represented.

[0076] In this case, the non-contact frame detection unit 211 detects information about the frame, which includes objects St5 and St15 that are in a non-contact state with the floor surface (which is an inanimate object), and supplies it to the position movement amount calculation unit 212.

[0077] The position movement amount calculation unit 212 calculates a frame-level position movement amount based on the frame information including objects St5 and St15, corresponding to an arbitrary position entered by the user, and supplies it to the position movement editing processing unit 213.

[0078] For example, if the target is located 4m from the starting point P0, then, as shown in the upper left of Figure 6, it is longer than 3m (2 loops of the original content) and shorter than 4.5m (3 loops of the original content).

[0079] Here, when adjusting the positional movement from the target starting point P0 to 4m over two loops, the positional movement amount calculation unit 212 calculates the difference between the target 4m and the movement distance of 3m, which is the movement distance for two loops according to the original content. Furthermore, it divides the difference by the number of loops (=2) to calculate the positional movement amount for one loop as 0.5 (=1 / 2)m. Then, the positional movement amount calculation unit 212 calculates the positional movement amount per frame of 0.5 (=0.5 / 1)m by dividing the positional movement amount for one loop by the number of frames in the non-contact state.

[0080] Alternatively, the movement from the target starting point P0 to 4m can be adjusted over 3 loops. The difference between the target 4m and the original content's movement distance of 4.5m over 3 loops is calculated, and this difference is then divided by the number of loops (=3) to calculate the position movement amount per loop, such that the movement distance decreases by 0.33 (=1 / 3)m per loop. This results in a movement amount of 0.33 (=0.33 / 1)m per frame. However, the following explanation will focus on an example where adjustment is made over 2 loops.

[0081] The position movement editing processing unit 213 edits each frame of objects St1~St6 and St11~St16 in the original content to create objects St101~St106 and St111~St116 that represent movement from the target starting point P0 to a position of 4m, as shown in the lower left of Figure 6.

[0082] More specifically, the position movement editing processing unit 213 edits the frames of objects St5 and St15 in each loop so that the amount of position movement changes by 0.5m per frame.

[0083] In other words, since objects St5 and St15 are in a non-contact state with inanimate objects such as the floor, even if the distance traveled is changed relative to the original distance, it is difficult to visually perceive that the distance traveled has been changed due to the contrast between objects St5 and St15 and the floor.

[0084] Therefore, the position movement editing processing unit 213 edits the original objects St1 to St6 of the first loop to objects St101 to St106, as shown in the lower right of Figure 6, and edits the movement distance of object St105 so that the total movement distance becomes 2m, which is the original movement distance of 1.5m plus a position movement amount of 0.5m.

[0085] Similarly, the position movement editing processing unit 213 edits the original second loop objects St11 to St16 as objects St111 to St116, as shown in the lower right of Figure 6, and edits the movement distance of object St115 to extend so that the total movement distance is 2m, which is the original movement distance of 1.5m plus a position movement amount of 0.5m.

[0086] Furthermore, in the lower right of Figure 6, the movement distance of objects St105 and St115 is increased by 0.5m, which represents the amount of movement. This shows that the content, which consists of frames made up of objects St101-St106 and objects St111-St116, has a total movement distance of 2m.

[0087] Through this editing process, the content is edited as shown in the lower left of Figure 6. Specifically, the content shown in the lower left of Figure 6 represents the landing at landing point P1', 2m away from the starting point P0, using objects St101 to St106 in the first loop, and the landing at landing point P2', 2m further away from the previous landing point P1' and 4m away from the starting point P0, using objects St111 to St116 in the second loop.

[0088] In this way, the original movement distance is edited to change by the amount of positional movement at a time that is difficult to perceive visually, namely when the object leaves the floor surface. This makes it possible to arbitrarily change and edit the movement distance of the object. As a result, the movement of the object, which is represented by looping playback of volumetric images (virtual viewpoint images, free viewpoint images), can be edited to represent any other movement desired by the user, different from the original movement.

[0089] As a result, by editing the amount of change in an object at a time when it is difficult to perceive visually, it becomes possible to improve the degree of freedom in editing content that loops volumetric images (virtual viewpoint images, free viewpoint images).

[0090] <First example of the specific processing performed by the location editing department> In the above explanation, we have used content as an example where the content repeats only one jump scene per loop, but the content may also include walking scenes in addition to jumping scenes during a single loop.

[0091] For example, consider a case where the original content, as shown in the upper left of Figure 7, consists of a scene in the first half where the object walks while in contact with the floor at a predetermined speed, a scene where the object jumps, and finally a scene where the object walks again while in contact with the floor at a predetermined speed. Here, we assume that the distance traveled by the object's walking + jumping + walking in one loop is distance d. Furthermore, as shown in the upper left of Figure 7, we will proceed with the explanation assuming that a frame consisting of two objects is assigned to the walking scene in the first half, a frame consisting of four objects is assigned to the jumping scene, and two frames consisting of two objects are assigned to the final walking scene, for a total of seven frames making up one loop.

[0092] Furthermore, Figure 7 shows that time t0~t1 is the first loop (Loop 1), time t1~t2 is the second loop, time t2~t3 is the third loop, and the distance traveled per loop is distance d.

[0093] For example, as shown in Figure 7, when the target position is at a distance Pt from the starting point, and the distance Pr (=3 × d) for three loops of the distance d for one loop is not a multiple of the distance Pt, then simply looping the original content three times will result in a discrepancy between the target position Pr and position Pt.

[0094] Therefore, as described above, the position movement amount calculation unit 212 calculates the difference (=Pr-Pt) between the distance Pt to the target position and the movement distance Pr (=3×d) of three loops of the original movement distance, and divides the difference by the number of loops, which is 3, to calculate the position movement amount α (=(Pr-Pt) / 3).

[0095] Then, as shown in Figure 7, the position movement editing processing unit 213 edits the movement distance of the jumping objects that are not in contact with the floor surface, among the objects in each of the original three loops, by adding a position movement amount α to the movement distance of the non-contacting objects, so that the total movement distance in each loop becomes the original movement distance d plus the movement amount α (d+α).

[0096] More specifically, as shown in the upper left of Figure 7, in the jumping state where the object is not in contact with the floor, there are four objects, i.e., four frames. Therefore, the position movement editing processing unit 213 edits each frame's object by adding a frame-unit position movement amount α / 4 to the original movement distance.

[0097] With this editing, as shown in Figure 8, the distance traveled in each loop becomes distance (d+α), so after 3 loops the distance traveled becomes distance (3d+3α). As a result, with the edited content, the distance traveled in 3 loops becomes distance Pt (=3d+3α=Pr-3×(Pr-Pt) / 3), making it possible to move to the target position.

[0098] In other words, even in this example, the distance the object moves will be edited to change at a time that is difficult to perceive visually, namely when the object leaves the floor surface. This makes it possible to arbitrarily change and edit the object's movement distance.

[0099] As a result, it becomes possible to improve the degree of freedom in editing content that loops volumetric images (virtual viewpoint images, free viewpoint images).

[0100] <Second application example of the specific processing performed by the location editing department> In addition, while the above has described an example of adjusting the amount of movement of a human-shaped object when it moves horizontally, it is also possible to adjust the amount of height movement when a human-shaped object moves vertically.

[0101] For example, consider a scene where the original content, as shown in the upper left of Figure 9, flies vertically at a predetermined speed in the first half of one loop. Also, here we assume that the vertical movement distance during one loop, as shown by objects Rn11 to Rn13, is 0.5m.

[0102] In other words, the content in Figure 9 depicts objects Rn11 to Rn13, which are set for each frame in a scene that spans one loop, jumping from the floor and leaping (moving) to a height of 0.5m.

[0103] The change in height of the original content object in the upper left of Figure 9 can be represented graphically as shown in the upper right of Figure 9. In the upper right of Figure 9, the vertical axis represents the height from the floor, and the horizontal axis represents the frame (and the object within it).

[0104] In other words, in the frames from object Rn11 to Rn13, object Rn11 is the floor (grounded), so its height is 0. In the frames from objects Rn12 and Rn13, it is not in contact with the floor, and the height changes from the floor up to 0.5m. However, because the objects are flying against gravity, the change in height of objects Rn11 to Rn13 follows a parabolic trajectory due to the effect of gravity.

[0105] In this case, the non-contact frame detection unit 211 detects information about a frame that includes objects Rn12 and Rn13 that are in a non-contact state with the floor surface, which is an inanimate object, and supplies it to the position movement amount calculation unit 212.

[0106] The position movement amount calculation unit 212 calculates a frame-unit position movement amount for a height that matches an arbitrary height entered by the user, based on the frame information including objects Rn12 and Rn13, and supplies it to the position movement editing processing unit 213.

[0107] Here, when adjusting the positional movement from the target floor surface to a height of 1m, the positional movement amount calculation unit 212 calculates the height of object R12 by finding a function that corresponds to a parabola such that the height of object R13 in the last frame is 1m, as shown in the upper right of Figure 9, due to the action of gravity in the original content, and the positional movement amount in frame units in the height direction is calculated.

[0108] The position movement editing processing unit 213 edits each frame of objects Rn11 to Rn13 in the original content based on the amount of position movement in the height direction per frame, and edits the final height of object R13 to Rn31 to Rn33 which represent the change in height up to the target position of 1m, as shown in the lower left of Figure 9.

[0109] Through this editing process, the content is edited as shown in the lower left of Figure 9. Specifically, the content shown in the lower left of Figure 9 depicts objects Rn31 to Rn33, which are set for each frame, jumping from the floor and leaping (moving) to a height of 1 meter.

[0110] In this way, the distance an object moves is edited to change at a time that is difficult to perceive visually, that is, when the object leaves the floor surface. This makes it possible to arbitrarily change and edit the distance an object moves.

[0111] As a result, by editing to adjust the amount of change in an object at a time when it is difficult to visually perceive, it becomes possible to improve the degree of editing flexibility.

[0112] <Specific procedures of the Rotating Editorial Department> For example, consider the case where the original volumetric data 151 is content as shown in the upper left of Figure 9.

[0113] In other words, the content in Figure 10 represents a scene for one loop, showing how human-shaped objects Rd1 to Rd7, set for each frame, rotate in 60-degree increments from 0 to 360 degrees. Note that the rotation of the human-shaped objects in Figure 10 is not strictly depicted to match the rotation angle, but we will proceed with the explanation assuming they are rotating at the set angle.

[0114] The change in rotation of the original content object in the upper left of Figure 10 can be represented graphically as shown in the upper right of Figure 10. In the upper right of Figure 10, the vertical axis represents the rotation angle, and the horizontal axis represents each frame (and the object within it).

[0115] In other words, in frames Rd1 to Rd7 of the object, it is represented that the rotation starts from a non-rotating state with a rotation angle of 0 degrees, and the rotation angle changes by 60 degrees between frames up to 360 degrees, returning to a non-rotating state.

[0116] In this case, the rotation frame detection unit 231 detects information about the frame containing the rotating objects Rd2 to Rd7 and supplies it to the rotation amount calculation unit 232.

[0117] The rotation amount calculation unit 232 calculates a frame-by-frame rotation amount that matches the arbitrary target rotation angle entered by the user, based on the frame information including objects Rd2 to Rd7, and supplies it to the rotation editing processing unit 233.

[0118] For example, if the target is 540 degrees, the number of rotations is greater than 360 degrees (one loop of the original content) but less than 720 degrees (two loops), as shown in the upper left of Figure 10.

[0119] Here, when adjusting the target rotation angle from 0 to 360 degrees in one loop, the rotation amount calculation unit 232 calculates the difference of 180 degrees (=540-360) between the target 540 degrees and the rotation angle of 360 degrees for one loop in the original content, and then divides the difference by the number of loops (=1) to calculate 180 degrees as the rotation amount for one loop. Furthermore, for every six frames from object Rd2 to Rd7 that are in a rotated state, the rotation amount per frame is calculated as 30 degrees (=180 / 6) by dividing the rotation amount of 180 degrees by the number of frames, 6.

[0120] As shown in the lower right of Figure 10, the rotation editing processing unit 233 edits the frames of the original objects Rd2 to Rd7 so that the overall rotation angle of the frames of objects Rd11 to Rd17 is 540 degrees, which is the original rotation angle of 360 degrees plus the amount of rotation of 180 degrees.

[0121] More specifically, the rotation editing processing unit 233 edits the original objects Rd2 to Rd7 by adding a rotation amount of 30 degrees per frame to the frame-by-frame rotation angle of 60 degrees, based on a frame-by-frame rotation amount of 30 degrees (=180 / 6). As a result, as shown in the lower left of Figure 9, objects Rd1 to Rd7 are edited so that the frame-by-frame rotation angle becomes 90 degrees (=60+30), and a frame consisting of objects Rd11 to Rd17 is generated.

[0122] In other words, since objects Rd2 to Rd7 are in a rotated state, even if you edit them to change the rotation amount by adding it to the original rotation angle and generate objects Rd12 to Rd17, it will be difficult to visually perceive the change in rotation amount between frames.

[0123] In this way, the amount of rotation is edited to change at a time that is difficult to perceive visually, that is, when the object is rotating, making it possible to arbitrarily change and edit the amount of rotation of the object.

[0124] As a result, it becomes possible to improve the degree of freedom in editing content that loops volumetric images (virtual viewpoint images, free viewpoint images).

[0125] <Figure 5: Volumetric data editing process by the volumetric data editing department> Next, referring to the flowchart in Figure 11, the volumetric data editing process performed by the volumetric data editing unit 131 in Figure 5 will be explained.

[0126] In step S31, the position editing unit 201 determines whether or not position editing has been instructed on the original volumetric data 151 by the user operating the input unit 112, and whether or not target position and height information has been entered.

[0127] If it is determined in step S31 that a position edit has been instructed, the process proceeds to step S32.

[0128] In step S32, the non-contact frame detection unit 211 reads the original volumetric data 151, detects a frame in which the original object is in a non-contact state with an inanimate object such as the floor surface, and supplies the information of the detected frame to the positional movement amount calculation unit 212.

[0129] In step S33, the position movement amount calculation unit 212 determines whether or not a frame in a non-contact state has been detected based on whether or not a frame in a non-contact state has been supplied from the non-contact frame detection unit 211.

[0130] If it is determined in step S33 that a frame in a non-contact state has been detected, the process proceeds to step S34.

[0131] In step S34, the position movement amount calculation unit 212 calculates the amount of position movement required to move the object to the target position and height, and further allocates it according to the number of frames in which the object is in a non-contact state with an inanimate object, calculating it as a frame-unit position movement amount allocated to each frame, and supplies it to the position movement editing processing unit 213.

[0132] In step S35, the position movement editing processing unit 213 edits the objects in each frame of the original volumetric data 151 to reflect the frame-by-frame position movement amount supplied by the position movement amount calculation unit 212, and outputs the edited content to the editing result output unit 203. The editing result output unit 203 then outputs the edited content.

[0133] Furthermore, if no frame in a non-contact state is detected in step S33, the process proceeds to step S36, where the position editing unit 201 controls the output unit 113 to inform the user that position editing is not possible.

[0134] In step S37, the rotation editing unit 202 determines whether rotation editing has been instructed on the original volumetric data 151 by the user operating the input unit 112, and whether a target rotation angle has been input.

[0135] If it is determined in step S37 that rotation editing has been instructed, the process proceeds to step S38.

[0136] In step S38, the rotation frame detection unit 231 reads the original volumetric data 151, detects the frame in which the original object is in a rotational state, and supplies the information of the detected frame to the rotation amount calculation unit 232.

[0137] In step S39, the rotation amount calculation unit 232 determines whether or not a frame in a rotating state has been detected based on whether or not a frame in a rotating state has been supplied from the rotating frame detection unit 231.

[0138] If it is determined in step S39 that a frame in a rotating state has been detected, the process proceeds to step S40.

[0139] In step S40, the rotation amount calculation unit 232 calculates the amount of rotation relative to the rotation angle of the original content in order to rotate the object to the target rotation angle, and further allocates it according to the number of frames in which the object is in a rotated state, calculating it as a frame-level rotation amount allocated to each frame, and supplies it to the rotation editing processing unit 233.

[0140] In step S41, the rotation editing processing unit 233 edits the objects in each frame of the original volumetric data 151 to reflect the rotation amounts assigned on a frame-by-frame basis supplied by the rotation amount calculation unit 232, and outputs the edited content to the editing result output unit 203. The editing result output unit 203 then outputs the edited content.

[0141] Furthermore, if no frame in a rotational state is detected in step S39, the process proceeds to step S42, where the rotation editing unit 202 controls the output unit 113 to inform the user that rotation editing is not possible.

[0142] Furthermore, if it is determined in step S37 that rotation editing is not instructed, the processing in steps S38 to S42 is skipped.

[0143] In step S43, it is determined whether or not the termination of the process has been instructed. If termination has not been instructed, the process returns to step S31, and the subsequent processes are repeated. In other words, the processes from steps S31 to S43 are repeated until the termination of the process is instructed.

[0144] Then, if it is determined in step S43 that the process should be terminated, the process is terminated.

[0145] Through the above process, it becomes possible to edit the movement distance, height, and rotation angle of an object to change at times that are difficult to perceive visually, namely when the object leaves the floor or when it is rotating.

[0146] As a result, it becomes possible to improve the editing flexibility of content that loops a scene of a predetermined duration using volumetric images (virtual viewpoint images, free viewpoint images).

[0147] In the above, we have described an example of editing object movement by setting target positions in the horizontal and vertical directions at the moment the object leaves the floor surface, and then calculating and editing the amount of movement relative to those positions. However, it is also possible to similarly calculate and edit the amount of movement in the depth direction.

[0148] Furthermore, in the above, we have described an example in which, when editing the rotation of an object, a target rotation angle is set relative to the rotation direction at the timing when the object rotates on a rotation axis set perpendicular to the floor surface, and the amount of rotation for that rotation angle is calculated and edited. However, the rotation axis is not limited to perpendicular to the floor surface; it may also be horizontal. For example, the amount of rotation may be calculated and edited in the so-called yaw, pitch, and roll directions in the same way.

[0149] <<3. Second Embodiment>> (Constraint axes related to movement within the object's motion) The above describes content editing that loops a scene for a predetermined time using volumetric images (virtual viewpoint images, free viewpoint images), assuming that the object is in contact with an immobile body or is rotating.

[0150] However, when it comes to object movement, specifically movement that changes position, there are various variations depending on the relationship with a fixed object such as the floor.

[0151] For example, as shown in Figure 12, from left to right, the states include standing still on the floor, marching, walking, moonwalking or skating, spinning, running, jumping, and hovering.

[0152] Here, since the stationary state is a state of complete ground contact, it can be considered that movement of position in the horizontal and vertical directions is constrained.

[0153] Furthermore, since the foot-treading position is almost in contact with the ground, it can be considered that horizontal movement is completely constrained, while vertical movement is only slightly altered.

[0154] Furthermore, since walking involves one foot being in contact with the ground, it can be considered that, similar to stepping, horizontal movement is completely constrained, while vertical movement changes only slightly.

[0155] In moonwalking, skating, and spinning, horizontal movement can be considered as gliding, allowing for free movement and thus unconstrained. However, vertical movement can be considered constrained. It should be noted that the spinning state discussed here differs from the rotation described above in that it involves moving the axis of rotation while rotating.

[0156] Running, jumping, and hovering states are not constrained by horizontal or vertical movement of position.

[0157] Based on the above, it can be concluded that for states such as standing still, stepping, and walking, minor changes in position only require slight adjustments to the object's position on a frame-by-frame basis, resulting in minimal visual changes and thus allowing for editing.

[0158] Furthermore, while the vertical position movement is constrained for actions such as moonwalking, skating, and spinning, the horizontal position movement is not constrained, allowing for editing of the horizontal position relative to the target position.

[0159] Furthermore, since there are no constraints on horizontal and vertical positional movement for running, jumping, and hovering states, it is possible to edit the character to move to the target position in either the horizontal or vertical direction.

[0160] In the following discussion, axes that are constrained to an object in the horizontal and vertical directions will be referred to as constrained axes, and directions that are not constrained will be referred to as unconstrained axes.

[0161] Therefore, as mentioned above, regarding position editing, it can be considered that for constrained axes, editing that moves the position by only a small distance without causing visual discomfort is permitted, and for unconstrained axes, editing that moves the position by a predetermined distance that is not too large but does not cause visual discomfort is permitted.

[0162] Therefore, one approach is to define the allowable amount of editing for moving the position of an object along both a constrained axis and an unconstrained axis, and then edit the object by allocating the amount of movement on a frame-by-frame basis to the frames with constrained axes and the frames with unconstrained axes, respectively.

[0163] Here, we will explain a specific example where the target object's position is moved 300 mm horizontally.

[0164] In this example, the allowable amount for editing to move an object's position on a constrained axis is set to 0.05 (mm / frame), and the allowable amount for moving an object's position on an unconstrained axis is set to 5 (mm / frame). Thus, while the allowable amount on a constrained axis is limited to very small movements, the allowable amount on an unconstrained axis is set to be sufficiently large because it allows for relatively large movements. This is because there are no restrictions on the movement of an object on an unconstrained axis, while the movement of an object on a constrained axis is restricted (constrained).

[0165] Furthermore, the content consisting of volumetric data to be edited comprises a total of 100 frames, of which 30 frames are for objects on a constrained axis in the horizontal direction and 70 frames are for objects on an unconstrained axis in the horizontal direction.

[0166] In this case, the distance that represents the allowable amount of movement of the object's position as a whole, based on the allowable amount described above, is calculated as shown in equation (1) below.

[0167] Allowable amount of movement = 0.05 (mm) × 30 (frames) +5(mm) × (100-30) (frame) = 351.5 (mm) ...(1)

[0168] In other words, if the amount of movement relative to the target position is 351.5 mm or more, editing is not possible. Therefore, in this example where the target is 300 mm, editing is possible.

[0169] Next, the frame-by-frame movement amount in the 30 frames belonging to the constrained axis and the frame-by-frame movement amount in the 70 frames belonging to the unconstrained axis are calculated as shown in equations (2) and (3) below.

[0170] The amount of movement of the constraint axis per frame = (300 × (1.5 / 351.5)) / 30 (frames) = 0.043 (mm / frame) ...(2)

[0171] Unconstrained axis movement per frame = (300 × (350 / 351.5)) / 70 (frames) = 4.267 (mm / frame) ...(3)

[0172] Based on the frame-by-frame positional movement calculated as described above, the object in 30 frames belonging to the constrained axis and the object in 70 frames belonging to the unconstrained axis are edited so that the frame-by-frame movement is reflected in each. In other words, the frame-by-frame movement of the constrained axis and the frame-by-frame movement of the unconstrained axis are calculated as the ratio of the frame-by-frame movement of the constrained axis to the frame-by-frame movement of the unconstrained axis at the maximum allowable value.

[0173] While this explanation has focused on an example where the constraint axis is horizontal, the same principles apply when it is vertical (height).

[0174] Through the above process, it becomes possible to edit the movement of an object, which is a change in its position, so that the position and height, which are expressed as movement, change with an amount of movement that is difficult to perceive visually, depending on whether the movement is along a constrained axis or an unconstrained axis.

[0175] (Constraint axes related to rotation in the movement of an object) In the above, we have used the concept of constraint axes to explain movement that changes the position of an object, but the concept of constraint axes can also be introduced to rotation, which is another type of movement of an object.

[0176] In terms of object movement, rotation refers to a change in the object's frontal direction (either a predetermined direction). Therefore, if there is a change in the object's frontal direction, it can be considered to be rotating; if there is no change, it can be considered not to be rotating.

[0177] Therefore, when considering the rotational directions of the three axes represented by yaw, pitch, and roll, with respect to the object's center position, if rotation is possible in the yaw direction but not in the pitch and roll directions, then the yaw direction can be considered an unconstrained axis, and the pitch and roll directions can be considered constrained axes.

[0178] Furthermore, as mentioned above, regarding the editing of rotation, it can be considered that for constrained axes, editing that changes the rotation angle by only a small amount of rotation without causing visual discomfort is permissible, and for unconstrained axes, editing that changes the rotation angle by a predetermined amount of rotation that is not too large but does not cause visual discomfort is permissible.

[0179] Therefore, it is conceivable to define the allowable amount of rotation for editing the rotation angle for both the constrained axis and the unconstrained axis, and then edit the object's rotation by allocating the amount of rotation on a frame-by-frame basis for the frames with the constrained axis and the frames with the unconstrained axis, respectively.

[0180] Here, we will explain a specific example where the rotation angle of the target object is 300 degrees horizontally.

[0181] In this example, the allowable amount of rotation editing for an object on a constrained axis is set to 0.05 degrees / frame, and the allowable amount of rotation editing for an object on an unconstrained axis is set to 5 degrees / frame. Thus, while the allowable amount of rotation on a constrained axis is limited to very small amounts, the allowable amount of rotation on an unconstrained axis can be set to be sufficiently large because it can tolerate relatively large rotations. This is because the rotation of an object on an unconstrained axis (rotation axis) is unrestricted, while the rotation of an object on a constrained axis is restricted (constrained).

[0182] Furthermore, the content consisting of volumetric data to be edited comprises a total of 100 frames, of which 30 frames are for objects on the rotational constraint axis and 70 frames are for objects on the rotational unconstrained axis.

[0183] In this case, the distance that represents the allowable amount of movement of the object's position as a whole, based on the allowable amount described above, is calculated as shown in equation (4) below.

[0184] Allowable rotation amount = 0.05 (degrees) × 30 (frames) +5 (degrees) × (100 - 30) (frame) = 351.5 (degrees) ...(4)

[0185] In other words, if the amount of rotation relative to the target rotation angle is 351.5 degrees or more, editing is not possible. Therefore, in this example where the target is 300 degrees, editing is possible.

[0186] Next, the amount of rotation per frame in the 30 frames belonging to the constrained axis and the amount of rotation per frame in the 70 frames belonging to the unconstrained axis are calculated as shown in equations (5) and (6) below.

[0187] Rotation amount per frame of the constraint axis = (300 × (1.5 / 351.5)) / 30 (frames) = 0.043 (degrees / frame) ...(5)

[0188] Rotation per frame for an unconstrained axis = (300 × (350 / 351.5)) / 70 (frames) = 4.267 (degrees / frame) ...(6)

[0189] Based on the frame-by-frame rotation amounts calculated as described above, the objects in 30 frames belonging to the constrained axis and the objects in 70 frames belonging to the unconstrained axis are edited so that the frame-by-frame rotation amounts are reflected. In other words, the frame-by-frame rotation amounts for the constrained axis and the unconstrained axis are calculated as the ratio of the frame-by-frame rotation amounts for the unconstrained axis at the maximum allowable value.

[0190] Through the above process, it becomes possible to edit the rotation angle in a way that is difficult to perceive visually, depending on the rotation of the constrained axis and the rotation of the unconstrained axis, when the object is rotating in the direction of its front view.

[0191] <Example configuration of the second embodiment of the volumetric data editing department> Next, with reference to Figure 14, an example configuration of the second embodiment of the volumetric data editing unit will be described.

[0192] In addition, in the volumetric data editing unit 131' of Figure 13, components with the same functions as the volumetric data editing unit 131 of Figure 5 are denoted by the same reference numerals, and their explanations are omitted as appropriate.

[0193] In other words, the volumetric data editing unit 131' in Figure 13 differs from the volumetric data editing unit 131 in Figure 5 in that it is equipped with a constrained axis / unconstrained axis frame detection unit 251 and position movement amount calculation unit 252, as well as a constrained axis / unconstrained axis frame detection unit 261 and rotation amount calculation unit 262, instead of a non-contact frame detection unit 211 and position movement amount calculation unit 212, and a rotation frame detection unit 231 and rotation amount calculation unit 232.

[0194] The constrained axis / unconstrained axis frame detection unit 251 reads the original volumetric data 151 and, frame by frame, detects whether the movement of the original object that changes position is on a constrained axis or an unconstrained axis, and supplies the respective detection results to the position movement amount calculation unit 252.

[0195] The position movement amount calculation unit 252 stores the allowable amount for editing the movement of the frame on the constrained axis and the frame on the unconstrained axis, and calculates the overall allowable amount as explained with reference to equation (1) to determine whether the target movement amount is within the allowable amount. If the target movement amount is within the allowable amount, the position movement amount calculation unit 252 calculates the amount of movement for moving each object on the constrained axis and the unconstrained axis on a frame-by-frame basis using the calculations shown in equations (2) and (3) above, and supplies it to the position movement editing processing unit 213.

[0196] Accordingly, the position movement editing processing unit 213 edits the objects in the frames on the constrained axis and the frames on the unconstrained axis based on the amount of movement per frame.

[0197] The constrained-axis / unconstrained-axis frame detection unit 261 reads the original volumetric data 151 and, frame by frame, detects whether the rotation of the original object's movement is on a constrained axis or an unconstrained axis, and supplies the respective detection results to the rotation amount calculation unit 262.

[0198] The rotation amount calculation unit 262 stores the allowable amount for editing the rotation of each frame on the constrained axis and the frame on the unconstrained axis, and calculates the overall allowable amount as explained with reference to equation (4) to determine whether the target rotation amount is within the allowable amount. If the target rotation amount is within the allowable amount, the rotation amount calculation unit 262 calculates the rotation amount for rotating each object on the constrained axis and the unconstrained axis on a frame-by-frame basis using the calculations shown in equations (5) and (6) above, and supplies it to the rotation editing processing unit 233.

[0199] Accordingly, the rotation editing processing unit 233 edits the objects in the frames on the constrained axis and the frames on the unconstrained axis based on the amount of rotation for each frame.

[0200] <Figure 13: Volumetric data editing process by the volumetric data editing department> Next, referring to the flowchart in Figure 14, the volumetric data editing process performed by the volumetric data editing unit 131' in Figure 13 will be explained.

[0201] In step S71, the position editing unit 201 determines whether or not the user has instructed the original volumetric data 151 to edit its position or height by operating the input unit 112, and whether or not the target position or height has been entered and whether or not position editing has been instructed.

[0202] If it is determined in step S71 that a position edit has been instructed, the process proceeds to step S72.

[0203] In step S72, the constrained-axis / unconstrained-axis frame detection unit 251 reads the original volumetric data 151, detects the frames on the constrained axis and the frames on the unconstrained axis for the movement of the original object, and supplies the detection results to the position movement amount calculation unit 212.

[0204] In step S73, the position movement amount calculation unit 252 calculates the total allowable amount of movement required to move the object to the target position by performing a calculation as shown in equation (1) above, and determines whether the amount of movement relative to the target position exceeds the allowable amount.

[0205] If it is determined in step S73 that the amount of movement relative to the target position or height does not exceed the allowable limit, the process proceeds to step S74.

[0206] In step S74, the position movement amount calculation unit 252 calculates the amount of position movement required to move the object to the target position, frame by frame for each frame in which the object is on a constrained axis and frame in which it is on an unconstrained axis, using calculations as shown in equations (2) and (3) above, and supplies this calculation to the position movement editing processing unit 213.

[0207] In step S75, the position movement editing processing unit 213 edits the objects in each frame of the original volumetric data 151 to reflect the position movement amounts allocated on a frame-by-frame basis supplied by the position movement amount calculation unit 252, and outputs the edited content to the editing result output unit 203. The editing result output unit 203 then outputs the edited content.

[0208] Furthermore, if the target movement amount exceeds the allowable amount in step S73, the process proceeds to step S76, where the position editing unit 201 controls the output unit 113 to indicate that position editing is not possible.

[0209] In step S77, the rotation editing unit 202 determines whether rotation editing has been instructed on the original volumetric data 151 by the user operating the input unit 112, and whether a target rotation angle has been input.

[0210] If it is determined in step S77 that rotation editing has been instructed, the process proceeds to step S78.

[0211] In step S78, the constrained-axis / unconstrained-axis frame detection unit 261 reads the original volumetric data 151, detects the frames on the constrained axis and the frames on the unconstrained axis for the rotation of the original object, and supplies the detection results to the rotation amount calculation unit 262.

[0212] In step S79, the rotation amount calculation unit 262 calculates the total allowable amount of rotation required to rotate the object to the target rotation angle by performing a calculation as shown in equation (4) above, and determines whether the amount of rotation relative to the target rotation angle exceeds the allowable amount.

[0213] If, in step S79, it is determined that the amount of rotation relative to the target rotation angle does not exceed the allowable limit, the process proceeds to step S80.

[0214] In step S80, the rotation amount calculation unit 262 calculates the amount of rotation required to rotate the object to the target rotation angle for each frame, for both frames in which the object is on a constrained axis and frames in which it is on an unconstrained axis, using calculations as shown in equations (5) and (6) above, and supplies this calculation to the rotation editing processing unit 233.

[0215] In step S81, the rotation editing processing unit 233 edits the objects in each frame of the original volumetric data 151 to reflect the rotation amounts assigned on a frame-by-frame basis supplied by the rotation amount calculation unit 262, and outputs the edited content to the editing result output unit 203. The editing result output unit 203 then outputs the edited content.

[0216] Furthermore, if the target rotation amount exceeds the allowable amount in step S79, the process proceeds to step S82, where the rotation editing unit 202 controls the output unit 113 to indicate that rotation editing is not possible.

[0217] In the above processes, for frames where the object lies on a constraint axis related to movement, editing the object's position based on minute positional movement amounts makes it possible to perform edits that are difficult to perceive visually. Furthermore, for frames where the object lies on an unconstrained axis related to movement, editing the object's position based on positional movement amounts that do not cause visual inconsistencies makes it possible to perform edits that are difficult to perceive visually.

[0218] Furthermore, for frames where an object lies on a constraint axis related to rotation, editing the object's rotation based on minute rotation amounts makes it possible to perform edits that are difficult to perceive visually. Similarly, for frames where an object lies on an unconstrained axis related to rotation, editing the object's position based on rotation amounts that do not create visual inconsistencies makes it possible to perform edits that are difficult to perceive visually.

[0219] As a result, it becomes possible to improve the editing flexibility of content that loops a scene of a predetermined duration using volumetric images (virtual viewpoint images, free viewpoint images).

[0220] Furthermore, while the above describes processing based on whether an object is constrained on a constraint axis or not, it is also possible to set a constraint range on the constraint axis, which is the range within which the object is constrained, and to set the allowable amount to change according to the constraint range.

[0221] In other words, for example, when the floor surface is the vertical constraint axis and the constraint state is one in which the object does not leave the floor surface, the allowable amount can only be set to a small amount relative to the target vertical height. However, in the case of a constraint range of a few centimeters from the floor surface, the allowable amount of positional movement in the height direction may be set to increase as the constraint range increases.

[0222] Furthermore, in the above examples, we have described cases where, for objects on a constrained axis whose direction of movement or rotation is restricted, the allowable amount of movement or rotation is set to a small value, while for objects on an unconstrained axis whose direction of movement or rotation is not restricted, the allowable amount of movement or rotation is set to a relatively large value.

[0223] Furthermore, in the first embodiment, an object that is not in contact with the floor surface is part of an object on an unconstrained axis with respect to the direction of movement in the second embodiment, and in the first embodiment, an object in a rotating state is part of an object on an unconstrained axis with respect to the direction of rotation in the second embodiment.

[0224] In other words, in the first embodiment, the position, height, and rotation angle of objects on an unconstrained axis were edited. In contrast, in the second embodiment, it can be said that editing of the position, height, and rotation angle of objects on a constrained axis is achieved in a way that makes it difficult to visually perceive, in addition to editing of objects on an unconstrained axis.

[0225] <<4. Description of a computer using this technology>>

[0226] The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up that software are installed on a computer. Here, "computer" includes computers built into dedicated hardware, as well as general-purpose personal computers, for example, that can perform various functions by installing various programs.

[0227] Figure 15 is a block diagram showing an example of a computer hardware configuration in which the series of processes described above are executed by a program.

[0228] In a computer, the processing circuit 1001, ROM (Read Only Memory) 1002, and RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0229] An input / output interface 1005 is further connected to the bus 1004. An input / output interface 1005 is connected to an input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010.

[0230] The input unit 1006 may include physical or virtual operating means that the user operates to input information, such as a keyboard, mouse, or touch panel, as well as means that the user inputs information through voice, eye gaze, etc. Furthermore, the input unit 1006 may include sensors for inputting various physical quantities into the computer. For example, the input unit 1006 may include sensors that acquire physical quantities such as light (including infrared light other than visible light) or sound, such as a camera or microphone. Also, for example, the input unit 1006 may include sensors that acquire other physical quantities such as temperature, moisture content, acceleration, and distance. The output unit 1007 may include means that present information to the user by stimulating the user's perception, such as a display, speaker, or haptic device. The storage unit 1008 is composed of a hard disk, non-volatile or volatile memory, etc., and stores various types of information (including programs). The communication unit 1009 is a network interface, etc., and performs wired or wireless communication with the outside. The drive 1010 drives removable media 1011 such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory.

[0231] The processing circuit 1001 includes a processor that executes programs such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor). The processing circuit 1001 (its processor) performs the series of processes described above by loading the program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it. The processing circuit 1001 can output the processing results of the series of processes from the output unit 1007, for example, via the bus 1004 and the input / output interface 1005, as needed. The processing circuit 1001 can also store the processing results in the memory unit 1008 or transmit them from the communication unit 1009.

[0232] The program executed by the computer (processing circuit 1001) can be provided by recording it on a removable medium 1011, such as a package medium. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0233] In a computer, a program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable media 1011 into the drive 1010. Alternatively, a program can be received by the communication unit 1009 from another device, such as a server, via a wired or wireless transmission medium, and installed in the storage unit 1008. Furthermore, programs can be pre-installed in the ROM 1002 or the storage unit 1008.

[0234] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0235] The processes that a computer performs according to a program do not necessarily have to follow the order described in the flowchart. In other words, the processes that a computer performs according to a program include processes that are executed in parallel or individually (e.g., parallel processing and object-based processing).

[0236] A program may be processed by a single computer (processor), or it may be processed in a distributed manner by multiple computers. Furthermore, a program may be transferred to a remote computer and executed there.

[0237] When the computer executes a program to perform the series of processes described above, the processing circuit 1001 (its processor) functions as the volumetric data editing unit 131 in Figure 5 and the volumetric data editing unit 131 in Figure 13 by executing the program.

[0238] In this specification, a system means one component or a collection of multiple components (devices, modules (parts), etc.). Therefore, one or more components of a computer, for example, only the processor, or a combination of the processor and memory, for example, only the processing circuit 1001, or a combination of the processing circuit 1001 to the bus 1004, etc., constitute a system. Regarding a collection of multiple components, it is not necessary whether all components reside in the same enclosure or not. Therefore, multiple devices housed in separate enclosures and connected via a network, or a single device containing multiple modules within a single enclosure, are all systems. Furthermore, for example, an entire computer, or a combination of a computer and other devices such as a server (not shown), also constitutes a system.

[0239] Furthermore, for example, each step of a flowchart may be executed by one device, or it may be divided among multiple devices. Additionally, if a single step includes multiple processes, these processes may be executed by one device, or they may be divided among multiple devices. In other words, multiple processes included in a single step can be executed as multiple steps. Conversely, processes described as multiple steps can be combined and executed as a single step.

[0240] Furthermore, for example, a program executed by a computer may be structured so that the steps of the program are executed chronologically in the order described herein, or they may be executed in parallel or individually at necessary times, such as when a call is made. In other words, the steps may be executed in an order different from the order described above, as long as no inconsistencies arise. Moreover, the steps of this program may be executed in parallel with the processing of other programs, or in combination with the processing of other programs.

[0241] Furthermore, for example, the various technologies relating to this disclosure can be implemented independently, as long as they do not conflict with each other. Of course, any combination of the disclosures can also be implemented. For example, some or all of the disclosures described in one embodiment can be implemented in combination with some or all of the disclosures described in another embodiment. Also, some or all of the aforementioned disclosures can be implemented in combination with other technologies not described above.

[0242] Furthermore, this disclosure can also be structured as follows: <1> An information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, A frame detection unit detects frames in which the object represented by the volumetric data is not constrained in movement along a predetermined axis direction as an unconstrained frame, An editing unit which edits the movement represented by the object in at least the unconstrained frame of the aforementioned multiple frames so that it becomes a movement different from the aforementioned movement. An information processing system equipped with the following features. <2> The frame detection unit detects, among the plurality of frames, the unconstrained frames in which the movement of the object represented by the plurality of frames is not constrained in terms of the movement in the predetermined axis direction, The editorial department edits the position of the object in at least the unconstrained frame among the multiple frames to a position different from the position used to represent the other movement. <1> The information processing system described above. <3> The system further includes a position movement amount calculation unit that calculates the position movement amount which is the difference between the position of the object in the unconstrained frame and the other position, and calculates the position movement amount per frame of the unconstrained frame based on the number of unconstrained frames. The editorial department edits the unconstrained frame by moving the position of each of the objects in the unconstrained frame based on the amount of positional movement of the unconstrained frame per frame. <2> The information processing system described above. <4> The frame detection unit further detects, among the plurality of frames, frames in which the movement of the object's position is constrained with respect to the predetermined axis direction, as constrained frames. The position movement amount calculation unit calculates the position movement amount of the object in the unconstrained frame per frame and the position movement amount of the object in the constrained frame per frame, The editorial department edits the position of each object in the unconstrained frame based on the frame-unit positional movement amount of the unconstrained frame, and edits the position of each object in the constrained frame based on the frame-unit positional movement amount of the constrained frame. <3> The information processing system described above. <5> The frame-unit positional movement amount of the unconstrained frame and the frame-unit positional movement amount of the constrained frame are determined as the ratio of the value obtained by multiplying the allowable frame-unit positional movement amount of the unconstrained frame by the number of unconstrained frames out of all frames, and the value obtained by multiplying the allowable frame-unit positional movement amount of the constrained frame by the number of constrained frames out of all frames. <4> The information processing system described above. <6> The aforementioned unconstrained frame is a frame in which the object is not in contact with an inanimate body. <1> The information processing system described above. <7> The frame detection unit detects, among the multiple frames, frames in which the rotation of the object represented by the multiple frames is not constrained with respect to the predetermined axis direction as unconstrained frames. The editorial department edits the rotation angle of the object in at least the unconstrained frame among the multiple frames to a rotation angle different from the rotation angle used to represent the other movement. <1> The information processing system described above. <8> The system further includes a rotation amount calculation unit that calculates the rotation amount which is the difference between the rotation angle of the object in the unconstrained frame and the other rotation angles, and calculates the rotation amount per frame of the unconstrained frame based on the number of unconstrained frames. The editorial department edits the rotation angle of each of the objects in the unconstrained frame based on the amount of rotation of the unconstrained frame per frame. <7> The information processing system described above. <9> The frame detection unit further detects, among the plurality of frames, frames in which the object is constrained to rotation in the predetermined axis direction as constrained frames, The rotation amount calculation unit calculates the rotation amount of the rotation angle of the object in the unconstrained frame on a per-frame basis, and the rotation amount of the position of the object in the constrained frame on a per-frame basis. The editorial department edits the rotation angle of each object in the unconstrained frame based on the rotation amount of the unconstrained frame per frame, and edits the rotation angle of each object in the constrained frame based on the rotation amount of the constrained frame per frame. <8> The information processing system described above. <10> The amount of rotation per frame of the unconstrained frame and the amount of rotation per frame of the constrained frame are the ratio of the allowable amount of rotation per frame of the unconstrained frame multiplied by the number of unconstrained frames and the allowable amount of rotation per frame of the constrained frame multiplied by the number of constrained frames. <9> The information processing system described above. <11> The unconstrained frame is the frame in which the object is rotating. <1> The information processing system described above. <12> An information processing method for an information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, A frame detection process is performed to detect frames in which the object represented by the volumetric data is not constrained in movement along a predetermined axis direction as an unconstrained frame, Editing the motion represented by the object in at least the unconstrained frame among the multiple frames so that it becomes a different motion from the motion described above. Information processing methods including <13> The frame detection process detects, among the multiple frames, frames in which the movement of the object represented by the multiple frames is not constrained in relation to the predetermined axis direction, as unconstrained frames. The editing process involves editing the position of the object in at least the unconstrained frame among the multiple frames to a position different from the position used to represent the other movement. <12> The information processing method described above. <14> The process further includes calculating the positional movement amount which is the difference between the position of the object in the unconstrained frame and the other positions, and calculating the positional movement amount per frame of the unconstrained frame based on the number of unconstrained frames, The editing process involves moving and editing the position of each object in the unconstrained frame based on the frame-by-frame positional movement amount of the unconstrained frame. <13> The information processing method described above. <15> The frame detection process further detects, among the multiple frames, frames in which the movement of the object's position is constrained with respect to the predetermined axis direction, as constrained frames. The position movement amount calculation process calculates the position movement amount of the object in the unconstrained frame per frame and the position movement amount of the object in the constrained frame per frame, The editing process edits the position of each object in the unconstrained frame based on the frame-unit positional movement amount of the unconstrained frame, and edits the position of each object in the constrained frame based on the frame-unit positional movement amount of the constrained frame. <14> The information processing method described above. <16> The frame detection process detects, among the multiple frames, frames in which the rotation of the object represented by the multiple frames is not constrained with respect to the predetermined axis direction as unconstrained frames. The editing process involves editing the rotation angle of an object in at least the unconstrained frame among the multiple frames to a rotation angle different from the rotation angle used to represent the other movement. <12> The information processing method described above. <17> The process further includes calculating the rotation amount which is the difference between the rotation angle of the object in the unconstrained frame and the other rotation angles, and calculating the rotation amount per frame of the unconstrained frame based on the number of unconstrained frames. The editing process edits the rotation angle of each object in the unconstrained frame based on the rotation amount of the unconstrained frame per frame. <16> The information processing method described above. <18> The frame detection process further detects, among the plurality of frames, the frame in which the object is constrained to rotation in the predetermined axis direction as a constrained frame, The rotation amount calculation process calculates the rotation amount per frame of the rotation angle of the object in the unconstrained frame and the rotation amount per frame of the position of the object in the constrained frame. The editing process edits the rotation angle of each object in the unconstrained frame based on the rotation amount per frame of the unconstrained frame, and edits the rotation angle of each object in the constrained frame based on the rotation amount per frame of the constrained frame. <17> The information processing method described above. <19> The amount of rotation per frame of the unconstrained frame and the amount of rotation per frame of the constrained frame are the ratio of the allowable amount of rotation per frame of the unconstrained frame multiplied by the number of unconstrained frames and the allowable amount of rotation per frame of the constrained frame multiplied by the number of constrained frames. <18> The information processing method described above. <20> A computer comprising an information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, A frame detection unit detects frames in which the object represented by the volumetric data is not constrained in movement along a predetermined axis direction as an unconstrained frame, An editing department edits the motion represented by the object in at least the unconstrained frame of the aforementioned frame to become a different motion from the aforementioned motion. A program that makes something work. [Explanation of symbols]

[0243] 101 Information processing unit, 131 Volumetric data editing unit, 201 Position editing unit, 202 Rotation editing unit, 203 Editing result output unit, 211 Non-contact frame detection unit, 212 Position movement amount calculation unit, 213 Position movement editing processing unit, 231 Rotation frame detection unit, 232 Rotation amount calculation unit, 233 Rotation editing processing unit, 251 Constrained axis / unconstrained axis frame detection unit, 252 Position movement amount calculation unit

Claims

1. An information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, A frame detection unit detects frames in which the object represented by the volumetric data is not constrained in movement along a predetermined axis direction as an unconstrained frame, An editing unit which edits the movement represented by the object in at least the unconstrained frame of the aforementioned multiple frames so that it becomes a movement different from the aforementioned movement. An information processing system equipped with the following features.

2. The frame detection unit detects, among the plurality of frames, the unconstrained frames in which the movement of the object represented by the plurality of frames is not constrained in terms of the movement in the predetermined axis direction, The editorial department edits the position of the object in at least the unconstrained frame among the multiple frames to a position different from the position used to represent the other movement. The information processing system according to claim 1.

3. The system further includes a position movement amount calculation unit that calculates the position movement amount which is the difference between the position of the object in the unconstrained frame and the other position, and calculates the position movement amount per frame of the unconstrained frame based on the number of unconstrained frames. The editorial department edits the unconstrained frame by moving the position of each of the objects in the unconstrained frame based on the amount of positional movement of the unconstrained frame per frame. The information processing system according to claim 2.

4. The frame detection unit further detects, among the plurality of frames, frames in which the movement of the object's position is constrained with respect to the predetermined axis direction, as constrained frames. The position movement amount calculation unit calculates the position movement amount of the object in the unconstrained frame per frame and the position movement amount of the object in the constrained frame per frame, The editorial department edits the position of each object in the unconstrained frame based on the frame-unit positional movement amount of the unconstrained frame, and edits the position of each object in the constrained frame based on the frame-unit positional movement amount of the constrained frame. The information processing system according to claim 3.

5. The frame-unit positional movement amount of the unconstrained frame and the frame-unit positional movement amount of the constrained frame are determined as the ratio of the value obtained by multiplying the allowable frame-unit positional movement amount of the unconstrained frame by the number of unconstrained frames out of all frames, and the value obtained by multiplying the allowable frame-unit positional movement amount of the constrained frame by the number of constrained frames out of all frames. The information processing system according to claim 4.

6. The aforementioned unconstrained frame is a frame in which the object is not in contact with an inanimate body. The information processing system according to claim 1.

7. The frame detection unit detects, among the multiple frames, frames in which the rotation of the object represented by the multiple frames is not constrained with respect to the predetermined axis direction as unconstrained frames. The editorial department edits the rotation angle of the object in at least the unconstrained frame among the multiple frames to a rotation angle different from the rotation angle used to represent the other movement. The information processing system according to claim 1.

8. The system further includes a rotation amount calculation unit that calculates the rotation amount which is the difference between the rotation angle of the object in the unconstrained frame and the other rotation angles, and calculates the rotation amount per frame of the unconstrained frame based on the number of unconstrained frames. The editorial department edits the rotation angle of each of the objects in the unconstrained frame based on the amount of rotation of the unconstrained frame per frame. The information processing system according to claim 7.

9. The frame detection unit further detects, among the plurality of frames, frames in which the object is constrained to rotation in the predetermined axis direction as constrained frames, The rotation amount calculation unit calculates the rotation amount of the rotation angle of the object in the unconstrained frame on a per-frame basis, and the rotation amount of the position of the object in the constrained frame on a per-frame basis. The editorial department edits the rotation angle of each object in the unconstrained frame based on the rotation amount of the unconstrained frame per frame, and edits the rotation angle of each object in the constrained frame based on the rotation amount of the constrained frame per frame. The information processing system according to claim 8.

10. The amount of rotation per frame of the unconstrained frame and the amount of rotation per frame of the constrained frame are the ratio of the allowable amount of rotation per frame of the unconstrained frame multiplied by the number of unconstrained frames and the allowable amount of rotation per frame of the constrained frame multiplied by the number of constrained frames. The information processing system according to claim 9.

11. The unconstrained frame is the frame in which the object is rotating. The information processing system according to claim 1.

12. An information processing method for an information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, A frame detection process is performed to detect frames in which the object represented by the volumetric data is not constrained in movement along a predetermined axis direction as an unconstrained frame, Editing the motion represented by the object in at least the unconstrained frame among the multiple frames so that it becomes a different motion from the motion described above. Information processing methods including

13. The frame detection process detects, among the multiple frames, frames in which the movement of the object represented by the multiple frames is not constrained in relation to the predetermined axis direction, as unconstrained frames. The editing process involves editing the position of the object in at least the unconstrained frame among the multiple frames to a position different from the position used to represent the other movement. The information processing method according to claim 12.

14. The process further includes calculating the positional movement amount which is the difference between the position of the object in the unconstrained frame and the other positions, and calculating the positional movement amount per frame of the unconstrained frame based on the number of unconstrained frames, The editing process involves moving and editing the position of each object in the unconstrained frame based on the frame-by-frame positional movement amount of the unconstrained frame. The information processing method according to claim 13.

15. The frame detection process further detects, among the multiple frames, frames in which the movement of the object's position is constrained with respect to the predetermined axis direction, as constrained frames. The position movement amount calculation process calculates the position movement amount of the object in the unconstrained frame per frame and the position movement amount of the object in the constrained frame per frame, The editing process edits the position of each object in the unconstrained frame based on the frame-unit positional movement amount of the unconstrained frame, and edits the position of each object in the constrained frame based on the frame-unit positional movement amount of the constrained frame. The information processing method according to claim 14.

16. The frame detection process detects, among the multiple frames, frames in which the rotation of the object represented by the multiple frames is not constrained with respect to the predetermined axis direction as unconstrained frames. The editing process involves editing the rotation angle of an object in at least the unconstrained frame among the multiple frames to a rotation angle different from the rotation angle used to represent the other movement. The information processing method according to claim 12.

17. The process further includes calculating the rotation amount which is the difference between the rotation angle of the object in the unconstrained frame and the other rotation angles, and calculating the rotation amount per frame of the unconstrained frame based on the number of unconstrained frames. The editing process edits the rotation angle of each object in the unconstrained frame based on the rotation amount of the unconstrained frame per frame. The information processing method according to claim 16.

18. The frame detection process further detects, among the plurality of frames, the frame in which the object is constrained to rotation in the predetermined axis direction as a constrained frame, The rotation amount calculation process calculates the rotation amount per frame of the rotation angle of the object in the unconstrained frame and the rotation amount per frame of the position of the object in the constrained frame. The editing process edits the rotation angle of each object in the unconstrained frame based on the rotation amount per frame of the unconstrained frame, and edits the rotation angle of each object in the constrained frame based on the rotation amount per frame of the constrained frame. The information processing method according to claim 17.

19. The amount of rotation per frame of the unconstrained frame and the amount of rotation per frame of the constrained frame are the ratio of the allowable amount of rotation per frame of the unconstrained frame multiplied by the number of unconstrained frames and the allowable amount of rotation per frame of the constrained frame multiplied by the number of constrained frames. The information processing method according to claim 18.

20. A computer comprising an information processing system that loops and plays a scene consisting of multiple frames generated from volumetric data a predetermined number of times, A frame detection unit detects frames in which the object represented by the volumetric data is not constrained in movement along a predetermined axis direction as an unconstrained frame, An editing department edits the motion represented by the object in at least the unconstrained frame of the aforementioned frame to become a different motion from the aforementioned motion. A program that makes something work.