Information processing device, information processing method, and program

Abstract

To reduce the load on time-series processing on shape data collected over time for a plurality of objects in a space.SOLUTION: An information processing apparatus performs predetermined time-series processing on a frame range set from among frame groups constituting time-series shape data, and excludes partial shape data of pieces of shape data belonging to frames in the set frame range.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to time series processing of shape data representing the three-dimensional shape of an object. [Background technology]

[0002] The shape data of objects used in fields such as virtual viewpoint video and 3D printing is placed in a space in which the objects exist, with the number of objects corresponding to the number of objects. In this case, the larger the space in which the objects exist, the more difficult it becomes to allocate memory sufficient to place all of the basic unit elements of the shape data (e.g., voxels when the shape data is volume data) within that space. Therefore, sparse data management is used, in which unit elements are assigned only to areas in the space where objects exist, and no unit elements are assigned to areas where no objects exist. Patent Document 1 discloses a method for obtaining high-resolution volume data by generating low-resolution voxel models for each of multiple objects existing in the space and 3D modeling the bounding boxes of these objects with smaller voxels. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-71749 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology of Patent Document 1 aims to speed up the overall processing by managing volume data sparsely and adjusting the spatial resolution in the process of generating a voxel model for each object. However, the volume data handled by Patent Document 1 is discontinuous in the time direction, making it difficult to speed up time-series processing of volume data collected over time. This is because time-series processing requires determining, one by one, to which voxel model a certain voxel in a certain voxel model in one frame belongs in another frame. Furthermore, the greater the number of objects present in a space, the greater the number of times processing is required to search for the voxel model to which each voxel belongs between frames, and the longer the processing time required. [Means for solving the problem]

[0005] The information processing device according to the present disclosure includes: an acquisition unit for acquiring time-series shape data constituted by a group of temporally consecutive frames, each of which contains shape data indicating a three-dimensional shape of an object; and a frame group for selecting a shape from among the acquired time-series shape data. a setting means for setting a frame range according to the content of a predetermined time series processing; configured The aforementioned For a range of frames, The aforementioned a processing means for performing predetermined time series processing, performing the predetermined time series processing by excluding shape data that does not satisfy a predetermined condition between the shape data of interest belonging to the frame of interest and the shape data of interest belonging to the frame of interest from the shape data of other frames in the set frame range other than the frame of interest; It is characterized by: [Effects of the Invention]

[0006] According to the technology of the present disclosure, it is possible to reduce the load of time-series processing on shape data collected over time for multiple objects in a space. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of a hardware configuration of an information processing apparatus. [Figure 2] FIG. 2 is a block diagram showing the software configuration (functional configuration) of the information processing device. [Figure 3](a) is a conceptual diagram of time-series volume data, and (b) is a diagram explaining the position information of the bounding box. [Figure 4] 4 is a flowchart showing the flow of time-series processing according to the first embodiment. [Figure 5] FIG. 1 is a schematic diagram illustrating the case where the time-series processing according to the first embodiment is applied to time-series volume data. [Figure 6] FIG. 10 is an explanatory diagram of a correction to the result of the smoothing process. [Figure 7] 10 is a flowchart showing the flow of time-series processing according to the second embodiment. [Figure 8] 10A and 10B are diagrams illustrating the reuse of collision detection results. [Figure 9] 10A and 10B are diagrams showing an example of a determination result list. [Figure 10] 11 is a flowchart showing the flow of time-series processing according to the third embodiment. [Figure 11] FIG. 11 is a schematic diagram illustrating the case where the time-series processing according to the third embodiment is applied to time-series volume data. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the present disclosure, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0009] Before describing each embodiment, the concept of object (subject) shape data will be reviewed. In this embodiment, volume data using voxels as basic unit elements representing a shape will be described as an example. In the case of volume data, the object's shape is represented by a plurality of unit elements (voxels) that divide the object's circumscribing rectangle (commonly called a "bounding box") set in three-dimensional space into a grid. Each voxel has information (hereinafter referred to as a "voxel value") indicating whether or not it exists within the object's physical entity in three-dimensional space. In this embodiment, the voxel value is described as having binary information of "0" or "1." A voxel with a voxel value of "1" is referred to as an "ON voxel" because some or all of it exists within the object. A voxel with a voxel value of "0" is referred to as an "OFF voxel" because none of it exists within the object (i.e., none of it exists outside the object).

[0010] In the description of each embodiment, time-series shape data consisting of multiple frames consecutive in the time direction, with one or more bounding boxes set for each frame, and using voxels as unit elements, will be referred to as "time-series volume data." Note that some frames may not contain shape data. Note that the data format to which this embodiment can be applied is not limited to volume data. In other words, it can also be applied to point cloud data, which represents the three-dimensional shape of an object as a set of points, mesh data, which represents a set of polygons, and the like.

[0011] [Embodiment 1] <Hardware configuration> FIG. 1 is a diagram showing an example of the hardware configuration of an information processing device 100 according to this embodiment, which performs time-series processing on time-series volume data. In FIG. 1, a CPU 101 uses a RAM 102 as a work memory, executes programs stored in a ROM 103 and a hard disk drive (HDD) 105, and controls the operation of each block (described later) via a system bus 112. An HDD I / F 104 is an interface, such as a serial ATA (SATA), that connects a secondary storage device, such as the HDD 105 or an optical disk drive. The CPU 101 can read data from and write data to the HDD 105 via the HDD I / F 104. Furthermore, the CPU 101 can load data stored in the HDD 105 into the RAM 102, and conversely, can save data loaded in the RAM 102 to the HDD 105. The CPU 101 can then execute a program using the data loaded in the RAM 102. The input I / F 106 is a serial bus interface such as USB or IEEE1394, which connects an input device 107 such as a keyboard, a mouse, a digital camera, or a scanner. The CPU 101 can read data from the input device 107 via the input I / F 106. The output I / F 108 is a video output interface such as DVI or HDMI (registered trademark), which connects the information processing device 100 to a display, which is an output device. The CPU 101 can send data to the display via the output I / F 108 and display a predetermined video on the display. The network I / F 110 is an interface that connects to a network such as the Internet or a LAN. The CPU 101 can exchange various data with an external server, etc., connected to the LAN or the Internet via the network I / F 110.

[0012] <Software configuration> 2 is a block diagram showing the software configuration of the information processing device 100, more specifically, the functional configuration for executing predetermined time-series processing on input time-series volume data. The various functions shown in FIG. 2 are realized by the CPU 101 executing dedicated programs.

[0013] The information processing device 100 of this embodiment includes a data acquisition unit 201 , a frame range setting unit 202 , a reference model determination unit 203 , and a time series processing unit 204 .

[0014] The data acquisition unit 201 acquires time-series volume data consisting of multiple frames collected over time, for example, from another information processing device (not shown) on a LAN via the network I / F 110. FIG. 3(a) is a conceptual diagram of time-series volume data. A frame at a certain time T includes bounding boxes for each object, the number of which is equal to the number of objects present in the target space at that time T. Here, as shown in FIG. 3(b), the three-dimensional position of each bounding box 301 is expressed by relative coordinates 302 with respect to the origin coordinates in a common space. Note that if multiple objects are in contact with each other, one bounding box will correspond to multiple objects.

[0015] The frame range setting unit 202 sets a frame range consisting of two or more consecutive frames to be processed when performing time-series processing on the time-series volume data acquired by the data acquisition unit 201, according to the content and intent of the time-series processing. Here, time-series processing means comparing voxel sets (hereinafter referred to as "voxel models") for each object included in each frame of the time-series volume data between multiple frames, and modifying the voxel model in a specific frame or changing the number of frames. Modification of the voxel model includes changes to the shape, position, voxel values, and assigned labels of the voxel model, and changing the number of frames includes adding (interpolating) or deleting frames.

[0016] The reference model determination unit 203 determines a voxel model of another frame to be referenced when changing the voxel model (target voxel model) that is the target of the smoothing process in the time direction, based on the frame range set by the frame range setting unit 202. A specific determination method will be described later. The time series processing unit 204 performs predetermined time series processing on a voxel model of interest in a frame of interest within a set frame range, using voxel models of other frames determined by the reference model determination unit 203. In this embodiment, an example will be described in which smoothing processing in the time direction is performed as the predetermined time series processing.

[0017] <Details of time series processing> 4 is a flowchart showing the flow of executing smoothing processing in the time direction on input time-series volume data according to this embodiment. Note that in the following explanation, the symbol "S" means step. First, in S401, the data acquisition unit 201 acquires time-series volume data. At this time, the data may be received and acquired from another information processing device (not shown), or may be previously received and saved and read from the HDD 105. Here, it is assumed that time-series volume data 500 consisting of multiple frames, created based on video captured of a space in which three dynamic objects exist, as shown in FIG. 5, has been acquired. In FIG. 5, white squares 501 represent off-voxels, and black squares 502 represent on-voxels. Note that although FIG. 5 schematically represents voxels as a two-dimensional grid, the actual entity being handled is a three-dimensional volume.

[0018] In S402, the frame range setting unit 202 sets a frame range to be subjected to the time-direction smoothing process for the time-series volume data acquired in S401. In the time-direction smoothing process, the frame to which the voxel model to be smoothed belongs is set as the frame of interest, and voxel models belonging to its adjacent frames (the previous frame and the subsequent frame) are referenced. Therefore, in this embodiment, three consecutive frames 503 as shown in FIG. 5 are set as the frame range to be processed. However, since there is no frame previous to the frame of interest immediately after the start of processing and no frame subsequent to the frame of interest immediately before the end of processing, two consecutive frames are set as the frame range. Note that the frame range to be set may be associated with the processing content and stored in advance in the HDD 105 or the like.

[0019] In S403, the reference model determination unit 203 sets a frame to which the voxel model to be smoothed belongs from the frame range set in S402 as a frame of interest. In the example of Fig. 5, the second frame (t=n) of the three frames 503 is set as the frame of interest.

[0020] In S404, the reference model determination unit 203 sets a target voxel model to be smoothed from one or more voxel models included in the target frame. Here, the description will be continued assuming that, in the example of Fig. 5, the voxel model 504a is set as the target voxel model from among the three voxel models 504a to 504c included in the frame t=n as the target frame.

[0021] In S405, the reference model determination unit 203 performs a reference determination between a voxel model included in a frame other than the frame of interest within the set frame range and the voxel model of interest. For the reference determination here, collision determination using the bounding box method, for example, is used. In the bounding box method, when the following formula (1) is satisfied for two voxel models v1 and v2, it is determined that the two voxel models v1 and v2 collide.

[0022] (v1x_max>v2x_min)∧(v2x_max>v1x_min)∧(v1y_max>v2y_min)∧(v2y_max>v1y_min)∧(v1z_max>v2z_min)∧(v2z_max>v1z_min) ...Equation (1) In the above formula (1), v1x_max, v1x_min, v1y_max, v1y_min, v1z_max, and v1z_min represent the maximum and minimum values ​​of the x, y, and z axes for voxel model v1. Furthermore, v2x_max, v2x_min, v2y_max, v2y_min, v2z_max, and v2z_min represent the maximum and minimum values ​​of the x, y, and z axes for voxel model v2. In the example of FIG. 5, assume that voxel model 504 is the target voxel model. In this case, for example, voxel model 504 may be designated as voxel model V1, and voxel models 505a to 505c included in the frame at t=n-1 and voxel models 506a to 506c included in the frame at t=n+1 may be designated as voxel models v2, and sequential determination may be performed. Note that collision detection may be performed using other methods, such as the bounding sphere method, which uses the radii of two voxel models to perform collision detection. Voxel models that are determined not to collide by this collision detection are excluded from the processing target in S407, which will be described later. In an example where voxel model 504 is the target voxel model, voxel model 505a is determined to collide with voxel model 504 in frame t=n-1, and voxel models 506a and 506b are determined to collide with voxel model 504 in frame t=n-1. As a result, voxel models 505b, 505c, and 506c are excluded from the processing target in S407.

[0023] In S406, it is determined whether collision detection between the voxel model of interest and each voxel model included in other frames other than the frame of interest within the set frame range has been completed. If collision detection with all voxel models included in other frames has been completed, the process proceeds to S407, and if not, the process returns to S405 to continue collision detection with the remaining voxel models.

[0024] In S407, the time series processing unit 204 performs a time-direction smoothing process on the voxel model of interest using the voxel model determined to collide in the collision determination in S405. Specifically, by referring to voxel models determined to collide in other frames, if an on-voxel exists in two or more frames, the voxel model is changed to an on-voxel, and if not, the voxel model is changed to an off-voxel using a majority filter. As a result, in the example of Fig. 5, the voxel model 504 in the frame at t=n is changed to the voxel model 507.

[0025] In S408, it is determined whether the smoothing process has been completed for all voxel models included in the frame of interest. If the smoothing process has been completed for all voxel models, the process proceeds to S409. If not, the process returns to S404 to determine the next voxel model of interest and continue the process.

[0026] In S409, it is determined whether the smoothing process for the input time-series volume data is complete. If it is complete, this process ends. If it is not complete, the process returns to S402, where the next frame range is determined and the process continues.

[0027] The above is the flow of the smoothing process in the time direction for time-series volume data according to this embodiment. In this way, in this embodiment, the smoothing process in the time direction for time-series volume data can be efficiently performed.

[0028] <Modification> In one frame, multiple objects represented as separate voxel models 601 and 602 (see FIG. 6(a)) may be represented by a single voxel model 603 (see FIG. 6(b)) because they are close to each other in another frame. Performing the aforementioned time-domain smoothing process on time-series volume data including such frames may result in an On voxel group 604 corresponding to part of another object being mixed into the voxel model 601' after the smoothing process on the voxel model 601, as shown in FIG. 6(c), for example. If the smoothing process results in two or more discontinuous On voxel groups being included within a single bounding box, an additional correction process may be performed to delete the extra On voxel groups, thereby correcting the shape represented by the voxel model. Specifically, in the case of FIG. 6(c) above, the On voxel groups with a high shape similarity to the On voxel groups in the voxel model 601 (see FIG. 6(a)) before the smoothing process are retained, and all On voxel groups 604 with a low shape similarity are replaced with Off voxels. The similarity between voxel models may be determined using, for example, the maximum value of a three-dimensional cross-correlation function or the distance between centers of gravity.

[0029] In addition, although the present embodiment has been described taking an example in which the voxel values ​​are binary information, the voxel values ​​may be multi-valued or continuous. When the voxel values ​​are multi-valued or continuous, the time series processing unit 204 can use a mean filter or a Gaussian filter instead of a majority filter to perform time series processing using halftone information of the voxel values ​​of each voxel that constitutes the voxel model.

[0030] As described above, according to this embodiment, the voxel models to be referenced in the time direction smoothing are narrowed down in advance, which allows the time direction smoothing process to be efficiently executed on the time-series volume data, thereby realizing faster processing.

[0031] [Embodiment 2] Next, an aspect in which the result of collision determination (S405) is saved and the number of times of executing collision determination from the next time onward is reduced to perform smoothing processing in the time direction more efficiently will be described as embodiment 2. Note that description of the contents common to embodiment 1, such as the hardware configuration of the information processing device 100, will be omitted or simplified, and the following description will focus on the differences.

[0032] <Details of time series processing> 7 is a flowchart showing the flow of executing smoothing processing in the time direction on input time-series volume data according to this embodiment. Note that in the following explanation, the symbol "S" represents a step.

[0033] In S701, the data acquisition unit 201 acquires time-series volume data, similar to S401 in the first embodiment. In the following S702, the reference model determination unit 203 initializes a list for saving the results of collision detection (hereinafter referred to as the "determination result list"). Here, the determination result list is array information for reusing the results of collision detection next time (in the smoothing step targeting a newly set frame range).

[0034] S703 to S705 correspond to S402 to S404, respectively, in embodiment 1. That is, the frame range setting unit 202 first sets a frame range to be processed for the time-series volume data (S703). Then, a frame of interest is set from the set frame range (S704), and then a voxel model of interest in the frame of interest is set (S705).

[0035] In S706, the reference model determination unit 203 refers to the determination result list and performs reference determination (here, collision determination) between each voxel model included in other undetermined frames in the set frame range and the voxel model of interest. Assume now that a frame range 801 shown in FIG. 8 is set in S703 for the time-series volume data 500 shown in FIG. 5 described in the first embodiment. At this point, a determination result list such as that shown in FIG. 9(a) has been obtained as information indicating the result of the collision determination in the previous process (the result of the collision determination when processing was performed on the frame range 503 in FIG. 5). Here, "Vol1," "Vol2," and "Vol3" in the determination result list in FIG. 9(a) are labels assigned to each voxel model in the frame. It is not necessary for the same voxel model to have the same label between frames. In the determination result list in FIG. 9(a), "1" indicates that a collision has been determined, and "0" indicates that a collision has not been determined. Since the processing target is currently frame range 801, the frame of interest is frame t=n+1. In this case, collision detection is required between the voxel model of interest in t=n+1 and each voxel model included in the previous frame t=n and each voxel model included in the subsequent frame t=n+2. In this case, in this embodiment, the detection result list shown in FIG. 9(a) is referenced, and collision detection is not performed with each voxel model included in the frame t=n for which detection processing has been completed. In other words, collision detection is only performed with each voxel model included in the frame t=n+2 for which detection processing has not been completed.

[0036] In S707, it is determined whether collision detection between the voxel model of interest and each voxel model included in other frames other than the frame of interest within the set frame range has been completed. If collision detection with all voxel models included in other frames has been completed, the process proceeds to S708, and if not, the process returns to S706 to continue collision detection with the remaining voxel models.

[0037] In S708, the reference model determination unit 203 writes the result of the collision determination in S706 into the determination result list, thereby updating the determination result list. When the processing for the frame range 801 described above is executed, a new determination result list is created, showing the result of collision determination when t=n+1 is the frame of interest, as shown in Fig. 9(b). The shaded portion in the determination result list in Fig. 9(b) indicates that collision determination has already been completed for the frame at t=n.

[0038] In S709, the time series processing unit 204 refers to the determination result list updated in S708, and performs smoothing on the target voxel model using the voxel model determined to collide.

[0039] In S710, it is determined whether the smoothing process has been completed for all voxel models included in the frame of interest. If the smoothing process has been completed for all voxel models, the process proceeds to S710. If not, the process returns to S705 to determine the next voxel model of interest and continue the process.

[0040] In S711, it is determined whether the smoothing process in the time direction for the time-series volume data is complete. If it is complete, this process ends. If it is not complete, the process returns to S703, where the next frame range is determined and the process continues.

[0041] The above is the flow of the smoothing process in the time direction for time-series volume data according to this embodiment.

[0042] As described above, according to this embodiment, the most recent collision determination result is saved and reused, thereby reducing the number of times collision determination is performed, and realizing faster processing.

[0043] [Embodiment 3] Next, a time-series process in which a frame interpolation process is performed to insert another frame between two adjacent frames in a frame group will be described as embodiment 3. Note that the description of the hardware configuration of the information processing device 100 and other aspects common to embodiment 1 will be omitted or simplified, and the following description will focus on the differences.

[0044] <Details of time series processing> 10 is a flowchart showing the flow of frame interpolation processing performed on input time-series volume data according to this embodiment. Note that in the following description, the symbol "S" represents a step.

[0045] S1001 to S1004 correspond to S401 to S404, respectively, in embodiment 1. First, in S1001, the data acquisition unit acquires time-series volume data to be processed.

[0046] In the next step S1002, the frame range setting unit 202 sets a frame range to be processed for the input time-series volume data. In the present embodiment, which performs frame interpolation processing, two adjacent frames (frame t=n-2 and frame t=n-1) 1101 in a frame group are set, as shown in Fig. 11. Note that the frame range to be set (i.e., the position where the frames are to be interpolated) may be stored in advance in the HDD 105 or the like in association with the processing content.

[0047] In S1003, the reference model determination unit 203 sets one of the two frames that make up the frame range set in S1002 as the frame of interest. Here, the frame that occurs earlier in time is set as the frame of interest. In the example of Fig. 11, of the two frames 1101, the frame at t=n-2 is set as the frame of interest.

[0048] In S1004, the reference model determination unit 203 sets a target voxel model to be the target of the next determination process from one or more voxel models included in the target frame. Here, in the example of Fig. 11, the explanation will be continued assuming that the voxel model 1102 is set as the target voxel model from among the three voxel models included in the frame t=n-2 as the target frame.

[0049] In S1005, the reference model determination unit 203 performs reference determination between each voxel model included in a non-target frame (a frame that is later in time) within the set frame range and the target voxel model. In this embodiment, the reference determination is performed by determining the spatial distance and the degree of shape similarity between the target voxel model in the target frame and each voxel model in the next frame. Then, among the voxel models included in the non-target frame, only voxel models that are close in spatial distance and have a high degree of shape similarity are used to create the interpolated voxel model described below (other voxel models are excluded from the creation process). Here, the distance determination may be performed by, for example, calculating the distance between the centers of gravity of both voxel models and determining that the spatial distance is close if the result is equal to or less than a threshold. Furthermore, the shape similarity determination may be performed by, for example, calculating the three-dimensional cross-correlation between both voxel models and determining that the degree of shape similarity is high if the maximum value is equal to or greater than a threshold. In the example of Figure 11, the above two judgment processes are performed between voxel model 1102 (voxel model of interest) and three voxel models 1103 to 1105 in the frame at t = n-1, and it is determined that voxel model 1103 satisfies both conditions.

[0050] In S1006, it is determined whether a complementary frame has already been inserted in the set frame range. If a complementary frame has not been inserted, the process proceeds to S1008; if a complementary frame has already been inserted, the process proceeds to S1009. Then, in S1008, the time series processing unit 204 creates a new frame and inserts it as an interpolation frame between two frames that make up the set frame range. In the example of FIG. 11, a complementary frame corresponding to t=n-1.5 is inserted between the frame at t=n-2 and the frame at t=n-1.

[0051] In S1009, the time series processing unit 204 creates a voxel model that interpolates between the voxel model of interest in the previous frame and the voxel model in the subsequent frame that is determined to satisfy both the distance determination and shape matching conditions in S1005. Specifically, the time series processing unit 204 performs calculations using the following equations (2) and (3) for the two voxel models v1 and v2.

[0052] Here, the timestamp of the frame to which voxel model v1 belongs is t1, the timestamp of the frame to which voxel model v2 belongs is t2, and the timestamp of the frame to which interpolated voxel model belongs is t_int. In this case, the translational movement amount shift_int of the interpolated voxel model from the target voxel model can be calculated using the following equation (2).

[0053] shift_int =(x_shift*r,y_shift*r,z_shift*r) ···(Formula 2) In the above formula (2), x_shift, y_shift, and z_shift represent the translational movement amount from two voxel models v1 to v2, and are calculated from the coordinates that take the maximum value of the three-dimensional cross-correlation between voxel models v1 and v2. The voxel value v_int of each voxel that makes up the interpolated voxel model can be calculated using the following formula (3).

[0054] v_int (x, y, z) = v1(x, y, z) * r + v2(x + x_shift, y + y_shift, z + z_shift) * (1 - r) ...(Formula 3) In the above formula (3), v1(x, y, z) and v2(x, y, z) represent the voxel values ​​at the coordinates (x, y, z) of the voxel models v1 and v2, respectively. Note that r in the above formulas (2) and (3) is a variable expressed by the following formula (4).

[0055] r = (t2 - t_int) / (t2 - t1) ···Equation (4) In the example of Fig. 11, a voxel model 1106 is obtained as an interpolated voxel model that fills the gap between a voxel model 1102 in the frame at t = n-2 and a voxel model 1103 in the frame at t = n-1. The voxel model 1106 is then placed in the inserted interpolated frame at t = n-1.5.

[0056] In S1010, it is determined whether all voxel models included in the frame of interest have been processed. If all voxel models have been processed, the process proceeds to S1011. If there are unprocessed voxel models, the process returns to S1004 to determine the next voxel model of interest and continue processing.

[0057] In S1011, it is determined whether or not the frame interpolation process for the time-series volume data is complete. If it is complete, this process ends. If it is not complete, the process returns to S1002, where the next frame range is determined and the process continues.

[0058] The above is the flow of the frame interpolation process for time-series volume data according to this embodiment.

[0059] According to this embodiment, the voxel models to be referenced in frame interpolation are narrowed down in advance, which allows frame interpolation processing of time-series volume data to be performed efficiently and speeds up processing.

[0060] [Other embodiments] The present disclosure can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions. [Explanation of symbols]

[0061] 100 Information processing device 201 Data Acquisition Department 202 Frame range setting section 203 Reference Model Determination Unit 204 Time Series Processing Unit

Claims

1. an acquisition means for acquiring time-series shape data consisting of a group of temporally consecutive frames, each frame containing shape data indicating a three-dimensional shape of an object; a setting means for setting a frame range from among a group of frames constituting the acquired time-series shape data in accordance with the content of a predetermined time-series process; a processing means for performing the predetermined time series processing on the set frame range; and the processing means performs the predetermined time series processing by excluding shape data that does not satisfy a predetermined condition between the shape data of interest belonging to the frame of interest and the shape data of interest belonging to the frame of interest from the shape data of other frames other than the frame of interest within the set frame range.

1. An information processing device comprising:

2. 2. The information processing device according to claim 1, further comprising a determination means for determining whether or not the shape data belonging to the other frame satisfies the predetermined condition corresponding to the content of the predetermined time-series processing between the shape data belonging to the other frame and the shape data of interest.

3. the predetermined time series processing is a smoothing process in the time direction, the setting means sets two or more consecutive frames including the frame of interest from among a group of frames constituting the time-series shape data as the frame range; The determining means determines whether or not reference is made between shape data included in the other frames other than the frame of interest within the set frame range and the shape data of interest. the processing means performs the smoothing process in the time direction while excluding shape data included in the other frames determined not to be referenced by the determination means.

3. The information processing apparatus according to claim 2, wherein:

4. Further, a storage means for storing information on the result of the reference determination is provided. the determining means performs the reference determination on frames within the set frame range for which the reference determination result does not exist in the information stored by the storing means.

4. The information processing apparatus according to claim 3,

5. 4. The information processing apparatus according to claim 3, wherein the determination means performs collision determination using a bounding box method or a bounding sphere method as the reference determination.

6. The information processing device described in claim 5, further comprising a correction means for leaving only one piece of shape data that has a high degree of shape similarity to the shape data before smoothing if the result of the time-direction smoothing process contains two or more discontinuous shape data within one bounding box.

7. the predetermined time-series processing is a frame interpolation processing, the setting means sets two consecutive frames including the frame of interest from the frame group as the frame range; the determining means performs a distance determination and a shape coincidence determination between the shape data of interest and shape data included in the other frames other than the frame of interest within the set frame range, the processing means performs the frame interpolation process by excluding shape data included in the other frames that are determined by the determination means not to satisfy the conditions regarding distance and shape coincidence.

3. The information processing apparatus according to claim 2, wherein:

8. The determination means In the distance determination, it is determined that the condition is satisfied when the distance between the centers of gravity is equal to or less than a threshold value; In the shape matching determination, it is determined that the condition is satisfied when the maximum value of the three-dimensional cross-correlation is equal to or greater than a threshold value.

8. The information processing apparatus according to claim 7,

9. 9. The information processing apparatus according to claim 1, wherein the shape data is volume data that uses voxels as unit elements representing a shape.

10. 9. The information processing apparatus according to claim 1, wherein the shape data is point cloud data that uses points as unit elements representing a shape.

11. 9. The information processing apparatus according to claim 1, wherein the shape data is mesh data that uses polygons as unit elements representing a shape.

12. an acquisition step of acquiring time-series shape data consisting of a group of temporally consecutive frames, each frame containing shape data indicating a three-dimensional shape of an object; a setting step of setting a frame range from among a group of frames constituting the acquired time-series shape data in accordance with the content of a predetermined time-series process; a processing step of performing the predetermined time series processing on the set frame range; and In the processing step, the predetermined time series processing is performed by excluding shape data that does not satisfy a predetermined condition between the shape data of interest belonging to the frame of interest and the shape data of interest belonging to the frame of interest from the shape data of other frames in the set frame range other than the frame of interest.

1. An information processing method comprising:

13. A program for causing a computer to function as the information processing device according to any one of claims 1 to 11.

Images