Method, apparatus, system and computer program for generating scene change video between locations in space

Abstract

The present invention relates to a method, an apparatus, a system, and a computer program for generating a scene change video between locations in a space and, more specifically, to a method, an apparatus, a system, and a computer program for generating a scene change video between locations in a space, which are capable of generating and providing a realistic scene change video along various paths in a given space. Disclosed in the present invention is a method performed by a device, comprising the steps of: setting a scene change path passing through a first point within an area defined by three or more selected video points in a given space; obtaining video data for the selected video points; and generating a scene change video at the first point according to weights for the selected video points on the basis of relative position of the first point with respect to the selected video points.

Description

Method, device, system and computer program for generating a scene transition video between locations in space

[0001] The present invention relates to a method, device, system and computer program for generating a scene transition image between locations in space, and more particularly, to a method, device, system and computer program for generating a scene transition image between locations in space that can generate and provide realistic scene transition images along various paths within a given space.

[0002] Recently, virtual experience technologies for indoor and outdoor spaces, such as street view services and cyber model house services, are being utilized in various fields, and virtual reality services such as the metaverse are also rapidly expanding.

[0003] In relation to this, a method has been used in the past where 360-degree videos are recorded in advance at multiple locations in indoor and outdoor spaces, a 360-degree video is provided at a location selected by the user, and when the user selects a neighboring location, the scene is switched to the selected location to provide a 360-degree video of that location.

[0004] However, conventional technology has been limited to simply providing scene transition images based on straight-line movement between neighboring locations on a plane, and has been limited to creating and providing scene transition images based on various paths, such as curved movement on a plane or movement in three-dimensional space.

[0005] Furthermore, in order to provide scene transition videos along various paths, there could be a problem of a significant increase in computing resources such as computational volume and memory due to the need to reflect a large amount of image data in the calculation.

[0006] Accordingly, there is a need for a method that can realistically provide scene transition images along a free path in a plane or three-dimensional space in a given space, and furthermore, effectively respond to the problem of rapidly increasing computing resources required for this, but an appropriate solution to this has not yet been implemented.

[0007] The present invention was created to solve the problems of the prior art as described above, and the purpose of the present invention is to provide a method, device, system and computer program for generating a scene transition image between locations in space, which can generate and provide a realistic scene transition image along various paths such as a plane or three-dimensional path within a given space.

[0008] In addition, the present invention aims to provide a method, device, system and computer program for generating a scene transition image between locations in space, which can effectively address the problem of a significant increase in computing resources such as computational amount and memory required while reflecting a large number of image data in calculations to generate a scene transition image along various paths.

[0009] Other detailed purposes of the present invention will be readily apparent and understandable to experts or researchers in this technical field through the specific contents described below.

[0010] According to one aspect of the present invention for solving the above problem, a method is provided, which is performed in a device, comprising: setting a scene transition path passing through a first point within an area defined by three or more selected image points in a given space; acquiring image data for the selected image points; and generating a scene transition image at the first point according to weights for the selected image points based on relative positions of the first point with respect to the selected image points.

[0011] Here, in the step of setting the scene transition path, when the scene transition path and the plurality of image points are located on the same plane, one or more triangular areas formed by connecting some or all of the plurality of image points may be configured to include the scene transition path, and each image point corresponding to a vertex of the one or more triangular areas may be selected.

[0012] In addition, in the step of generating the scene transition image, at the first point of the scene transition path, three image data corresponding to three image points located at each vertex in the first triangular area including the first point are selected, and a scene transition image at the first point can be generated by considering a weight for each image data corresponding to the first point.

[0013] At this time, the weight for each of the image data may be assigned to the image data of each vertex facing the three sub-triangle regions so as to be proportional to the area of ​​the three sub-triangle regions formed by connecting each vertex of the first triangular region and the first point.

[0014] Additionally, at the boundary of the second triangular region adjacent to the first triangular region, a scene transition image generated in the first triangular region and a scene transition image generated in the second triangular region can be generated continuously.

[0015] In addition, if one or more of the three image points located at each vertex at the first point are occluded by an object existing in the space, the weight for the occluded image point can be reduced and applied.

[0016] Furthermore, in the step of setting the scene transition path, if the scene transition path and the plurality of image points are located in a three-dimensional area that is not the same plane, one or more triangular regions formed by connecting some or all of the plurality of image points may be configured to include the scene transition path, and each image point corresponding to a vertex of the one or more triangular regions may be selected.

[0017] In addition, in the step of generating the scene transition image, at the first point of the scene transition path, four image data corresponding to four image points located at each vertex in the first triangular pyramid area including the first point are selected, and a scene transition image at the first point can be generated by considering a weight for each image data corresponding to the first point.

[0018] In addition, the weight for each of the image data may be assigned to the image data of each vertex opposite the four sub-triangular pyramid regions so as to be proportional to the volume of the four sub-triangular pyramid regions formed by connecting each vertex of the first triangular pyramid region and the first point.

[0019] Additionally, at the boundary of the second triangular pyramid region adjacent to the first triangular pyramid region, a scene transition image generated in the first triangular pyramid region and a scene transition image generated in the second triangular pyramid region can be generated continuously.

[0020] In addition, if one or more of the four image points located at each vertex at the first point are occluded by an object existing in the space, the weight for the occluded image point can be reduced and applied.

[0021] Additionally, in the step of calculating the scene transition path, the scene transition path can be calculated based on an image point selected by the user among the plurality of image points.

[0022] Additionally, the scene transition image for the scene transition path can be generated using the barycentric coordinate system.

[0023] In addition, a method according to another aspect of the present invention may include a method performed in a device, the method including: calculating a scene transition path including at least a portion of a path other than a straight line connecting a plurality of image points located in a given space; selecting three or more image points among the plurality of image points based on the scene transition path; and generating a scene transition image for the scene transition path in the space based on three or more image data corresponding to the three or more image points among the respective image data of the plurality of image points.

[0024] In addition, according to another aspect of the present invention, a device including a processor; and a memory, wherein the memory includes instructions configured to cause the device to perform a specific operation when executed by the processor, the specific operation may include: setting a scene transition path passing through a first point within an area defined by three or more selected image points; acquiring image data for the selected image points; and generating a scene transition image at the first point according to a weight for the selected image points based on a relative position of the first point with respect to the selected image points.

[0025] Accordingly, the method, device, system and computer program for generating a scene transition image between locations in space according to one embodiment of the present invention can generate and provide a realistic scene transition image along various paths, such as a planar or three-dimensional path, within a given space.

[0026] In addition, the method, device, system and computer program for generating a scene transition image between locations in space according to one embodiment of the present invention can effectively address the problem of a significant increase in computing resources such as the amount of computation and memory required while reflecting a large number of image data in calculations to generate a scene transition image along various paths.

[0027] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the contents described in this specification.

[0028] The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide examples of the present invention and, together with the detailed description, explain the technical idea of ​​the present invention. The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide examples of the present invention and, together with the detailed description, explain the technical idea of ​​the present invention.

[0029] FIG. 1 is a diagram illustrating a configuration of a scene transition video generation system according to one embodiment of the present invention.

[0030] FIG. 2 is a drawing illustrating a flowchart of a method for generating a scene transition image according to one embodiment of the present invention.

[0031] Figures 3 to 5 are drawings illustrating scene transition images according to one embodiment of the present invention.

[0032] FIGS. 6 to 11 are drawings explaining the generation of a scene transition image according to one embodiment of the present invention.

[0033] FIGS. 12 and 13 are drawings explaining exception processing according to object position during scene transition video generation according to one embodiment of the present invention.

[0034] FIG. 14 is a drawing illustrating the generation of a scene transition image in a three-dimensional path according to one embodiment of the present invention.

[0035] FIG. 15 is a diagram illustrating a flowchart of a method for generating a scene transition image according to one embodiment of the present invention.

[0036] FIG. 16 is a drawing illustrating a specific configuration of a scene transition video generation device according to one embodiment of the present invention.

[0037] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. The purpose, specific advantages, and novel features of the present invention will become more apparent from the following detailed description and preferred embodiments, which are linked to the attached drawings.

[0038] Prior to this, the terms and words used in this specification and claims should be interpreted as meanings and concepts that are consistent with the technical idea of ​​the present invention and are appropriately defined by the inventor to explain his own invention in the best possible way, and are only for the purpose of explaining embodiments and should not be construed as limiting the present invention.

[0039] When assigning reference numerals to components, identical or similar components will be assigned the same reference numerals, regardless of the reference numerals, and any duplicate descriptions thereof will be omitted. The suffixes "module" and "part" used in the following descriptions are assigned or used interchangeably for the convenience of writing specifications, and do not have distinct meanings or roles in themselves, and may refer to either software or hardware components.

[0040] When describing components of the present invention, if a component is expressed in the singular, it should be understood that the component also includes the plural, unless otherwise specified. Furthermore, terms such as "first," "second," etc. are used to distinguish one component from another, and the components are not limited by these terms. Furthermore, when a component is connected to another component, it means that another component may be connected between the component and the other component.

[0041] In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included within the spirit and technical scope of the present invention.

[0042] Hereinafter, exemplary embodiments of a method, device, system and computer program for generating a scene transition image between spatial locations according to the present invention will be described in detail with reference to the attached drawings.

[0043] First, FIG. 1 illustrates the configuration of a scene transition image generation system (100) according to an embodiment of the present invention. As can be seen in FIG. 1, the scene transition image generation system (100) according to an embodiment of the present invention may include a scene transition image generation device (120) that generates a scene transition image from a first location to a second location in a given space based on image data captured in the space, and a terminal (110a, 110b) that receives the generated scene transition image and provides the image to a user.

[0044] In addition, the terminal (110a, 110b) can transmit a location or image point within the space selected by the user to the scene transition image generation device (120) to generate the scene transition image based on this.

[0045] In addition, the terminal (110a, 110b) may be a terminal device such as a personal computer (PC), laptop, tablet, or smartphone that can provide an input device so that a user can receive the scene transition video through a display or the like, or select a location or image point within the space, or a server device, but the present invention is not necessarily limited thereto.

[0046] In addition, the scene transition video generation device (120) may be implemented using one or more servers, but the present invention is not necessarily limited thereto, and may be implemented in various forms, such as being implemented as a dedicated device or being implemented as an integrated unit with the terminal (110a, 110b).

[0047] At this time, the scene transition video generation device (120) may generate and provide a video based on OpenGL or WebGL, but the present invention is not necessarily limited thereto.

[0048] Furthermore, as the communication network (130) connecting the terminals (110a, 110b) and the scene transition video generation device (120) in FIG. 1, a wired network and a wireless network can be used, and specifically, various communication networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN) can be included. In addition, the communication network (130) may also include the well-known World Wide Web (WWW).

[0049] In addition, in the present invention, the scene transition image generation device (120) does not necessarily have to be connected to the terminal (110a, 110b) through the communication network (130) to perform communication, and it is also possible to implement it in various forms, such as directly transmitting and receiving data through an input / output port or a memory card of the scene transition image generation device (120).

[0050] In addition, FIG. 2 illustrates a flowchart for a method for generating a scene transition image according to one embodiment of the present invention.

[0051] The method illustrated in FIG. 2 can be performed, for example, by a scene transition image generation device (120) illustrated in FIG. 1, and further, as can be seen with reference to FIG. 16 and the description of FIG. 16 described below, the scene transition image generation device (120) is a device having a processor (10), in which the processor (10) can generate a scene transition image by executing a command configured to implement a specific operation for generating a scene transition image.

[0052] At this time, as can be seen in FIG. 2, a method for generating a scene transition image according to an embodiment of the present invention may include a step (S110) of setting a scene transition path (220) passing through a first point within an area defined by three or more selected image points (210) in a given space, as a method performed in a device (120), a step (S120) of obtaining image data for the selected image points (210), and a step (S130) of generating a scene transition image at the first point according to a weight for the selected image points (210) based on a relative position of the first point with respect to the selected image points (210).

[0053] Here, in the step (S110) of setting the scene transition path, when the scene transition path (220) and the plurality of image points (210) are located on the same plane, one or more triangular areas (230) formed by connecting some or all of the plurality of image points (210) are configured to include the scene transition path (220), and each image point (210) corresponding to a vertex of the one or more triangular areas (230) can be selected.

[0054] In addition, in the step (S130) of generating the scene transition image, at the first point of the scene transition path (220), three image data corresponding to three image points (210) located at each vertex of the first triangular area (230a) including the first point are selected, and a scene transition image at the first point can be generated by considering the weights for each image data corresponding to the first point.

[0055] At this time, the weight for each of the image data may be assigned to the image data of each vertex facing the three sub-triangle regions (231) so as to be proportional to the area of ​​the three sub-triangle regions (231) formed by connecting each vertex of the first triangle region (230a) and the first point.

[0056] In addition, at the boundary between the first triangular region (230a) and the adjacent second triangular region (230b), a scene transition image generated in the first triangular region (230a) and a scene transition image generated in the second triangular region (230b) can be generated continuously.

[0057] In addition, if one or more of the three image points (210) located at each vertex at the first point are obscured by an object (250) existing in the space, the weight for the obscured image point (210) can be reduced and applied.

[0058] Furthermore, in the step (S110) of setting the scene transition path, if the scene transition path (220) and the plurality of image points (210) are located in a three-dimensional area that is not the same plane, one or more triangular areas (240) formed by connecting some or all of the plurality of image points (210) may be configured to include the scene transition path (220), and each image point (210) corresponding to a vertex of the one or more triangular areas (240) may be selected.

[0059] In addition, in the step (S130) of generating the scene transition image, at the first point of the scene transition path (220), four image data corresponding to four image points (210) located at each vertex of the first triangular pyramid area (240) including the first point are selected, and a scene transition image at the first point can be generated by considering the weight for each image data corresponding to the first point.

[0060] In addition, the weight for each of the image data may be assigned to the image data of each vertex opposite the four sub-triangular pyramid regions so as to be proportional to the volume of the four sub-triangular pyramid regions formed by connecting each vertex of the first triangular pyramid region (240) and the first point.

[0061] In addition, at the boundary between the first triangular pyramid area (240) and the adjacent second triangular pyramid area (240), a scene transition image generated in the first triangular pyramid area (240) and a scene transition image generated in the second triangular pyramid area (240) can be generated continuously.

[0062] In addition, if one or more of the four image points (210) located at each vertex at the first point are covered by an object (250) existing in the space, the weight for the covered image point (210) can be reduced and applied.

[0063] In addition, in the step (S110) of calculating the scene transition path, the scene transition path (220) can be calculated based on an image point (210) selected by the user among the plurality of image points (210).

[0064] Additionally, the scene transition image for the scene transition path (220) can be generated using the barycentric coordinate system.

[0065] Accordingly, the method, device, system, and computer program for generating a scene transition image between locations in space according to one embodiment of the present invention can generate and provide realistic scene transition images along various paths, such as planar or three-dimensional paths, within a given space, and further, can effectively address the problem of a significant increase in computing resources, such as the amount of computation and memory required when reflecting a large amount of image data in calculations to generate scene transition images along various paths.

[0066] Hereinafter, a method for generating a scene transition image according to one embodiment of the present invention will be described in more detail with reference to FIGS. 1 and 2.

[0067] First, in the above S110, the scene transition image generation device (120) sets a scene transition path (220) that passes through a first point within an area defined by three or more selected image points (210) in a given space.

[0068] Here, the space may be an indoor space including a room inside a building, but the present invention is not necessarily limited thereto, and may be an outdoor space or a space in which indoor and outdoor spaces are mixed, and may further include spaces of various environments such as virtual spaces.

[0069] At this time, the image point (210) may be a location where an image such as a 360-degree image or a planar image was captured at one or more locations in the space, but the present invention is not necessarily limited thereto, and in addition, it may be a location corresponding to a viewpoint of an image synthesized for the space, or a location where an image is generated in the space for an image generated in various other ways.

[0070] For example, as can be seen in Fig. 3, when an image such as a 360-degree image or a planar image is captured from multiple image points (210) for a given space, each location where the image is captured can become an image point.

[0071] Accordingly, when a user selects an image point (210) using a terminal (110) (for example, (A) of FIG. 3), the scene transition image generation device (120) can transmit an image captured or generated at the selected image point (210) to the terminal (110) and provide it to the user (for example, (a) of FIG. 5).

[0072] Next, when a user selects another image point (210) using the terminal (110) (e.g., (B) of FIG. 3), the scene transition image generation device (120) can transmit an image captured or generated at the selected other image point (210) to the terminal (110) and provide it to the user (e.g., (d) of FIG. 5).

[0073] At this time, the scene transition video generation device (120) can generate and provide a scene transition video between the image points (210) prior to providing the video for the other image point (210).

[0074] For a more specific example, as can be seen in FIG. 4, in a situation where the scene transition video generation device (120) provides a video for the first video point (210a), when the user selects the second video point (210b), the scene transition video can be generated and provided along the scene transition path (220) between the first video point (210a) and the second video point (210b), and accordingly, the user can be provided with a video that appears to move from the first video point (210a) to the second video point (210b), and can more naturally transition to an video shot or generated at the second video point (210b) (for example, FIG. 5 (a) to (d)).

[0075] For a more specific example, as can be seen in FIG. 4, it is possible for the scene transition path (220) to be configured as a straight path ((p1) of FIG. 4), but the present invention further provides a corresponding scene transition image even if the scene transition path (220) is configured as various paths, such as a curved path ((p2) of FIG. 4).

[0076] In this regard, FIG. 6 illustrates a case where the scene transition image is generated in a manner in which the camera (310) moves along a straight-line scene transition path (220) between the first image point (210a) and the second image point (210b), but the present invention is not limited thereto, and further, in the present invention, the scene transition path (220) can be configured by selecting or combining straight lines or curves so as to include at least a portion of a path other than a straight line connecting a plurality of image points (210) located in space, and accordingly, it becomes possible to freely configure the scene transition path (220) according to space to generate and provide a more realistic scene transition image.

[0077] For a more specific example, as can be seen in (a) of FIG. 7, in the present invention, a curved scene transition path (220) is configured from a first image point (210a) to a second image point (210b), and a scene transition image can be generated and provided based on the curved scene transition path (220).

[0078] Furthermore, as can be seen in FIG. 8, in S110, it is possible to calculate the scene transition path (220) based on an image point (e.g., 210a, 210b of FIG. 8) selected by the user among the plurality of image points (210).

[0079] However, the present invention is not necessarily limited to this, and as can be seen in FIG. 9, it is also possible to generate the scene transition path (210) in various ways, such as calculating a scene transition path (220) that uses a location other than the image point (210) in the space (260a, 260b in FIG. 9) as a starting point or an end point.

[0080] For a more specific example, in the present invention, a user can select one or more of the starting point and the end point of a scene transition path (220) using a terminal (110), and further, it is possible to select or preset the position of a transit point through which the scene transition path (220) passes or the shape of the scene transition path (curve, straight line, or a combination of curves and straight lines, etc.).

[0081] In another embodiment, when a user selects a new location, such as another image point (210), based on the location where the current image is provided, the scene transition image generation device (120) can automatically configure and provide an optimal scene transition path according to a preset scene transition mode, such as a dynamic scene transition mode or a natural scene transition mode.

[0082] In addition, the scene transition video generation device (120) can select three or more image points (210) among the plurality of image points (210) based on the scene transition path (220).

[0083] At this time, as can be seen in (a) of FIG. 7, the scene transition path (220) may be included in one triangular area (230) composed of three image points (210), but as can be seen in FIG. 8, the scene transition path (220) may be positioned across multiple triangular areas (230).

[0084] At this time, the scene transition image generation device (120) can select each image point (210) corresponding to a vertex of one or more triangular areas (230) including the scene transition path (220).

[0085] For a more specific example, in (a) of FIG. 7, three image points (210a, 210b, 210c in (a) of FIG. 7) corresponding to each vertex of a triangular area (230) including the scene transition path (220) can be selected, and in FIG. 8, five image points (210a, 210b, 210c, 210d, 210e in FIG. 8) corresponding to each vertex of three triangular areas (230a, 230b, 230c in FIG. 8) including the scene transition path (220) can be selected.

[0086] In addition, although the above description is made for the case where the scene transition path (220) and the plurality of image points (210) are located on the same plane, the present invention is not necessarily limited thereto, and further, the present invention can be applied to the case where the scene transition path (220) and the plurality of image points (210) are located in a three-dimensional area rather than on the same plane.

[0087] For a more specific example, as can be seen in (a) of FIG. 14, when the scene transition path (220) and the plurality of image points (210) are located in a three-dimensional area rather than the same plane, one or more triangular regions (240) formed by connecting some or all of the plurality of image points (210) can be configured to include the scene transition path (220), and accordingly, in S120, each image point (e.g., 210a, 210b, 210c, 210d in (a) of FIG. 14) corresponding to a vertex of the one or more triangular regions (240) can be selected.

[0088] Hereinafter, as an embodiment of the present invention, a case in which the scene transition path (220) and the plurality of image points (210) are located on the same plane as in the case of FIG. 8 will be described, but the present invention is not limited thereto, and as described above, it is possible to apply the present invention to a case in which the scene transition path (220) and the plurality of image points (210) are located in a three-dimensional area rather than on the same plane, thereby generating and providing a scene transition image.

[0089] Next, in the above S120, the scene transition image generation device (120) acquires image data for the selected image points (210).

[0090] At this time, the scene transition video generation device (120) can obtain video data for the selected video points (210) from a database or memory in which a plurality of video data corresponding to the plurality of video points (210) are stored, but the present invention is not necessarily limited thereto, and it is possible to obtain the video data in various other ways.

[0091] Accordingly, in the S130, the scene transition image generation device (120) generates a scene transition image at the first point based on the relative position of the first point with respect to the selected image points (210) and the weights for the selected image points (210).

[0092] At this time, the scene transition image generation device (120) may generate the scene transition image by blending image data corresponding to each image point (210) based on a fragment shader such as WebGL, but the present invention is not necessarily limited thereto.

[0093] More specifically, as can be seen in FIG. 10, at the first point (260c) of the scene transition path (220), three image data corresponding to three image points (210a, 210b, 210c) located at each vertex of the first triangular area (230a in FIG. 8) including the first point (260c) are selected, and a scene transition image at the first point (260c) can be generated by considering the weight for each image data according to the first point (260c).

[0094] At this time, referring to FIG. 10, each weight for each image data may be allocated to the image data of each vertex facing the three sub-triangle regions (231a, 231b, 231c) so as to be proportional to the area of ​​the three sub-triangle regions (231a, 231b, 231c) formed by connecting each vertex of the first triangle region (230a in FIG. 8) and the first point (260c).

[0095] For a more specific example, at the first point (260c) of FIG. 10, the first weight (w1) for the first image data of the first image point (210a) may be allocated in proportion to the area of ​​the first sub-triangle area (231a) facing the first image point (210a).

[0096] Similarly, at the first point (260c) of FIG. 10, the second weight (w2) for the second image data of the second image point (210b) may be allocated in proportion to the area of ​​the second sub-triangle area (231b) facing the second image point (210b), and the third weight (w3) for the third image data of the third image point (210c) may be allocated in proportion to the area of ​​the third sub-triangle area (231c) facing the third image point (210c) (for example, w1 = 0.39, w2 = 0.28, w3 = 0.33 at the first point (260c) of FIG. 10).

[0097] At this time, as can be seen in (b) of FIG. 7, the scene change image (v) at the first point (260c) can be generated by blending by applying the first weight (w1) to the first image data (v1), applying the second weight (w2) to the second image data (v2), and applying the third weight (w3) to the third image data (v3) (for example, v = v1 * w1 + v2 * w2 + v3 * w3).

[0098] Accordingly, in a situation where the first point (260c) is close to the first image point (210a), the first weight (w1) for the first image data may increase, so that an image close to the first image data may be generated. However, when the first point (260c) moves away from the first image point (210a), the first weight (w1) for the first image data decreases, so that the generated image gradually changes.

[0099] Furthermore, when the first point (260c) moves away from the first image point (210a) and reaches the second point (260d), which is the boundary between the first triangular area (230a in FIG. 8) and the second triangular area (230b in FIG. 8) adjacent to the first triangular area, the scene transition image of the first triangular area (230a in FIG. 8) and the scene transition image of the second triangular area (230b in FIG. 8) can be generated continuously.

[0100] More specifically, at the second point (260d) on the straight line connecting the second image point (210b) and the third image point (210c) in FIG. 10, the first weight (w1) for the first image data may be assigned as 0, and the second weight (w2) for the second image data and the third weight (w3) for the third image data may be assigned based on the distance from the second image point (210b) to the third image point (210c) (for example, w1 = 0, w2 = 0.6, w3 = 0.4 at the second point (260d) in FIG. 10).

[0101] Furthermore, in the case where the scene transition path (220) and the plurality of image points (210) are located in a three-dimensional area rather than the same plane, at the first point of the scene transition path (220), four image data corresponding to four image points (210) located at each vertex of the first triangular pyramid area (240) including the first point are selected, and a scene transition image at the first point can be generated by considering the weights for each image data corresponding to the first point.

[0102] At this time, the weight for each of the image data may be assigned to the image data of each vertex facing the four sub-triangular pyramid areas (240) so as to be proportional to the volume of the four sub-triangular pyramid areas formed by connecting each vertex of the first triangular pyramid area (240) and the first point.

[0103] For a more specific example, at the first point of FIG. 14 (a) (e.g., the position of the camera (310) in FIG. 14 (a)), the first weight (w1) for the first image data of the first image point (210a) may be allocated in proportion to the volume of the first sub-pyramid region facing the first image point (210a), similarly, the second weight (w2) for the second image data of the second image point (210b) may be allocated in proportion to the volume of the second sub-pyramid region facing the second image point (210b), the third weight (w3) for the third image data of the third image point (210c) may be allocated in proportion to the volume of the third sub-pyramid region facing the third image point (210c), and the fourth weight (w4) for the fourth image data of the fourth image point (210d) may be allocated in proportion to the volume of the fourth sub-pyramid region facing the fourth image point (210d). can be allocated in proportion to the volume of the sub-pyramidal region (e.g., w1 = 0.3, w2 = 0.25, w3 = 0.25, w4 = 0.2 at the first point in Fig. 14 (a)).

[0104] At this time, as can be seen in (b) of FIG. 14, the scene change image (v) at the first point (e.g., the position of the camera (310)) can be generated by blending by applying the first weight (w1) to the first image data (v1), applying the second weight (w2) to the second image data (v2), applying the third weight (w3) to the third image data (v3), and applying the fourth weight (w4) to the fourth image data (v4) (e.g., v = v1 * w1 + v2 * w2 + v3 * w3 + v4 * w4).

[0105] Accordingly, in a situation where the first point is close to the first image point (210a), the first weight (w1) for the first image data may increase, so that an image close to the first image data may be generated. However, when the first point moves away from the first image point (210a), the first weight (w1) for the first image data decreases, so that the generated image gradually changes.

[0106] Additionally, in the present invention, at the boundary between the first triangular pyramid region and the adjacent second triangular pyramid region, a scene transition image generated in the first triangular pyramid region and a scene transition image generated in the second triangular pyramid region can be generated continuously.

[0107] More specifically, when the second image point (210b), the third image point (210c) and the fourth image point (210d) are located at the boundary of the second triangular pyramid region adjacent to the first triangular pyramid region, the first weight (w1) for the first image data may be assigned as 0 at the second point on the plane connecting the second image point (210b), the third image point (210c) and the fourth image point (210d), and the second weight (w2) for the second image data, the third weight (w3) for the third image data and the fourth weight (w4) for the fourth image data may be assigned based on the area of ​​the sub-triangular pyramid region facing the second image point (210b), the third image point (210c) and the fourth image point (210d), as described above in FIG. 9 (for example, the second of 10 of FIG. 14 (b)). At point (260d) w1 = 0, w2 = 0.6, w3 = 0.4).

[0108] In addition, in the present invention, the scene transition image for the scene transition path (220) can be generated using a barycentric coordinate, thereby calculating a triangular area (230) or a triangular pyramid area (240) corresponding to the scene transition path (220) in the plane or three-dimensional area, and further calculating a sub-triangular area (231) and a sub-triangular pyramid area in each triangular area (230) or triangular pyramid area (240), thereby calculating a weight for the image data of each image point (210), and accordingly, based on the weight, the scene transition image according to the scene transition path (220) can be efficiently generated using a small amount of computation and memory.

[0109] Furthermore, as can be seen in FIG. 11, in the present invention, when one or more of the three image points (210b, 210d, 210e of FIG. 11) located at each vertex at the first point (260c of FIG. 11) is obscured by an object (250) existing in the space, the weight for the obscured image point (210b of FIG. 11) can be reduced and applied.

[0110] More specifically, referring to FIG. 12, when three image points are not covered at the first point (260c), the scene transition image can be normally generated as in (a) of FIG. 12, but when one or more of the three image points is covered, the image data by the object (250) is affected, and an incorrect scene transition image ((A2) and (A3) of FIG. 12) can be generated as can be seen in (b) and (c) of FIG. 12.

[0111] In this regard, as can be seen in FIG. 13, in the present invention, when one or more of the three image points (210b, 210d, 210e of FIG. 11) located at each vertex at the first point (260c of FIG. 13) is occluded by an object (250) existing in the space, the weight for the occluded image point (210b of FIG. 11) is reduced and applied to 0 or a predetermined minimum value ((B) of FIG. 13), thereby reducing the influence of the image data of the object (250) and improving the quality of the generated scene transition image.

[0112] In addition, FIG. 15 illustrates a flowchart for a method for generating a scene transition image according to one embodiment of the present invention.

[0113] The method illustrated in FIG. 15 can be performed, for example, by a scene transition image generation device (120) illustrated in FIG. 1, and further, as can be seen with reference to FIG. 16 and the description of FIG. 16 described below, the scene transition image generation device (120) is a device having a processor (10), in which the processor (10) can generate a scene transition image by executing a command configured to implement a specific operation for generating a scene transition image.

[0114] At this time, as can be seen in FIG. 15, a method for generating a scene transition image according to an embodiment of the present invention may include a step (S110) of calculating a scene transition path (220) that includes at least a portion of a path other than a straight line connecting a plurality of image points (210) located in a given space, a step (S120) of selecting three or more image points (210) among the plurality of image points (210) based on the scene transition path (220), and a step (S130) of generating a scene transition image for the scene transition path (220) in the space based on three or more image data corresponding to the three or more image points (210) among the respective image data of the plurality of image points (210).

[0115] In addition, the method illustrated in the above-described Figure 15 can apply a specific embodiment of the method for generating a series of scene transition images described above, and a detailed description thereof is omitted.

[0116] In addition, a computer program according to another aspect of the present invention is characterized by being a computer program stored in a computer-readable medium for executing each operation of the scene transition video generation method discussed above on a computer. The computer program may be a computer program including a machine language code created by a compiler, as well as a computer program including a high-level language code that can be executed on a computer using an interpreter, etc. In this case, the computer is not limited to a personal computer (PC) or a laptop computer, and includes any information processing device equipped with a central processing unit (CPU) capable of executing a computer program, such as a server, a smartphone, a tablet PC, a PDA, or a mobile phone.

[0117] Furthermore, a computer-readable medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. Furthermore, the medium may be a variety of recording or storage means in the form of a single or multiple hardware combinations, and is not limited to media directly connected to a computer system, but may also be distributed over a network. Therefore, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

[0118] In addition, FIG. 16 illustrates a device (50) to which the proposed method of the present invention can be applied.

[0119] Referring to FIG. 16, a device (50) may be configured to implement a method for generating a scene transition image according to the proposal of the present invention. For example, the device (50) may be a server device (50) capable of generating a scene transition image for a given space or providing the generated scene transition image.

[0120] For example, the device (50) to which the proposed method of the present invention can be applied may include network devices such as repeaters, hubs, bridges, switches, routers, gateways, etc., computer devices such as desktop computers, workstations, etc., mobile terminals such as smartphones, portable devices such as laptop computers, etc., home appliances such as digital TVs, etc., and moving means such as automobiles, etc. As another example, the device (50) to which the present invention can be applied may be included as a part of an ASIC (Application Specific Integrated Circuit) implemented in the form of a SoC (System On Chip).

[0121] The memory (20) can be connected to the processor (10) when it operates, and can store programs and / or commands for processing and controlling the processor (10), and can store data and information used in the present invention, control information required for data and information processing according to the present invention, temporary data generated during data and information processing, etc. The memory (20) can be implemented as a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, an SRAM (Static RAM), an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.

[0122] The processor (10) may be operatively connected to a memory (20) and / or a network interface (30), and controls the operation of each module within the device (50). In particular, the processor (10) may perform various control functions for performing the proposed method of the present invention. The processor (10) may also be called a controller, a microcontroller, a microprocessor, a microcomputer, etc. The proposed method of the present invention may be implemented by hardware, firmware, software, or a combination thereof. When the present invention is implemented using hardware, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. configured to perform the present invention may be provided in the processor (10). Meanwhile, when the proposed method of the present invention is implemented using firmware or software, the firmware or software may include instructions related to modules, procedures, or functions that perform functions or operations necessary to implement the proposed method of the present invention, and the instructions may be stored in the memory (20) or stored in a computer-readable recording medium (not shown) separate from the memory (20) and, when executed by the processor (10), the device (50) may be configured to implement the proposed method of the present invention.

[0123] In addition, the device (50) may include a network interface device (30). The network interface device (30) is connected to the processor (10) when in operation, and the processor (10) controls the network interface device (30) to transmit or receive wireless / wired signals carrying information and / or data, signals, messages, etc. through a wireless / wired network. The network interface device (30) supports various communication standards, such as, for example, IEEE 802 series, 3GPP LTE(-A), 3GPP 5G, etc., and may transmit and receive control information and / or data signals according to the communication standards. The network interface device (30) may also be implemented outside the device (50) as needed.

[0124] Accordingly, the method, device, system and computer program for generating a scene transition image between locations in space according to one embodiment of the present invention can generate and provide a realistic scene transition image along various paths, such as a planar or three-dimensional path, within a given space.

[0125] In addition, the method, device, system and computer program for generating a scene transition image between locations in space according to one embodiment of the present invention can effectively address the problem of a significant increase in computing resources such as the amount of computation and memory required while reflecting a large number of image data in calculations to generate a scene transition image along various paths.

[0126] The embodiments and drawings described herein are merely exemplary and do not limit the scope of the present invention in any way. In addition, the lines connecting or connecting members between the components depicted in the drawings are merely exemplary of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, if there is no specific mention such as "essential" or "important," the component may not be absolutely necessary for the application of the present invention.

[0127] The use of the term "above" and similar referential terms in the specification of the present invention (especially in the claims) may refer to both singular and plural. In addition, when a range is described in the present invention, it includes inventions that apply individual values ​​belonging to the range (unless stated to the contrary), and it is the same as describing each individual value constituting the range in the detailed description of the invention. In addition, the steps presented in the method inventions of the present invention are not necessarily intended to be bound by the order of their chronological order, and the order may be appropriately changed as needed, unless a certain step must come first depending on the nature of each process. The use of all examples or exemplary terms (e.g., "for example," etc.) in the present invention is merely to describe the present invention in detail, and the scope of the present invention is not limited by the examples or exemplary terms, unless limited by the claims. In addition, those skilled in the art will understand that various modifications, combinations, and variations can be configured according to design conditions and elements within the scope of the appended claims or their equivalents.

Claims

1. A method performed in a device, A step of setting a scene transition path passing through a first point within an area defined by three or more selected image points in a given space; A step of acquiring image data for the selected image points; and A method comprising the step of generating a scene transition image at the first point according to a weight for the selected image points based on the relative position of the first point with respect to the selected image points.

2. In claim 1, In the step of setting the above scene transition path, If the above scene transition path and the above multiple image points are located on the same plane, One or more triangular areas formed by connecting some or all of the plurality of image points are configured to include the scene transition path, A method for selecting each image point corresponding to a vertex of one or more of the above triangular regions.

3. In claim 2, In the step of generating the above scene transition video, At the first point of the above scene transition path, Select three image data corresponding to three image points located at each vertex in the first triangle area including the first point, A method for generating a scene transition image at the first point by considering weights for each image data corresponding to the first point.

4. In claim 3, The weights for each of the above image data are: In proportion to the area of ​​the three sub-triangle regions formed by connecting each vertex of the first triangular region and the first point, A method in which image data is allocated to each vertex facing the three sub-triangle areas above.

5. In claim 3, At the boundary of the second triangular region adjacent to the first triangular region, A method in which a scene transition image generated in the first triangular area and a scene transition image generated in the second triangular area are generated continuously.

6. In claim 3, If one or more of the three image points located at each vertex at the first point are obscured by an object existing in the space, A method of applying a weight reduction to the above-mentioned hidden image points.

7. In claim 1, In the step of setting the above scene transition path, If the above scene transition path and the above multiple image points are located in a three-dimensional area that is not on the same plane, One or more triangular regions formed by connecting some or all of the plurality of image points are configured to include the scene transition path, A method for selecting each image point corresponding to a vertex of one or more triangular pyramidal regions.

8. In claim 7, In the step of generating the above scene transition video, At the first point of the above scene transition path, Select four image data corresponding to four image points located at each vertex in the first triangular pyramid area including the first point, A method for generating a scene transition image at the first point by considering weights for each image data corresponding to the first point.

9. In claim 8, The weights for each of the above image data are: In proportion to the volume of the four sub-triangular pyramidal regions formed by connecting each vertex of the first triangular pyramidal region and the first point, A method in which image data is allocated to each vertex facing the above four sub-triangular pyramidal regions.

10. In claim 8, At the boundary of the second triangular pyramid region adjacent to the first triangular pyramid region, A method in which a scene transition image generated in the first triangular pyramid area and a scene transition image generated in the second triangular pyramid area are generated continuously.

11. In claim 8, If one or more of the four image points located at each vertex at the first point are obscured by an object existing in the space, A method of applying a weight reduction to the above-mentioned hidden image points.

12. In claim 1, In the step of calculating the above scene transition path, A method for calculating the scene transition path based on an image point selected by the user from among the plurality of image points.

13. In claim 1, A method for generating a scene transition image for the scene transition path using a barycentric coordinate system.

14. A method performed in a device, A step of generating a scene transition path that includes at least some paths other than straight lines connecting a plurality of image points located in a given space; A step of selecting three or more image points among the plurality of image points based on the scene transition path; and A method comprising the step of generating a scene transition image for the scene transition path in the space based on three or more image data corresponding to the three or more image points among the image data of each of the plurality of image points.

15. A device including a processor and a memory, The memory comprises instructions configured to cause the device to perform a specific operation when executed by the processor, the specific operation comprising: Setting a scene transition path passing through a first point within an area defined by three or more selected image points; Obtaining image data for the selected image points; and A device comprising generating a scene transition image at the first point based on a weight for the selected image points based on the relative position of the first point with respect to the selected image points.

Images