Information processing system and information processing method

Abstract

This enables comprehensive support for asset creation and utilization. [Solution] An information processing system comprising: a generation support unit having a plurality of different generation units for generating assets; and a utilization support unit having a plurality of different utilization units for utilizing the assets generated by the generation support unit, wherein the assets used in the utilization units are assets generated by any of the generation units and corresponding to the utilization units.

Description

Technical Field

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, it has become active to generate 3D assets from a plurality of 2D image groups and use the generated 3D assets for the production of various contents.

[0003] For example, in Patent Document 1 below, according to an operation on a 3D stroboscopic image seen from a virtual viewpoint of a stroboscopic model in which 3D models of objects at a plurality of times generated from a plurality of viewpoint images captured from a plurality of viewpoints are arranged in a three-dimensional space, a technique for playing back free viewpoint data generated from the plurality of viewpoint images is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, the production of assets such as 3D assets and the production of various contents using the assets are each performed by independent applications, and the user has to grasp and prepare assets suitable for the content to be produced. Since these applications are not linked, it has been a problem of high working cost and inefficient content production.

[0006] Therefore, the present disclosure proposes an information processing system, an information processing method, and a program that can comprehensively support the generation and utilization of assets.

Means for Solving the Problems

[0007] According to this disclosure, an information processing system is provided, comprising: a generation support unit having a plurality of different generation units for generating assets; and a utilization support unit having a plurality of different utilization units for utilizing the assets generated by the generation support unit, wherein the assets used in the utilization units are assets generated by any of the generation units and corresponding to the utilization units.

[0008] Furthermore, the present disclosure provides an information processing method comprising: a processor generating an asset using at least one of a plurality of different generation units; and utilizing the asset generated by the generation unit using at least one of a plurality of different utilization units, wherein the asset used in the utilization unit is an asset generated by any of the generation units and corresponding to the utilization unit.

[0009] Furthermore, according to this disclosure, a program is provided for causing a computer to function as: a generation support unit having a plurality of different generation units for generating assets; a utilization support unit having a plurality of different utilization units for utilizing the assets generated by the generation support unit, wherein the assets used in the utilization units are assets generated by any of the generation units and corresponding to the utilization units, and a communication unit that communicates with a system; and a control unit that controls the generation units to transmit sensing data used for generating the assets to the system, and acquires information from the system regarding the asset generation results by the generation units and the utilization results by the utilization units. [Brief explanation of the drawing]

[0010] [Figure 1] This figure illustrates an overview of a production support system 30 (an example of an information processing system) according to one embodiment of the present disclosure. [Figure 2] This block diagram shows an example of the configuration of the capture device 10 according to this embodiment. [Figure 3] This block diagram shows an example of the configuration of the output device 20 according to this embodiment. [Figure 4]This is a block diagram showing an example of the functional configuration of the production support system 30 according to this embodiment. [Figure 5] This figure shows an example of the functional configuration of the asset generation support unit 322 according to this embodiment. [Figure 6] This figure shows an example of the generation unit included in the 3D asset generation support unit 322A according to this embodiment. [Figure 7] This figure shows an example of a generation unit included in the content generation unit 324 according to this embodiment. [Figure 8] This diagram illustrates the overall processing flow according to this embodiment. [Figure 9] This figure shows an example of the image processing flow according to this embodiment. [Figure 10] This figure shows an example of the production support processing flow according to this embodiment. [Figure 11] This figure shows an example of the application selection screen according to this embodiment. [Figure 12] This figure shows another example of the application selection screen according to this embodiment. [Figure 13] This figure shows another example of the application selection screen according to this embodiment. [Figure 14] This figure shows another example of the application selection screen according to this embodiment. [Figure 15] This figure shows an example of a size selection screen for a 3D asset generated according to this embodiment. [Figure 16] This figure shows an example of a selection screen for the generation algorithm used to generate 3D assets according to this embodiment. [Figure 17] This figure shows an example of a selection screen for the output format of a 3D asset according to this embodiment. [Figure 18] This figure shows an example of a display screen that includes applications where asset generation according to this embodiment is subject to AI utilization or paid billing. [Figure 19] This is an example of a warning screen that appears when an option marked with the "Generating AI" icon is selected on the application selection screen according to this embodiment. [Figure 20]It is a diagram showing an example of a usage selection screen presented after asset generation according to this embodiment. [Figure 21] It is a diagram showing an example of the flow of production support processing according to a modification of this embodiment. [Figure 22] It is a block diagram showing an example of the hardware configuration of an information processing apparatus 900 according to an embodiment of the present disclosure.

Modes for Carrying Out the Invention

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0012] Also, the description will be made in the following order. 1. Overview 2. Configuration 2-1. Configuration of the capture device 10 2-2. Configuration of the output device 20 2-3. Configuration of the production support system 30 3. Operation processing 4. UI example 4-1. Usage selection screen 4-2. Other selection screens 4-3. Regarding AI utilization and paid charging 5. Modification example 6. Hardware configuration 7. Supplementary

[0013] <1. Overview> FIG. 1 is a diagram for explaining the overview of a production support system 30 (an example of an information processing system) according to an embodiment of the present disclosure. As shown in FIG. 1, the production support system 30 is communicatively connected to a capture device 10, an output device 20, an external cloud service 40, and an external plugin application server 50. Note that the configuration shown in FIG. 1 is an example, and the present embodiment is not limited thereto.

[0014] The production support system 30 is a system that comprehensively supports the generation and utilization of assets. The production support system 30 may be formed from one or more servers. Each function of the production support system 30 may be realized by various applications and cloud services. Each application and cloud service may cooperate as appropriate to enable processing in the preceding stage according to the planned processing of subsequent stages, and processing in the subsequent stages according to the results of the processing of preceding stages.

[0015] The production support system 30 is a system that can handle everything from asset generation to utilization. This allows for more efficient content creation by utilizing information from both the pre-processing (e.g., asset generation) and post-processing (e.g., content generation) stages. For example, it can generate appropriate assets according to the intended use of the content (or assets) to be produced, suggest uses based on the generated assets, and automatically select assets to be used according to the intended purpose.

[0016] Examples of its applications (purposes) include virtual production, film production, film prop production, game background production, game prop production, VR game production, metaverse production, digital twin world production, digital twin world prop production, Previz (Pre-visualization) production, Previz prop production, 3D commerce production, 3D exhibitions, 3D digital signage, advertising production, corporate videos, and digital archives.

[0017] Utilization may include, for example, the generation of content or the output of the generated content. Examples of content output include sending the content to the output device 20, uploading the content to an external cloud service, or distributing the content via an external distribution platform.

[0018] The capture device 10 is a device that acquires sensing data of an object. For example, the capture device 10 may include an imaging device that captures images of the object, a measuring device that measures the distance to the object and the shape of the object, a motion capture device that tracks the movement of the object, and a sound collection device that acquires sound from the object, its surroundings, and the surrounding space. The capture device 10 may be composed of multiple devices. Furthermore, the capture device 10 may be a dedicated device (for example, a mirrorless camera, a ToF (Time Of Flight) camera, a 3D laser scanner, a LiDAR (Light Detection and Ranging) device, a recording device, etc.), or it may be implemented using a general-purpose device such as a smartphone, smartwatch, or tablet terminal.

[0019] The sensing data of the object acquired by the capture device 10 is input to the production support system 30. The sensing data of the object may be uploaded from the capture device 10 to the production support system 30 via a network, or it may be transmitted to the production support system 30 via a wired connection or other device.

[0020] Output device 20 is an example of an information processing terminal that outputs various types of information. Output device 20 can be implemented as, for example, a smartphone, smartwatch, tablet, PC (Personal Computer), HMD (Head Mounted Display), stereoscopic display device, large display device, projector, etc. Output device 20 also has an operation input function. Output device 20 may be used as a user interface when generating and utilizing assets using the production support system 30. Alternatively, output device 20 may be used as a playback device that plays back content generated by the production support system 30.

[0021] The external cloud service 40 can provide various services via the internet. The external cloud service 40 may be implemented by a server or a system consisting of multiple servers. The production support system 30 can communicate with the external cloud service 40 and use the cloud service as appropriate when generating assets or content.

[0022] The external plugin application server 50 can provide applications for plugins via the internet. The external plugin application server 50 may be implemented as a system consisting of multiple servers. The production support system 30 can, for example, acquire applications as needed from the external plugin application server 50 and use the external applications for asset generation and content generation. This allows the production support system 30 to be functionally expanded.

[0023] <2. Structure> <<2-1. Configuration of Capture Device 10>> Figure 2 is a block diagram showing an example of the configuration of the capture device 10 according to this embodiment. As shown in Figure 2, the capture device 10 includes a communication unit 110, a control unit 120, a sensor unit 130, an operation display unit 140, and a storage unit 150.

[0024] (Communications Department 110) The communication unit 110 includes a transmitting unit that transmits data to an external device and a receiving unit that receives data from the external device. The communication unit 110 according to this embodiment communicates with an external device or the Internet using, for example, a wired / wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), a mobile communication network, etc.

[0025] (Sensor unit 130) The sensor unit 130 is an acquisition unit that acquires sensing data of an object. The sensor unit 130 may have multiple sensors. For example, the sensor unit 130 may include an imaging unit. The imaging unit has one or more lenses (optical system) and an image sensor consisting of a CCD image sensor or a CMOS image sensor, and captures an image of an object according to the control of the control unit 120 and outputs an captured image (RGB image).

[0026] Furthermore, the sensor unit 130 may include a ToF camera. The ToF camera measures the distance to an object and outputs a depth image that visualizes the distance information. The ToF camera is just one example of a distance sensor, and this disclosure is not limited thereto.

[0027] Furthermore, the sensor unit 130 may include a position and orientation sensor for detecting the position and orientation of the capture device 10. Specifically, the sensor unit 130 may include a position sensor, an IMU (Inertial Measurement Unit), and a geomagnetic sensor. The position sensor calculates absolute or relative position information of the capture device 10. The sensor unit 130 outputs the three-dimensional position and orientation information of the capture device 10 to the control unit 120.

[0028] (Control unit 120) The control unit 120 functions as an arithmetic processing unit and control unit, and controls the overall operation within the capture device 10 according to various programs. The control unit 120 is implemented by electronic circuits such as a CPU (Central Processing Unit) or a microprocessor. The control unit 120 may also include a ROM (Read Only Memory) for storing programs and calculation parameters to be used, and a RAM (Random Access Memory) for temporarily storing parameters that change as needed.

[0029] The control unit 120 according to this embodiment can function as a production support AP (application) execution unit 122.

[0030] The production support AP execution unit 122 executes the production support application installed on the capture device 10. The capture device 10 acquires sensing data used to generate assets used in content creation, but the appropriate accuracy, quantity, and method of acquiring sensing data vary depending on the asset to be generated. The production support AP execution unit 122 displays a usage selection screen on the operation display unit 140 for the user to select how the content will ultimately be used, and acquires usage information. The production support AP execution unit 122 also controls the acquisition of sensing data used to generate assets corresponding to the usage information (i.e., assets used for utilization according to the usage).

[0031] For example, when the production support AP execution unit 122 acquires sensing data (captured images) using the imaging unit, it may perform imaging support such as setting camera parameters (aperture, exposure, white balance, focal length, etc.), shutter control, and navigation of the imaging position and orientation (guidance on how far and in what direction to shoot relative to the object) so as to appropriately acquire data to be used to generate assets corresponding to the application information. The production support AP execution unit 122 is an example of an imaging support unit. The production support AP execution unit 122 automatically sets camera parameters, etc., according to the size of the asset to be generated, the required quality (resolution), etc., such as landscapes, structures, indoor spaces, automobiles, and small objects. The production support AP execution unit 122 also displays navigation of the imaging position and orientation on the operation display unit 140. Note that shutter control may be automatically controlled by the production support AP execution unit 122, or the user may operate it manually by referring to the navigation.

[0032] Furthermore, the production support AP execution unit 122 saves imaging information (e.g., position and orientation information (IMU output data, etc.)), camera metadata (e.g., F-number, focal length, exposure value, image stabilization information, subject distance), lens metadata (e.g., distortion correction, chromatic aberration correction, monochromatic aberration correction information) as supplementary data. This supplementary data is output to the production support system 30 along with the captured image. It is also conceivable that an additional device such as a smartphone may be attached to the capture device 10, and information acquired by such an additional device (e.g., depth data, illuminance data, etc.) may be transmitted to the capture device 10. The capture device 10 may also save the information acquired by the additional device as camera parameters and output it to the production support system 30 along with the captured image. In addition, the production support AP execution unit 122 may select the data to acquire, save, and output to the production support system 30 according to the application information. In other words, the production support AP execution unit 122 can reduce the power consumption, processing load, and data capacity of the capture device 10 by acquiring, saving, and outputting data necessary for subsequent processing (in this case, asset generation or content generation). Note that the accompanying data is not limited to the examples described above, and could also include, for example, advanced information of the capture device 10, recognition results of the object (subject), and material information of the object.

[0033] Furthermore, the production support AP execution unit 122 may transmit usage information along with the captured image group and associated data to the production support system 30.

[0034] (Operation display section 140) The operation display unit 140 has the function of an operation unit that accepts user input and the function of a display unit that displays various screens. For example, the operation display unit 140 may be implemented by a touch panel display. Alternatively, the operation unit function and the display unit function may be implemented separately, or an operation unit or display unit may be provided separately from the operation display unit 140. The operation unit may be implemented by, for example, a touch sensor, a switch, a button, etc. The display unit may be implemented by, for example, a display panel such as a liquid crystal display (LCD) or an organic EL (electroluminescence) display.

[0035] (Storage unit 150) The storage unit 150 is implemented by a storage medium that stores programs used in the processing of the control unit 120, calculation parameters, parameters that change as needed, etc. In this embodiment, the storage unit 150 may store, for example, captured images and associated data.

[0036] The configuration of the capture device 10 has been described in detail above, but the configuration of the capture device 10 according to this disclosure is not limited to the example shown in Figure 2. For example, the capture device 10 does not have to have an operation display unit 140, and it may also have other sensors such as an illuminance sensor.

[0037] Furthermore, an information processing terminal such as a smartphone (hereinafter also referred to as a user terminal) that communicates wirelessly or via a wired connection with the capture device 10 may be attached to the capture device 10 (fixed by a mounting device), and the production support AP execution unit 122 may be provided on such user terminal. In this case, the application selection screen, imaging support navigation, etc., may be displayed on the display unit of the user terminal.

[0038] The production support AP execution unit 122 installed in the user terminal may set camera parameters and control the shutter of the connected capture device 10 according to the usage information obtained from user operations on the usage selection screen. The user terminal may also acquire captured images and associated data from the capture device 10 and transmit (upload) them to the production support system 30. In this case, the user terminal may also transmit associated data obtained at the time of imaging (absolute position information from GPS, etc., altitude information, depth data, object recognition results, etc.) obtained by the sensor unit installed in the user terminal as camera parameters to the production support system 30.

[0039] In the example described above, it was stated that the capture device 10 (or user terminal) is equipped with a production support AP execution unit 122. However, in this embodiment, the capture device 10 (or user terminal) does not necessarily have to be equipped with a production support AP execution unit 122. Camera parameters and shutter control may be set manually by the user as usual, and the obtained captured image may be output to the production support system 30. In addition, all imaging-related data that can be acquired by the capture device 10 may be output to the production support system 30 along with the captured image as supplementary data.

[0040] <<2-2. Configuration of Output Device 20>> Figure 3 is a block diagram showing an example of the configuration of the output device 20 according to this embodiment. As shown in Figure 3, the output device 20 includes a communication unit 210, a control unit 220, an operation display unit 230, and a storage unit 240.

[0041] (Communications Department 210) The communication unit 210 includes a transmitting unit that transmits data to an external device and a receiving unit that receives data from the external device. The communication unit 210 according to this embodiment may communicate with an external device or the Internet using, for example, a wired or wireless LAN, Wi-Fi, Bluetooth, or a mobile communication network.

[0042] (Control unit 220) The control unit 220 functions as an arithmetic processing unit and control unit, and controls the overall operation of the output device 20 according to various programs. The control unit 220 is implemented by electronic circuits such as a CPU or microprocessor. The control unit 220 may also include a ROM for storing the programs and calculation parameters to be used, and a RAM for temporarily storing parameters that change as needed.

[0043] Furthermore, the control unit 220 also functions as a production support AP execution unit 222. The production support AP execution unit 222 executes production support applications installed on the output device 20. Specifically, the production support AP execution unit 222 displays a usage selection screen on the operation display unit 230 for the user to select how the content will ultimately be used, and acquires usage information. The production support AP execution unit 222 then transmits the usage information to the production support system 30 and can cooperate with the production support system 30 to generate assets and generate content using the generated assets. In cooperation with the production support system 30, the output device 20 is mainly used as the user interface for the production support system 30. The production support AP execution unit 222 may also upload sensing data (such as a group of captured images acquired by the capture device 10) used for asset generation to the production support system 30.

[0044] It should also be noted that the application information may already be selected at the time of sensing data acquisition. The application information may be shared with the output device 20 via the production support system 30.

[0045] Furthermore, the production support AP execution unit 222 may also have the functions of the production support AP execution unit 122 described above. For example, if the output device 20 is implemented by a smartphone or the like, the output device 20 may communicate with the capture device 10 and perform camera parameter settings and shutter control of the capture device 10 according to the usage information.

[0046] Furthermore, the production support AP execution unit 222 may control the display unit 230 to show the assets and content generated by the production support system 30. While it is assumed here that the assets and content generated are visual information, this is not limited to visual information. For example, in the case of audio information, the production support AP execution unit 222 may output the audio information (such as stereophonic sound) generated as assets or content from an audio output unit (not shown). The generated content may also be video with audio.

[0047] (Operation display section 230) The operation display unit 230 has the function of an operation unit that accepts user input and the function of a display unit that displays various screens. For example, the operation display unit 230 may be implemented by a touch panel display. Alternatively, the operation unit function and the display unit function may be implemented separately, or an operation unit or display unit may be provided separately from the operation display unit 230. The operation unit may be implemented by, for example, a touch sensor, a switch, a button, etc. The display unit may be implemented by, for example, a display panel such as a liquid crystal display (LCD) or an organic EL (electroluminescence) display.

[0048] (Storage unit 240) The storage unit 240 is implemented by a storage medium that stores programs used in the processing of the control unit 220, calculation parameters, parameters that change as needed, etc. In this embodiment, the storage unit 240 may store, for example, a group of captured images, accompanying data, usage information, assets, and content.

[0049] Although the configuration of the output device 20 has been described in detail above, the configuration of the output device 20 according to this disclosure is not limited to the example shown in Figure 3. For example, the output device 20 does not necessarily have all the configurations shown in Figure 3. Also, the output device 20 may be implemented by multiple devices. Furthermore, at least some of the functions of the output device 20 may be provided on a server on the network. For example, the output device 20 may further have an audio input unit, an audio output unit, a sensor unit, etc.

[0050] <<2-3. Configuration of Production Support System 30>> Figure 4 is a block diagram showing an example of the functional configuration of the production support system 30 according to this embodiment. The production support system 30 may be implemented by one or more servers, by an information processing terminal such as a smartphone, tablet terminal, or PC, or by a user terminal such as a smartphone (an example of an information processing terminal) and a server. Furthermore, the production support system 30 may be implemented by one or more servers and the output device 20 described above (an example of an information processing terminal).

[0051] As shown in Figure 4, the production support system 30 has the functions of an image capture management unit 321, an asset generation support unit 322, a utilization support unit 323, a plug-in application acquisition unit 326, an account management unit 327, a license granting unit 328, and a sales management unit 329. Each of these functions may be implemented by an application or cloud service. Furthermore, a single function is not necessarily implemented by a single application or cloud service; each function can be implemented by various combinations, such as a group of applications, a combination of an application and a cloud service, or multiple cloud services.

[0052] (Image management unit 321) The image management unit 321 acquires captured images and manages them, including saving them. The captured images are uploaded to the production support system 30 from, for example, the capture device 10 or a user terminal such as a smartphone used by the user (it may also be an output device 20). Various supplementary data, such as camera parameters at the time of capture, may be attached to the captured images. The image management unit 321 is an example of an acquisition unit that acquires sensing data.

[0053] (Asset Generation Support Unit 322) The asset generation support unit 322 assists in generating assets according to usage information based on sensing data. Usage information can be obtained from a user terminal (which may also be an output device 20). The asset generation support unit 322 may also display a usage selection screen on the user terminal and obtain information on user operations on the usage selection screen.

[0054] Various types of assets can be generated by the asset generation support unit 322. Figure 5 shows an example of the functional configuration of the asset generation support unit 322 according to this embodiment. As shown in Figure 5, the asset generation support unit 322 includes, as an example, a 3D asset generation support unit 322A, a human asset generation support unit 322B, a motion asset generation support unit 322C, a voice asset generation support unit 322D, a 2D-3D video conversion support unit 322E, and other asset generation support units 322F.

[0055] The 3D asset generation support unit 322A generates 3D assets, which are 3DCG model data of various objects (landscapes, large structures, buildings, indoor spaces, vehicles, furniture, clothing, accessories, food, plants, animals, etc.). The human asset generation support unit 322B generates human assets, which are 3DCG model data specifically for people, and are equipped with bone data.

[0056] The motion asset generation support unit 322C creates assets such as the movements of people, animals, moving objects, and facial movements (generates 3D motion). The audio asset generation support unit 322D generates sound effects, object audio, and sound sources for spatial content. The audio asset generation support unit 322D can also generate spatial audio (3D audio) as an audio asset.

[0057] The 2D-3D video conversion support unit 322E can convert 2D videos into 3D videos and generate 3D video assets. The 2D videos may be generated, for example, by the 2D video generation unit 324C described later. By converting them into 3D videos, they can be used as assets that can replace the background of spatial content. The other asset generation support unit 322F generates various other assets. The 2D-3D video conversion support unit 322E can also convert 2D videos into 3D videos. In addition, although the conversion target is video here, it is not limited to this, and the asset generation support unit 322 may also have a 2D-3D still image conversion support unit that converts 2D still images into 3D still images, and 3D still images into 2D still images.

[0058] Each generation support unit can generate assets according to application information based on sensing data. Furthermore, the functions of each generation support unit can be implemented using one or more applications or, as appropriate, cloud services.

[0059] The asset generation support unit 322 determines what kind of asset to generate based on usage information indicating the intended use desired by the user, and controls the generation of the asset by the corresponding generation support unit. Depending on the intended use, the conditions for asset generation may differ, such as desired quality (resolution, frame rate, sound quality, etc.), data volume, size, editability, and AI tolerance. If the use is for distribution, the distribution conditions (data volume, etc.) stipulated by the distribution platform (distribution service) may also be considered when determining what kind of asset to generate. Distribution conditions may also differ depending on the time, such as when an event is held. In addition, the asset generation support unit 322 may, as appropriate, propose asset types etc. according to the use to the user and allow the user to select, or introduce asset types etc. according to the use to the user. For example, the asset generation support unit 322 may clearly indicate to the user what kind of asset to generate for the use selected by the user, for example, by displaying a sample 3D asset. The asset generation support unit 322 may also clearly indicate to the user information regarding the data necessary for generating such assets. Furthermore, the asset generation support unit 322 may clearly indicate to the user whether or not it is possible to generate an asset corresponding to the intended use from the uploaded data (e.g., captured images). Alternatively, the asset generation support unit 322 may select data from the uploaded data that can be used to generate an asset corresponding to the intended use and allow the user to choose which data to use. Additionally, if there is insufficient data to generate an asset corresponding to the intended use, the asset generation support unit 322 may clearly indicate the missing data to the user and prompt them to add more data.

[0060] Each of the aforementioned generation support units has multiple generation units (which can also be called generation functions or generation applications) that use different generation methods to generate assets. Even when generating the same type of asset (for example, a 3D asset), there is an appropriate generation method depending on the application, and the asset generation support unit 322 controls the generation of the appropriate asset using the appropriate generation unit (appropriate generation method) based on the application information.

[0061] The generation method, more specifically, refers to the generation algorithm. Differences in generation algorithms result in variations in, for example, the image quality type and generation speed of the generated assets. Furthermore, the appropriate generation algorithm differs depending on the size, image quality type, and editability (whether or not the asset should be editable) of the asset to be generated. Generation algorithms that utilize generation AI are also envisioned. While it is possible for users to manually select a generation algorithm, the asset generation support unit 322 can automatically determine the appropriate generation algorithm based on the usage information, allowing even users unfamiliar with production to obtain suitable assets.

[0062] The following will specifically describe several different generation units, using the 3D asset generation support unit 322A as an example. Figure 6 shows an example of the generation units included in the 3D asset generation support unit 322A according to this embodiment. As shown in Figure 6, the 3D asset generation support unit 322A includes a first 3D asset generation unit 322A-1, a second 3D asset generation unit 322A-2, and a third 3D asset generation unit 322A-3. The 3D asset generation methods used in each generation unit, that is, the generation algorithms used for 3D asset generation, are all different.

[0063] For example, the first generation algorithm used in the 3D asset generation support unit 322A is an algorithm that generates ultra-high-resolution 3D assets with the highest possible image quality (though the generation speed is slow). It is also envisioned that the 3D assets generated by the first generation algorithm may be output as content without editing (for example, uploaded to a metaverse production server).

[0064] The second generation algorithm used in the second 3D asset generation unit 322A-2 is an algorithm that generates 3D assets with medium image quality (and medium generation speed). The second generation algorithm may also perform image uniformity correction (noise reduction, brightness uniformity, etc.) to generate 3D assets with flat image quality. Furthermore, the second generation algorithm may also generate editable 3D assets. There may be multiple patterns for the second generation algorithm.

[0065] The third generation algorithm used in the third 3D asset generation unit 322A-3 prioritizes generation speed and is faster than the first and second generation algorithms in generating 3D assets.

[0066] The 3D asset generation support unit 322A generates 3D assets using a first generation algorithm, for example, when the application is background creation for virtual production. Furthermore, the 3D asset generation support unit 322A generates 3D assets using a second generation algorithm, for example, when the application is film production, film prop creation, game background creation, game prop creation, VR game production, or metaverse production. Note that multiple second generation algorithms may be provided, differing in the size, image quality level, editability, and AI tolerance of the 3D assets to be generated, and the 3D asset generation support unit 322A may select the appropriate second generation algorithm according to the application.

[0067] Furthermore, the 3D asset generation support unit 322A generates 3D assets using a third generation algorithm, for example, when the application is digital twin world creation, digital twin world prop creation, Previz creation, or Previz prop creation.

[0068] Furthermore, the 3D asset generation support unit 322A may, depending on the application, select an appropriate generation algorithm from the first to third generation algorithms, such as 3D commerce production, 3D exhibitions, 3D digital signage, advertising production, corporate videos, and digital archives.

[0069] For example, the 3D asset generation support unit 322A may select an appropriate generation algorithm based on the object recognition results (landscape, building, car, small object, etc.) of the captured image used for generation.

[0070] Furthermore, the 3D asset generation support unit 322A may select an appropriate generation algorithm based on the camera position and orientation information of the captured images used for generation, and the position and orientation information of the smartphone that performed the shooting navigation while attached to the camera. The 3D asset generation support unit 322A can determine how the user took the photos from the camera position and orientation information of each image in the group of captured images, and can also estimate the size of the object from the shooting method, thereby selecting an appropriate generation algorithm. For example, if the user was taking photos surrounding an object, the 3D asset generation support unit 322A may use a generation algorithm specifically for generating 3D models of objects. Also, if the user was moving horizontally while taking photos, the 3D asset generation support unit 322A may use a generation algorithm specifically for generating 3D models of landscapes. Furthermore, if the user was moving around in space and taking detailed photos of various places, a photogrammetry algorithm may be used.

[0071] The examples of generation algorithms described above are all just examples. The generation algorithm selected when generating an asset may change due to advancements in generation algorithm technology. For example, system updates may update information on which algorithm to select when generating certain types of assets. The production support system 30 also has information in advance on which generation algorithm is appropriate for which use case. Furthermore, users can arbitrarily change the default settings for which generation algorithm to use, as presented by the production support system 30. Alternatively, the user may input conditions regarding the assets to be generated, and the production support system 30 may select or generate an appropriate generation algorithm in response to the user input. Conditions regarding the assets to be generated include, for example, size, image quality level, generation speed, editability, and AI tolerance.

[0072] As explained above with reference to Figure 6, three generative algorithms have been given, but the number of generative algorithms is not limited to these. Furthermore, the properties of each generative algorithm are not limited to the examples given above. For example, a generative algorithm using generative AI may also be used.

[0073] The 3D asset generation support unit 322A can select a generation algorithm from among various generation algorithms that is appropriate for the planned subsequent processing by obtaining information on the subsequent processing stage, i.e., information on the intended use such as the content to be generated and the output destination, in advance. This increases the efficiency of asset and content production and improves user convenience.

[0074] (Utilization Support Department 323) The utilization support unit 323 provides support for the utilization of assets. Specifically, this includes content generation by the content generation unit 324 and content output by the content output unit 325.

[0075] The content generation unit 324 generates content based on one or more assets generated by the asset generation support unit 322. The content generation unit 324 also supports the generation of content corresponding to usage information. For example, the content generation unit 324 generates content used as a background for virtual productions, content used as a prop for games, and so on.

[0076] Figure 7 shows an example of the generation units included in the content generation unit 324 according to this embodiment. As shown in Figure 7, the content generation unit 324 includes a scene generation unit 324A, a Previs generation unit 324B, a 2D video generation unit 324C, an asset / content management unit 324D, and a 3D rendering unit 324E. These are examples of multiple different utilization units.

[0077] The scene generation unit 324A generates a scene (a video of a certain length of time) using one or more assets. The scene generation unit 324A may generate scenes using LLM (Large Language Models) or AI. The conditions for the scene to be generated may be input by the user as appropriate. The Previs generation unit 324B generates a Previs used in video production using one or more assets. A Previs is a simplified 3DCG video that is similar to the final video before actual filming in video production. The 2D video generation unit 324C generates a 2D video using one or more assets. The conditions for the 2D video to be generated may be input by the user as appropriate.

[0078] The Asset & Content Management Unit 324D manages assets and content. For example, the Asset & Content Management Unit 324D processes the creation of assets and content and saves them in association with user IDs. The 3D Rendering Unit 324E performs 3D rendering based on one or more assets. For example, the 3D Rendering Unit 324E converts 3D assets (3D models) into 2D images.

[0079] The content output unit 325 outputs the content generated by the content generation unit 324 to an external source. The content output unit 325 is an example of a utilization unit. For example, the content output unit 325 may output the content to the output device 20. Alternatively, the content output unit 325 may output the content to an external cloud service 40. For example, if the external cloud service 40 is a metaverse creation service or a game creation service, the content generation unit 324 can send the generated content to the external cloud service 40 in response to user operations, providing content for metaverse creation or game creation. The external cloud service 40 may also be a distribution service, and the content output unit 325 can distribute the content via the external cloud service 40. Assets may also be output as content directly.

[0080] (Plugin application acquisition unit 326) The plugin application acquisition unit 326 can acquire plugin applications from the external plugin application server 50 and extend the functionality of the production support system 30.

[0081] (Account Management Department 327) The account management unit 327 manages the accounts of users who use the production support system 30. For example, the account management unit 327 stores user IDs, login passwords, and email addresses in an associated manner.

[0082] (License granting section 328) The licensing unit 328 grants licenses to the generated assets and content that guarantee the origin and authenticity of the digital data, that is, that the assets and content were generated from user-captured data (sensing data) and not from AI-generated data with unknown rights. Specifically, the licensing unit 328 may embed metadata as a license into the assets or content, including information such as when, where, and by whom the asset or content was created, the editing content, an electronic signature for creator authentication, and a timestamp. Even when the assets and content are distributed, the reliability of the assets and content can be guaranteed by such licenses.

[0083] (Sales Management Department 329) The sales management unit 329 manages the sales of the generated assets and content. Specifically, the sales management unit 329 may send the generated assets and content to a store service server included in the external cloud service 40 and have it sold in the store. The sales management unit 329 may manage the user's sales license by associating it with the user ID.

[0084] The functions of the production support system 30 have been described in detail above, but the functional configuration described with reference to Figure 4 is just one example, and this disclosure is not limited thereto. The production support system 30 may have various other functions. For example, the production support system 30 may generate various UIs, such as application selection screens, to be displayed on the capture device 10 and the output device 20, and transmit them to the capture device 10 and the output device 20. The production support system 30 may also control setting parameters for the capture device 10 according to the application information. Furthermore, the production support system 30 may also present imaging navigation to the user on the capture device 10 according to the application information.

[0085] <3. Operation Processing> Figure 8 is a diagram illustrating the overall processing flow according to this embodiment. As shown in Figure 8, the processes performed in this embodiment are broadly as follows: S1: Acquisition of captured images (sensing data), S2: Generation of assets based on captured images, and S3: Utilization of assets (generation of content using assets, output of generated content).

[0086] In this embodiment, as an example, production efficiency can be improved by performing the preceding processing according to the processing schedule for the subsequent processing. That is, for example, in S1, by acquiring usage information, captured images corresponding to the assets to be generated are acquired. Also, in S2, by acquiring usage information, assets corresponding to the content to be generated and the output destination are generated (using the corresponding generation algorithm).

[0087] Figure 9 shows an example of the imaging process flow according to this embodiment. As shown in Figure 9, first, the capture device 10 acquires application information through user operation on the displayed application selection screen (step S103).

[0088] Then, the capture device 10 performs imaging support processing according to the acquired application information (step S106). Specifically, the capture device 10 sets camera parameters, controls the shutter, and presents imaging navigation to the user according to the application information.

[0089] Figure 10 shows an example of the production support processing flow according to this embodiment. As shown in Figure 10, first the production support system 30 acquires the captured image and associated data output from the capture device 10 (step S133).

[0090] Next, the production support system 30 acquires usage information through user operation on the displayed usage selection screen (step S136). Note that usage information obtained before imaging by the capture device 10 may be shared.

[0091] Next, the production support system 30 selects a generation algorithm suitable for generating assets to be used in the subsequent content generation, according to the usage information (step S139).

[0092] Next, the production support system 30 generates assets using the selected generation algorithm (step S142).

[0093] Next, the production support system 30 generates content using one or more generated assets (step S145).

[0094] The production support system 30 then outputs the generated content (step S148).

[0095] The operation process according to this embodiment has been described above. The process described above is merely an example, and this disclosure is not limited thereto. For example, not all of the processes described above are performed, and the order of some of the processes described above may be different.

[0096] <4.UI example> Next, we will specifically describe examples of UIs presented to the user in the production support system 30 according to this embodiment. Each UI may be displayed on the capture device 10 or on the output device 20, which is an example of a user terminal, under the control of the production support system 30. Alternatively, the production support AP execution units 122 and 222 may display each UI on the capture device 10 or the output device 20.

[0097] <<4-1. Application Selection Screen>> Figure 11 shows an example of an application selection screen according to this embodiment. The application selection screen 410 shown in Figure 11 displays options for asset utilization applications such as VP (Visual Presentation) exhibition, video production, game production, digital twin, and metaverse. Users can select an application by selecting any option. This application selection screen 410 is displayed on the capture device 10 before sensing data is acquired, or it is displayed on the user terminal after the sensing data is uploaded and before the production support system 30 generates the asset.

[0098] The usage options in the usage selection screen 410 shown in Figure 11 are displayed as text, but this disclosure is not limited thereto. For example, the usage options may be displayed as images. Figure 12 shows another example of the usage selection screen according to this embodiment. As shown in Figure 12, the usage selection screen 420 displays the usage options as images. Each image may be one that makes it easy to intuitively understand the usage (thumbnail, eye-catching image, icon, etc.).

[0099] Furthermore, the production support system 30 may estimate the intended use from the captured image uploaded by the user and, when displaying the intended use selection screen, present the estimated intended use to the user with it automatically selected as a preset. Figure 13 shows another example of the intended use selection screen according to this embodiment. As shown in Figure 13, the intended use estimated from the captured image (2D image) (in the example shown in Figure 13, "VP exhibition") is selected on the intended use selection screen 430. The user can then select an intended use after confirming the estimation result by the system.

[0100] Furthermore, the production support system 30 may present the user with one or more uses estimated from the captured images uploaded by the user as options for use. Figure 14 shows another example of the use selection screen according to this embodiment. As shown in Figure 14, the use selection screen 440 displays uses estimated from the captured images (2D images) (in the example shown in Figure 14, "video production," "game production," and "metaverse"). The user selects any use from these options.

[0101] <<4-2. Other Selection Screen>> In the example described above, the production support system 30 generates assets and content according to the usage information, but this disclosure is not limited to this. For example, if the user has a certain level of knowledge or if the assets they want to generate are clear, it is conceivable that the user may make more specific choices when generating assets.

[0102] Figure 15 shows an example of a size selection screen for the 3D assets generated according to this embodiment. As shown in Figure 15, the selection screen 450 displays options for selecting the size of the 3D assets to be generated, such as buildings (assuming large structures), landscapes (assuming spaces), and objects (assuming small objects). The production support system 30 determines the generation algorithm considering the size of the 3D assets to be generated selected by the user.

[0103] The production support system 30 may, after the user selects the size of the 3D asset, transition to a usage selection screen and accept the usage selection. The production support system 30 then determines an appropriate generation algorithm, taking into account the size and usage of the 3D asset to be generated selected by the user.

[0104] Figure 16 shows an example of a screen for selecting a generation algorithm used to generate 3D assets according to this embodiment. As shown in Figure 16, the selection screen 460 displays the generation algorithm options in text format. Examples of selectable generation algorithms include Photogrammetry, which constructs a 3D mesh of an object from a 2D image; Advanced Photogrammetry, which reproduces the shape of an object captured by surrounding it with high quality; Instant 3D Gaussian Splatting, which transforms reality into photorealistic 3DCG at high speed; and Advanced 3D Gaussian Splatting, which reproduces photorealistic 3DCG of reality in ultra-high definition. In this way, the user may be able to directly select the generation algorithm used to generate 3D assets. Note that the generation algorithm options are not limited to text formatting as shown in Figure 16, but may also be represented by images. The image representation may be, for example, a display of a sample asset.

[0105] Figure 17 shows an example of a selection screen for the output format of a 3D asset according to this embodiment. As shown in Figure 17, the selection screen 470 displays the options for the output format of the 3D asset (for example, "Mesh" and "3D Gaussian Splatting") in text notation. The production support system 30 determines the generation algorithm according to the output format of the 3D asset to be generated selected by the user. Note that the output format options are not limited to text notation as shown in Figure 17, but may also be represented by images such as sample 3D assets, eye-catching images, and icons.

[0106] <<4-3. AI Usage and Paid Charging>> When selecting a use on the usage selection screen, it is conceivable that the generation of the corresponding assets may involve the use of AI or be subject to paid charges.

[0107] Figure 18 shows an example of a display screen that includes uses in which asset generation according to this embodiment is subject to AI utilization or paid charges. Among the uses displayed on the selection screen 480 in Figure 18, for example, the option for use as a background for virtual production displays the mark "PRO" indicating that it is subject to paid charges. Also, for example, the option for use in VR SNS displays the mark "AI Support" indicating that processing using generation AI is included. This allows users to intuitively understand whether the generation of the corresponding asset is subject to AI utilization or paid charges. Note that the display manner of each mark is just an example, and this embodiment is not limited thereto.

[0108] Figure 19 shows an example of a warning screen that appears when an option marked with the generation AI icon is selected on the application selection screen according to this embodiment. As shown in Figure 19, the warning screen 500 displays a confirmation message such as, "This process will generate 3D assets using the generation AI model provided by Company A. Is this OK?" The user taps the "Generate" button if they agree to use the generation AI model, or the "Cancel" button if they do not agree.

[0109] <5. Variation> The production support system 30 may generate multiple assets using multiple generation algorithms during asset generation, then present a usage selection screen to accept the user's selection of usage, and output the asset corresponding to the selected usage.

[0110] More specifically, first, the user uploads the captured images taken with the capture device 10 to the production support system 30 by operating on the upload screen. Next, the user selects one or more captured images to be used for asset generation from the list of uploaded captured images and taps the 3D generation button. When the 3D generation button is tapped, the user is not asked about the purpose at this point, and the production support system 30 starts generating multiple assets using multiple generation algorithms. After the assets are generated, a purpose selection screen is presented, and the user selects the purpose. The production support system 30 outputs (exports) the assets corresponding to the selected purpose from the generated assets. The data of assets that were not output is deleted after a certain period of time.

[0111] Figure 20 shows an example of a usage selection screen presented after asset generation according to this embodiment. As shown in Figure 20, the usage selection screen 520 displays usage options along with a message such as, "Please select the usage of the 3D asset. We will perform the optimal file conversion according to the usage." The usage selection screen 520 may also display a generation preview screen. This allows the user to preview the generated image for each usage, making it easier to select.

[0112] Another advantage of selecting the intended use at a later stage is that users can upload captured images and begin generating assets immediately, which can contribute to more efficient production.

[0113] Figure 21 shows an example of the production support process flow according to a modified version of this embodiment. As shown in Figure 21, first the production support system 30 acquires the captured image and associated data output from the capture device 10 (step S203).

[0114] Next, the production support system 30 generates assets using all generation algorithms (step S206).

[0115] Next, the production support system 30 displays a usage selection screen and acquires usage information based on user operation (step S209).

[0116] Next, the production support system 30 selects an asset from the generated assets that corresponds to the usage information (step S212).

[0117] Next, the production support system 30 generates content using one or more selected assets (step S215).

[0118] The production support system 30 then outputs the generated content (step S218).

[0119] <6. Hardware Configuration> An embodiment of the present disclosure has been described above. Next, with reference to Figure 22, an example of a hardware configuration used in a capture device 10, an output device 20, or a production support system 30 according to an embodiment of the present disclosure will be described.

[0120] Figure 22 is a block diagram showing an example of the hardware configuration of an information processing device 900 according to one embodiment of the present disclosure. The information processing device 900 is an example of a hardware configuration applied to the capture device 10, output device 20, or production support system 30 according to this embodiment. Note that the information processing device 900 does not necessarily have all of the hardware configurations shown in Figure 22.

[0121] As shown in Figure 22, the information processing device 900 includes a processing circuit 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925.

[0122] The processing circuit 901 functions as an arithmetic processing unit and control unit, controlling all or part of the operation within the information processing unit 900 according to various programs recorded in the ROM 902, RAM 903, storage device 919, or removable recording medium 927. The ROM 902 stores programs and arithmetic parameters used by the processing circuit 901. The RAM 903 temporarily stores programs used in the execution of the processing circuit 901 and parameters that change as appropriate during its execution. The processing circuit 901, ROM 902, and RAM 903 are interconnected by a host bus 907, which is composed of an internal bus. Furthermore, the host bus 907 is connected to an external bus 911, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 909.

[0123] The input device 915 is a device operated by the user, such as a button. The input device 915 may also include a mouse, keyboard, touch panel, switch, and lever. The input device 915 may also include a microphone that detects the user's voice. The input device 915 may be, for example, a remote control device that uses infrared or other radio waves, or an external connection device 929 such as a mobile phone that is compatible with the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs it to the processing circuit 901. By operating this input device 915, the user inputs various data to the information processing device 900 or instructs it to perform processing operations.

[0124] The input device 915 may also include an imaging device and sensors. The imaging device is a device that captures real space and generates an image using various components such as an image sensor, such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens for controlling the imaging of a subject onto the image sensor. The imaging device may capture still images or motion images. The sensors are various types of sensors, such as distance sensors, acceleration sensors, gyro sensors, geomagnetic sensors, vibration sensors, light sensors, and sound sensors. The sensors acquire information about the state of the information processing device 900 itself, such as the orientation of the housing of the information processing device 900, and information about the surrounding environment of the information processing device 900, such as the brightness and noise around the information processing device 900. The sensors may also include a GPS sensor that receives GPS (Global Positioning System) signals and measures the latitude, longitude, and altitude of the device.

[0125] The output device 917 is comprised of a device capable of visually or audibly notifying the user of the acquired information. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or an audio output device such as a speaker or headphones. The output device 917 may also include a PDP (Plasma Display Panel), a projector, a hologram, a printer, etc. The output device 917 outputs the results obtained from the processing of the information processing device 900 as text or images, or as sound such as voice or sound. The output device 917 may also include a lighting device that brightens the surroundings.

[0126] The storage device 919 is a data storage device configured as an example of the storage unit of the information processing device 900. The storage device 919 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. This storage device 919 stores programs and various data executed by the processing circuit 901, as well as various data acquired from external sources.

[0127] The drive 921 is a reader / writer for removable recording media 927, such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory, and is either built into or external to the information processing device 900. The drive 921 reads information recorded on the installed removable recording media 927 and outputs it to the RAM 905. The drive 921 also writes data to the installed removable recording media 927.

[0128] The connection port 923 is a port for directly connecting equipment to the information processing device 900. The connection port 923 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. Alternatively, the connection port 923 may be an RS-232C port, an optical audio terminal, or an HDMI (High-Definition Multimedia Interface) port. By connecting an external device 929 to the connection port 923, various types of data can be exchanged between the information processing device 900 and the external device 929.

[0129] The communication device 925 is a communication interface, for example, consisting of a communication device for connecting to an external network 70. The communication device 925 may be, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth®, Wi-Fi®, or WUSB (Wireless USB). Alternatively, the communication device 925 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication. The communication device 925 sends and receives signals, for example, to the Internet or other communication devices using a predetermined protocol such as TCP / IP. The external network 70 connected to the communication device 925 is a network connected by wire or wireless, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication.

[0130] For example, when the information processing device 900 functions as a capture device 10, an output device 20, or a production support system 30 according to the embodiments of this disclosure, the processing circuit 901 of the information processing device 900 functions as the control unit 120, control unit 220, or various functions of the production support system 30 (image capture management unit 321, asset generation support unit 322, utilization support unit 323, plug-in application acquisition unit 326, account management unit 327, license granting unit 328, sales management unit 329) by executing a program loaded on the RAM 903. In addition, the storage device 919 stores the information processing program related to this disclosure and various data stored in the memory unit 150, memory unit 240, or memory unit (not shown) included in the production support system 30.

[0131] The processing circuit 901 reads and executes program data from the storage device 919, but as an alternative, these programs may be obtained from other devices via the external network 70. In other words, the storage device 919 is not limited to being inside the information processing device 900, but may be located outside the information processing device 900.

[0132] The processing circuit 901 is an example of an integrated circuit, and CPUs (Central Processing Units), MPUs (Micro Processing Units), GPUs (Graphics Processing Units), APUs (Accelerated Processing Units), ASICs (Application Specific Integrated Circuits), and FPGAs (Field Programmable Gate Arrays) can all be considered integrated circuits.

[0133] Furthermore, when the information processing device 900 functions as a capture device 10, an output device 20, or a production support system 30 according to the embodiments of this disclosure, the communication device 925 corresponds to the communication unit 110, the communication unit 210, or a communication unit (not shown) included in the production support system 30. Also, the input device 915 and the output device 917 correspond to the operation display unit 140 or the operation display unit 230. In addition, the input device 915 corresponds to the sensor unit 130.

[0134] <7. Supplementary Information> While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the present technology is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0135] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0136] The information processing system described herein may consist of a single device (such as an information processing terminal or a server), or it may consist of multiple devices. Multiple devices may include, for example, an information processing terminal and one or more servers.

[0137] Furthermore, the embodiments and modifications of this disclosure described above can be combined as appropriate in areas where the processing content is not contradictory. Also, the order of each step shown in the sequence diagram or flowchart of this embodiment can be changed as appropriate. For example, each step may be processed chronologically, repeatedly, or partially in parallel.

[0138] Furthermore, one or more computer programs can be created to enable the capture device 10, output device 20, or production support system 30 to perform their functions, using hardware such as the CPU, ROM, and RAM built into the capture device 10, output device 20, or production support system 30. A computer-readable storage medium containing such one or more computer programs is also provided.

[0139] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0140] Furthermore, this technology can also be configured as follows. (1) A generation support unit having multiple different generation units for generating assets, A utilization support unit having multiple different utilization units that utilize the assets generated by the generation support unit, Equipped with, The assets used in the utilization unit are assets generated by any of the generation units that correspond to the utilization unit. Information processing system. (2) The information processing system according to (1), wherein each of the multiple different generation units has a different generation method for generating the assets. (3) The information processing system according to (2), wherein the plurality of different generation units generate assets that differ in at least one of the following: size, image quality level, generation speed, editability, and AI tolerance. (4) The information processing system according to any one of items (1) to (3) above, wherein the generation support unit determines the generation unit that generates the corresponding asset based on usage information indicating the intended use desired by the user, and generates the asset using the determined generation unit. (5) The aforementioned generation support unit generates multiple assets using multiple generation units, The aforementioned utilization support unit is an information processing system according to any one of items (1) to (3) above, which selects a corresponding asset from among the multiple assets based on usage information indicating the intended use of the asset desired by the user, and utilizes the selected asset. (6) The information processing system described in any one of items (1) to (5) above, wherein the asset is a 3D asset, a human asset, a motion asset, an audio asset, or a 3D video. (7) The aforementioned multiple different generation units use different generation methods for one type of asset among the different types of assets. The information processing system described in (6) above, wherein each asset of a single type generated by each generation unit differs in at least one of the following: size, image quality type, generation speed, editability, and AI tolerance. (8) The aforementioned information processing system further comprises an acquisition unit that acquires sensing data of an object, The generation support unit is an information processing system according to any one of items (1) to (7) above, wherein the generation unit generates the asset based on the sensing data. (9) The sensing data is one or more captured images. The generation support unit is an information processing system according to (8) above, which generates the asset based on one or more captured images. (10) The information processing system further comprises an imaging support unit that assists in imaging the object, The imaging support unit performs imaging support processing to acquire sensing data used to generate a corresponding asset, according to usage information indicating the intended use desired by the user. The information processing system described in (9) above. (11) The imaging support unit is the information processing system described in (10) above, which sets camera parameters, controls the shutter, or navigates the position and orientation of the image. (12) The utilization unit generates content using one or more assets generated by the generation unit, as described in any one of items (1) to (11) above. (13) The utilization unit is an information processing system according to any one of items (1) to (12) above, which outputs the asset or content generated using one or more of the assets. (14) The information processing system described in (12) or (13) above, wherein the utilization support unit determines the utilization unit according to the intended use based on the utilization information that indicates the intended use desired by the user, and utilizes the asset using the determined utilization unit. (15) The information processing system according to any one of items (12) to (14) above, wherein each of the different types of content generated by each of the multiple different utilization units is a scene, a video for video production simulation, a 2D video, or a 3D model. (16) The aforementioned utilization unit is an information processing system according to any one of items (12) to (15) above, which generates the content that satisfies the distribution conditions of the output destination indicated by the usage information. (17) The information processing system described in any one of items (1) to (16) above, which presents a display screen that accepts input for selecting the use of the asset. (18) The information processing system described in (17) above, wherein the options for use displayed on the aforementioned display screen include marks indicating that asset generation is subject to AI utilization or is subject to paid billing. (19) The processor, The asset is generated by at least one of several different generating units, The assets generated by the generation unit are utilized by at least one of several different utilization units, Includes, The assets used in the utilization unit are assets generated by any of the generation units that correspond to the utilization unit. Information processing methods. (20) Computers, A system comprising: a generation support unit having multiple different generation units for generating assets; and a utilization support unit having multiple different utilization units for utilizing the assets generated by the generation support unit, wherein the assets used in the utilization units are assets generated by any of the generation units corresponding to the utilization units, and a communication unit that communicates with the system, A control unit controls the generation unit to transmit sensing data used for generating the asset to the system, and acquires information from the system regarding the asset generation results by the generation unit and the utilization results by the utilization unit. A program designed to function as such. [Explanation of Symbols]

[0141] 10 Capture Devices 110 Communications Department 120 Control Unit 122 Production Support AP Execution Department 130 Sensor section 140 Operation display section 150 Storage section 20 Output Devices 210 Communications Department 220 Control Unit 222 Production Support AP Execution Department 230 Operation display section 240 Storage section 30 Production support system 321 Image Management Department 322 Asset Generation Support Department 322A 3D Asset Generation Support Department 322A-1 First 3D Asset Generation Unit 322A-2 Second 3D Asset Generation Unit 322A-3 Third 3D Asset Generation Unit 323 Utilization Support Department 324 Content Generation Unit 325 Content Output Section 326 Plugin application acquisition section 327 Account Management Department 328 Licensing Department 329 Sales Management Department 40 External Cloud Services 50 External Plugin Application Server

Claims

1. The generation support unit assists in the creation of assets, A utilization support unit that utilizes assets generated by the first generation support unit or the second generation support unit, An analysis unit analyzes the attributes of the asset generated by the generation support unit, Equipped with, The aforementioned utilization support unit determines the utilization unit to be used for utilizing the asset based on the analysis results from the aforementioned analysis unit. Information processing system.

2. The information processing system according to claim 1, wherein each of the plurality of different generation units has a different generation method for generating the assets.

3. The information processing system according to claim 2, wherein the plurality of different generation units generate assets that differ in at least one of the following: size, image quality level, generation speed, editability, and AI tolerance.

4. The aforementioned generation support unit generates multiple assets using multiple generation units, The information processing system according to claim 1, wherein the utilization support unit selects a corresponding asset from the plurality of assets based on the usage information and utilizes the selected asset.

5. The information processing system according to claim 1, wherein the asset is a 3D asset, a human asset, a motion asset, an audio asset, or a 3D video.

6. The information processing system according to claim 5, wherein the plurality of different generation units use different generation methods for one type of asset from among different types of assets, and each asset of one type generated by each generation unit differs in at least one of the following: size, image quality type, generation speed, editability, and AI tolerance.

7. The aforementioned information processing system further comprises an acquisition unit that acquires sensing data of an object, The generation support unit generates the asset based on the sensing data, The information processing system according to claim 1.

8. The sensing data is one or more captured images. The information processing system according to claim 7, wherein the generation support unit generates the asset based on one or more captured images.

9. The information processing system further comprises an imaging support unit that assists in imaging the object, The information processing system according to claim 8, wherein the imaging support unit performs camera parameter setting, shutter control, or navigation of the position and orientation of the image being taken.

10. The information processing system according to claim 1, wherein the utilization unit generates content using one or more assets generated by the generation unit.

11. The information processing system according to claim 1, wherein the utilization unit outputs the asset, or content generated using one or more of the assets.

12. The generation support unit generates the asset using the generation unit determined above. The aforementioned utilization support unit will utilize the assets according to the utilization unit determined above. The information processing system according to claim 11.

13. The information processing system according to claim 1, wherein the information processing system presents a display screen that accepts input for selecting an application for utilizing the asset.

14. The processor, The generation support unit assists in the generation of assets, The assets generated by the first generation support unit or the second generation support unit are utilized by the utilization support unit, The attributes of the asset generated by the generation support unit are analyzed by the analysis unit, Includes, Furthermore, based on the analysis results from the analysis unit, the utilization support unit determines the utilization unit to be used for utilizing the asset, Information processing methods, including those mentioned above.

Images