Information processing system, information processing method, and program

Abstract

It enables comprehensive support for asset creation and utilization. [Solution] An information processing system comprising a generation support unit having a plurality of different generation units that generate assets, and a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support unit, wherein the assets used by the utilization units are assets corresponding to the utilization units that were generated by any of the generation units.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, there has been an increase in the use of 3D assets generated from multiple 2D images to create various types of content.

[0003] For example, Patent Document 1 below discloses a technology for reproducing free viewpoint data generated from multiple viewpoint images in response to operations on a 3D strobe image viewed from a virtual viewpoint, in which a strobe model is created in which 3D models of an object at multiple times, generated from multiple viewpoint images captured from multiple viewpoints, are arranged in three-dimensional space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-047548 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the creation of assets such as 3D assets and the creation of various content using those assets were each done in separate applications, and users had to identify and prepare the assets appropriate for the content they wanted to create. Because these applications were not linked, this increased work costs and hindered efficient content creation.

[0006] Therefore, the present disclosure proposes an information processing system, an information processing method, and a program that can provide overall support for the creation and utilization of assets. [Means for solving the problem]

[0007] According to the present disclosure, an information processing system is provided that includes a generation support unit having a plurality of different generation units that generate assets, and a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support unit, and the assets used in the utilization units are assets that correspond to the utilization units and were generated by any of the generation units.

[0008] Furthermore, according to the present disclosure, an information processing method is provided, which includes a processor generating assets using at least one of a plurality of different generation units, and utilizing the assets generated by the generation units using at least one of a plurality of different utilization units, and the assets used by the utilization units are assets corresponding to the utilization units and generated by any of the generation units.

[0009] Furthermore, according to the present disclosure, a program is provided to cause a computer to function as: a generation support unit having a plurality of different generation units that generate assets; and a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support units, wherein the assets used by the utilization units are assets generated by any of the generation units and corresponding to the utilization units; a communication unit that communicates with a system; and a control unit that controls the transmission of sensing data used in the generation of the assets by the generation units to the system, and acquires information regarding the results of the generation of the assets by the generation units and the results of the utilization by the utilization units from the system. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an overview of a production support system 30 (an example of an information processing system) according to an embodiment of the present disclosure. [Figure 2] 1 is a block diagram showing an example of the configuration of a capture device 10 according to the present embodiment. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of an output device 20 according to the present embodiment. [Figure 4]FIG. 2 is a block diagram showing an example of the functional configuration of a production support system 30 according to the present embodiment. [Figure 5] FIG. 2 is a diagram illustrating an example of the functional configuration of an asset generation support unit 322 according to the present embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a generation unit included in a 3D asset generation support unit 322A according to this embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a generation unit included in a content generation unit 324 according to this embodiment. [Figure 8] FIG. 2 is a diagram for explaining the overall processing flow according to the present embodiment. [Figure 9] FIG. 10 is a diagram showing an example of the flow of imaging processing according to the present embodiment. [Figure 10] FIG. 10 is a diagram showing an example of the flow of production support processing according to the present embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a use selection screen according to the present embodiment. [Figure 12] FIG. 10 is a diagram showing another example of the use selection screen according to the present embodiment. [Figure 13] FIG. 10 is a diagram showing another example of the use selection screen according to the present embodiment. [Figure 14] FIG. 10 is a diagram showing another example of the use selection screen according to the present embodiment. [Figure 15] FIG. 10 is a diagram showing an example of a size selection screen for a 3D asset to be generated according to this embodiment. [Figure 16] FIG. 10 is a diagram showing an example of a selection screen for a generation algorithm used to generate a 3D asset according to this embodiment. [Figure 17] FIG. 10 is a diagram showing an example of a selection screen for an output format of a 3D asset according to this embodiment. [Figure 18] FIG. 10 is a diagram showing an example of a display screen including uses in which asset generation according to this embodiment is subject to AI use or paid charges. [Figure 19] 10 is an example of a warning screen that is displayed when an option marked with a generated AI is selected on the application selection screen according to this embodiment. [Figure 20]FIG. 10 is a diagram showing an example of a use selection screen presented after asset generation according to the present embodiment. [Figure 21] FIG. 10 is a diagram showing an example of the flow of production support processing according to a modified example of the present embodiment. [Figure 22] FIG. 9 is a block diagram showing an example of a hardware configuration of an information processing device 900 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0012] The explanation will be given in the following order: 1. Overview 2. Configuration 2-1. Configuring Capture Device 10 2-2. Configuration of the output device 20 2-3.Configuration of Production Support System 30 3. Operation processing 4.UI example 4-1. Use selection screen 4-2.Other selection screen 4-3. Use of AI and paid charges 5. Variations 6. Hardware Configuration 7. Supplementary Information

[0013] <1. Overview> Fig. 1 is a diagram illustrating an overview of a production support system 30 (an example of an information processing system) according to an embodiment of the present disclosure. As shown in Fig. 1, the production support system 30 is communicatively connected to a capture device 10, an output device 20, an external cloud service 40, and an external plug-in application server 50. Note that the configuration shown in Fig. 1 is an example, and the present embodiment is not limited to this.

[0014] The production support system 30 is a system that provides overall support for the creation and utilization of assets. The production support system 30 can be formed from one or more servers. Each function of the production support system 30 can be realized by each application or each cloud service. Each application and each cloud service can work together as appropriate to realize processing in an earlier stage according to the processing schedule of the later stage, and processing in a later stage according to the processing results of the earlier stage.

[0015] The production support system 30 is a system that can execute processes from asset generation to utilization. This allows for more efficient content production by utilizing information from both pre-processing (e.g., asset generation) and post-processing (e.g., content generation) to mutually utilize information, such as generating appropriate assets depending on the intended use of the content (or assets) to be ultimately produced, proposing uses based on the generated assets, and automatically selecting assets to be used depending on the intended use.

[0016] Examples of uses (purposes) include virtual production, film production, film prop production, game background production, game prop production, VR game production, metaverse production, digital twin world production, digital twin world prop production, Previz (pre-visualization) production, Previz prop production, 3D commerce production, 3D exhibitions, 3D digital signage, advertising production, corporate videos, and digital archives.

[0017] Utilization may be, for example, the generation of content or the output of the generated content, such as sending the content to the output device 20, uploading the content to an external cloud service, or distributing the content via an external distribution platform.

[0018] The capture device 10 is a device that acquires sensing data of an object. Examples of the capture device 10 include an imaging device that captures an image of the object, a measuring device that measures the distance to the object or the shape of the object, a motion capture device that tracks the movement of the object, and a sound collection device that captures sound from the object, its surroundings, or the space. The capture device 10 may be composed of multiple devices. Furthermore, the capture device 10 may be a dedicated device (e.g., a mirrorless camera, a ToF (Time Of Flight) camera, a 3D laser scanner, a LiDAR (Light Detection and Ranging) device, a sound recording device, etc.), or may be realized by a general-purpose device such as a smartphone, a smartwatch, or a tablet terminal.

[0019] The sensing data of the object acquired by the capture device 10 is input to the production support system 30. The sensing data of the object may be uploaded from the capture device 10 to the production support system 30 via a network, or may be transmitted to the production support system 30 via a wire or other device.

[0020] The output device 20 is an example of an information processing terminal that outputs various types of information. The output device 20 is realized by, for example, a smartphone, a smart watch, a tablet terminal, a PC (Personal Computer), an HMD (Head Mounted Display), a stereoscopic display device, a large display device, a projector, or the like. The output device 20 also has an operation input function. The output device 20 may be used as a user interface when the production support system 30 generates and utilizes assets. The output device 20 may also be used as a playback device that plays back content generated by the production support system 30.

[0021] The external cloud service 40 may provide various services via the Internet. The external cloud service 40 may be implemented as a server or a system consisting of multiple servers. The production support system 30 can communicate with the external cloud service 40 and use the cloud service as appropriate for generating assets and content.

[0022] The external plug-in application server 50 can provide plug-in applications via the Internet. The external plug-in application server 50 may be implemented as a system consisting of multiple servers. The production support system 30 can, for example, acquire applications as needed from the external plug-in application server 50 and use the external applications to generate assets and content. This allows the production support system 30 to expand its functionality.

[0023] <2. Configuration> <<2-1. Configuration of Capture Device 10>> 2 is a block diagram showing an example of the configuration of the capture device 10 according to this embodiment. As shown in FIG. 2, the capture device 10 includes a communication unit 110, a control unit 120, a sensor unit 130, an operation display unit 140, and a storage unit 150.

[0024] (Communication unit 110) The communication unit 110 has a transmitting unit that transmits data to an external device and a receiving unit that receives data from the external device. The communication unit 110 according to this embodiment is connected to an external device or the Internet for communication using, for example, a wired / wireless local area network (LAN), Wi-Fi (registered trademark), Bluetooth (registered trademark), a mobile communication network, or the like.

[0025] (Sensor unit 130) The sensor unit 130 is an acquisition unit that acquires sensing data of an object. The sensor unit 130 may have a plurality of sensors. For example, the sensor unit 130 may include an imaging unit. The imaging unit has one or more lenses (optical systems) and an imaging element such as a CCD image sensor or a CMOS image sensor, and captures an image of the object under the control of the control unit 120 and outputs the captured image (RGB image).

[0026] The sensor unit 130 may also include a ToF camera. The ToF camera measures the distance to an object and outputs a depth image that visualizes the distance information. The ToF camera is an example of a distance sensor, and the present disclosure is not limited thereto.

[0027] The sensor unit 130 may also include a position and orientation sensor that detects the position and orientation of the capture device 10. Specifically, the sensor unit 130 may include a position sensor, an IMU (Inertial Measurement Unit), and a geomagnetic sensor. The position sensor calculates absolute position information or relative position information of the capture device 10. The sensor unit 130 outputs three-dimensional position and orientation information of the capture device 10 to the control unit 120.

[0028] (control unit 120) The control unit 120 functions as an arithmetic processing unit and a control unit, and controls the overall operation of the capture device 10 in accordance with various programs. The control unit 120 is realized by an electronic circuit such as a CPU (Central Processing Unit) or a microprocessor. The control unit 120 may also include a ROM (Read Only Memory) that stores the programs to be used, arithmetic parameters, etc., and a RAM (Random Access Memory) that temporarily stores parameters that change as appropriate.

[0029] The control unit 120 according to this embodiment can function as a production support AP (application) execution unit 122.

[0030] The production support AP execution unit 122 executes a production support application installed in the capture device 10. The capture device 10 acquires sensing data used to generate assets for content production, and the accuracy, amount, and method of acquiring the sensing data appropriate for the asset to be generated vary depending on the asset. The production support AP execution unit 122 displays a usage selection screen on the operation display unit 140, allowing the user to select the intended use of the content they ultimately want to create, and acquires usage information. The production support AP execution unit 122 also controls the appropriate acquisition of sensing data used to generate assets corresponding to the usage information (i.e., assets to be used for usage according to the intended use).

[0031] For example, when acquiring sensing data (captured images) using the imaging unit, the production support AP execution unit 122 may perform imaging support such as setting camera parameters (aperture, exposure, white balance, focal length, etc.), shutter control, and imaging position and orientation navigation (guidance on the distance and direction to capture images from a target object) so as to appropriately acquire data used to generate assets corresponding to the usage information. The production support AP execution unit 122 is an example of an imaging support unit. The production support AP execution unit 122 automatically sets camera parameters according to the size of an asset to be generated, the required quality (resolution), etc., such as a landscape, a structure, an indoor space, a car, or a small item. The production support AP execution unit 122 also presents imaging position and orientation navigation on the operation display unit 140. The shutter control may be automatic by the production support AP execution unit 122, or the user may manually operate the shutter control by referring to the navigation.

[0032] The production support AP execution unit 122 also saves, as ancillary data, imaging information at the time of image capture (for example, position and orientation information (IMU output data, etc.)), camera metadata (for example, F-number, focal length, exposure value, image stabilization information, subject distance), lens metadata (for example, information on distortion correction, chromatic aberration correction, and monochromatic aberration correction). This ancillary data is output to the production support system 30 together with the captured image. It is also possible that an additional device such as a smartphone is attached to the capture device 10, and information acquired by the additional device (for example, depth data, illuminance data, etc.) is transmitted to the capture device 10. The capture device 10 may also save the information acquired by the additional device as camera parameters and output this to the production support system 30 together with the captured image. The production support AP execution unit 122 may also select data to acquire, save, and output to the production support system 30 according to application information. That is, the production support AP execution unit 122 can acquire, save, and output data required for subsequent processing (here, asset generation or content generation), thereby reducing the power consumption, processing load, and data capacity of the capture device 10. Note that the attached data is not limited to the above-mentioned examples, and other data may also be considered, such as altitude information of the capture device 10, recognition results of the target object (subject), material information of the target object, etc.

[0033] Furthermore, the production support AP execution unit 122 may transmit the use information to the production support system 30 together with the captured image group and the attached data.

[0034] (Operation display section 140) The operation display unit 140 functions as an operation unit that accepts operation inputs from the user and as a display unit that displays various screens. For example, the operation display unit 140 may be realized by a touch panel display. Furthermore, the function as the operation unit and the function as the display unit may be realized separately, or an operation unit or a display unit may be provided separately from the operation display unit 140. The operation unit is realized by, for example, a touch sensor, a switch, a button, etc. The display unit may be realized by, for example, a display panel such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display.

[0035] (Storage unit 150) The storage unit 150 is realized by a storage medium that stores programs used in the processing of the control unit 120, calculation parameters, appropriately changing parameters, etc. The storage unit 150 according to this embodiment may store, for example, captured images and associated data.

[0036] Although the configuration of capture device 10 has been specifically described above, the configuration of capture device 10 according to the present disclosure is not limited to the example shown in Fig. 2. For example, capture device 10 may not have operation display unit 140, and may further have other sensors such as an illuminance sensor.

[0037] Furthermore, an information processing terminal (hereinafter also referred to as a user terminal) such as a smartphone that is connected to the capture device 10 for wireless or wired communication may be attached (fixed with an attachment device) to the capture device 10, and the production support AP execution unit 122 may be provided in the user terminal. In this case, the purpose selection screen, imaging support navigation, etc. may be displayed on the display unit of the user terminal.

[0038] The production support AP execution unit 122 provided in the user terminal may set camera parameters and control the shutter of the communicatively connected capture device 10 in accordance with application information acquired from user operations on the application selection screen. The user terminal may also acquire captured images and associated data from the capture device 10 and transmit (upload) them to the production support system 30. At this time, the user terminal may also transmit, as camera parameters, associated data acquired at the time of image capture by a sensor unit or the like provided in the user terminal (absolute position information by GPS or the like, altitude information, depth data, object recognition results, etc.) to the production support system 30.

[0039] In the above-described example, it has been described that the production support AP execution unit 122 is provided in the capture device 10 (or the user terminal), but in this embodiment, the production support AP execution unit 122 does not necessarily have to be provided in the capture device 10 (or the user terminal). As usual, the user may manually set camera parameters and control the shutter, and the obtained captured images may be output to the production support system 30. Furthermore, all image-related data that can be acquired by the capture device 10 may be output as ancillary data to the production support system 30 together with the captured images.

[0040] <<2-2. Configuration of the output device 20>> 3 is a block diagram showing an example of the configuration of the output device 20 according to this embodiment. As shown in FIG. 3, the output device 20 includes a communication unit 210, a control unit 220, an operation display unit 230, and a storage unit 240.

[0041] (Communication unit 210) The communication unit 210 includes a transmitting unit that transmits data to an external device and a receiving unit that receives data from the external device. The communication unit 210 according to this embodiment may be communicatively connected to an external device or the Internet using, for example, a wired or wireless LAN, Wi-Fi, Bluetooth, a mobile communication network, or the like.

[0042] (control unit 220) The control unit 220 functions as an arithmetic processing unit and a control unit, and controls the overall operation of the output device 20 in accordance with various programs. The control unit 220 is realized by an electronic circuit such as a CPU or a microprocessor. The control unit 220 may also include a ROM for storing the programs to be used, arithmetic parameters, etc., and a RAM for temporarily storing parameters that change as appropriate.

[0043] The control unit 220 also functions as a production support AP execution unit 222. The production support AP execution unit 222 executes a production support application installed in the output device 20. Specifically, the production support AP execution unit 222 displays a usage selection screen on the operation display unit 230, allowing the user to select what the content to be ultimately used for and how the content will be utilized, and acquires usage information. The production support AP execution unit 222 then transmits the usage information to the production support system 30 and, in cooperation with the production support system 30, generates assets and generates content using the generated assets. In cooperation with the production support system 30, the output device 20 is mainly used as a user interface for the production support system 30. The production support AP execution unit 222 may also upload sensing data (such as a group of captured images acquired by the capture device 10) used for asset generation to the production support system 30.

[0044] It is also possible that the usage information has already been selected at the time of acquiring the sensing data. The usage information may be shared with the output device 20 via the production support system 30.

[0045] Furthermore, the production support AP execution unit 222 may further have the functions of the above-mentioned production support AP execution unit 122. For example, if the output device 20 is realized by a smartphone or the like, the output device 20 may be connected to and communicate with the capture device 10, and may set camera parameters and control the shutter of the capture device 10 according to the application information.

[0046] The production support AP execution unit 222 may also control the display of assets and content generated by the production support system 30 on the operation display unit 230. Note that while it is assumed here that what is generated as assets and content is visual information, this is not limiting, and in the case of audio information, for example, the production support AP execution unit 222 may output audio information (such as stereophonic sound) generated as assets or content from an audio output unit (not shown). It is also assumed that the generated content may be video with audio.

[0047] (Operation display section 230) The operation display unit 230 functions as an operation unit that accepts operation inputs from the user and as a display unit that displays various screens. For example, the operation display unit 230 may be realized by a touch panel display. Furthermore, the function as the operation unit and the function as the display unit may be realized separately, or an operation unit or a display unit may be provided separately from the operation display unit 230. The operation unit is realized by, for example, a touch sensor, a switch, a button, etc. The display unit may be realized by, for example, a display panel such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display.

[0048] (Storage unit 240) The storage unit 240 is realized by a storage medium that stores programs used in the processing of the control unit 220, calculation parameters, parameters that change as appropriate, etc. The storage unit 240 according to this embodiment may store, for example, a group of captured images, attached data, usage information, assets, content, etc.

[0049] Although the configuration of the output device 20 has been specifically described above, the configuration of the output device 20 according to the present disclosure is not limited to the example shown in FIG. 3. For example, the output device 20 does not necessarily have all of the components shown in FIG. 3. Furthermore, the output device 20 may be realized by a plurality of devices. Furthermore, at least some of the functions of the output device 20 may be provided in a server on a network. For example, the output device 20 may further have an audio input unit, an audio output unit, a sensor unit, etc.

[0050] <<2-3. Configuration of Production Support System 30>> 4 is a block diagram showing an example of the functional configuration of a production support system 30 according to this embodiment. The production support system 30 may be realized by one or more servers, or by an information processing terminal such as a smartphone, a tablet terminal, or a PC, or by a user terminal such as a smartphone (an example of an information processing terminal) and a server. The production support system 30 may also be realized by one or more servers and the above-mentioned output device 20 (an example of an information processing terminal).

[0051] 4, the production support system 30 has functions as a captured image management unit 321, an asset generation support unit 322, a utilization support unit 323, a plug-in application acquisition unit 326, an account management unit 327, a license granting unit 328, and a sales management unit 329. Each of these functions may be realized by an application or a cloud service. Furthermore, each function is not necessarily realized by a single application or cloud service, but each function may be realized by various combinations, such as a group of applications, a combination of an application and a cloud service, or multiple cloud services.

[0052] (Captured image management unit 321) The captured image management unit 321 acquires captured images and manages them, such as by storing them. The captured images are uploaded to the production support system 30 from, for example, the capture device 10 or a user terminal (which may be the output device 20) such as a smartphone used by the user. The captured images may be accompanied by various types of additional data, such as camera parameters at the time of capture. The captured image management unit 321 is an example of an acquisition unit that acquires sensing data.

[0053] (Asset Creation Support Unit 322) The asset generation support unit 322 supports the generation of assets according to the usage information based on the sensing data. The usage information can be acquired from a user terminal (which may be the output device 20). The asset generation support unit 322 may cause the user terminal to display a usage selection screen and acquire information on user operations on the usage selection screen.

[0054] Various types of assets are envisioned to be generated by the asset generation support unit 322. Fig. 5 is a diagram showing an example of the functional configuration of the asset generation support unit 322 according to this embodiment. As shown in Fig. 5, the asset generation support unit 322 includes, for example, a 3D asset generation support unit 322A, a human asset generation support unit 322B, a motion asset generation support unit 322C, an audio asset generation support unit 322D, a 2D-3D video conversion support unit 322E, and an other asset generation support unit 322F.

[0055] The 3D asset generation support unit 322A generates 3D assets, which are 3DCG model data of various objects (landscapes, huge structures, buildings, indoor spaces, vehicles, furniture, clothing, accessories, food, plants, animals, etc.). The human asset generation support unit 322B generates human assets, which are 3DCG model data specialized for people and to which bone data is added.

[0056] The motion asset generation support unit 322C creates assets such as the movements of people, animals, and moving objects, and facial movements (generation of 3D motion). The audio asset generation support unit 322D generates sound effects, object audio, sound sources for spatial content, etc. The audio asset generation support unit 322D can also generate stereophonic sound (3D audio) as an audio asset.

[0057] The 2D-3D video conversion support unit 322E may convert 2D video into 3D video and generate a 3D video asset. The 2D video may be generated by, for example, a 2D video generation unit 324C described later. By converting to 3D video, the spatial content can be converted into an asset that can be used by replacing the background. The other asset generation support unit 322F generates various other assets. Note that the 2D-3D video conversion support unit 322E can also convert 2D video into 3D video. Furthermore, although the conversion target here is video, this is not limited thereto. The asset generation support unit 322 may also have a 2D-3D still image conversion support unit that converts 2D still images into 3D still images and 3D still images into 2D still images.

[0058] Each production support unit can generate assets according to usage information based on sensing data. The functions of each production support unit can be realized using one or more applications or cloud services as appropriate.

[0059] The asset generation support unit 322 determines what kind of asset to generate based on usage information indicating the usage desired by the user and controls the corresponding generation support unit to generate the asset. It is expected that asset generation conditions, such as desired quality (resolution, frame rate, sound quality, etc.), data volume, size, editability, and AI tolerance, may vary depending on the usage. When the usage is distribution, the type of asset to generate may be determined taking into consideration distribution conditions (data volume, etc.) specified by the distribution platform (distribution service). Distribution conditions may vary depending on the time of year, such as when an event is held. Furthermore, the asset generation support unit 322 may appropriately suggest asset types, etc., according to the usage and allow the user to select them, or may introduce asset types, etc., according to the usage to the user. For example, the asset generation support unit 322 may clearly indicate to the user what kind of asset will be generated for the usage selected by the user, for example, by displaying a sample 3D asset. Furthermore, the asset generation support unit 322 may clearly indicate to the user information regarding data necessary for generating such assets. The asset generation support unit 322 may also indicate to the user whether or not an asset corresponding to the intended use can be generated from the uploaded data (e.g., captured images). The asset generation support unit 322 may also select data from the uploaded data that can be used to generate an asset corresponding to the intended use and allow the user to select which data to use. If there is insufficient data to generate an asset corresponding to the intended use, the asset generation support unit 322 may also indicate to the user the missing data and prompt the user to add the data.

[0060] Each of the above-mentioned generation support units has multiple generation units (which can also be called generation functions or generation applications) that use different generation methods to generate assets. Even when generating the same type of asset (e.g., a 3D asset), there are appropriate generation methods depending on the application, and the asset generation support unit 322 controls the generation of an appropriate asset using an appropriate generation unit (appropriate generation method) based on application information.

[0061] More specifically, a generation method is a generation algorithm. Different generation algorithms result in different image quality types and generation speeds for the generated assets. The appropriate generation algorithm also differs depending on the size, image quality type, and editability (whether the asset is editable, etc.) of the asset to be generated. Generation algorithms that utilize generation AI are also envisioned. While the user can manually select a generation algorithm, the asset generation support unit 322 can automatically determine the generation algorithm to apply according to the usage information, allowing even users who are not familiar with production to obtain appropriate assets.

[0062] Below, the multiple different generation units will be specifically described using the 3D asset generation support unit 322A as an example. FIG. 6 is a diagram showing an example of the generation units included in the 3D asset generation support unit 322A according to this embodiment. As shown in FIG. 6, the 3D asset generation support unit 322A includes a first 3D asset generation unit 322A-1, a second 3D asset generation unit 322A-2, and a third 3D asset generation unit 322A-3. The 3D asset generation method used by each generation unit, i.e., the generation algorithm used to generate the 3D asset, is different for each generation unit.

[0063] For example, the first generation algorithm used by the 3D asset generation support unit 322A is an algorithm that generates 3D assets with the highest image quality (ultra-high quality) (slow generation speed). It is expected that the 3D assets generated by the first generation algorithm will be output as content (for example, uploaded to a metaverse production server) without being edited.

[0064] The second generation algorithm used by the second 3D asset generation unit 322A-2 is an algorithm that generates a 3D asset with medium image quality (and medium generation speed). The second generation algorithm may correct imaging irregularities (noise removal, uniform brightness, etc.) to generate a 3D asset with flat image quality. The second generation algorithm may also generate an editable 3D asset. There may be multiple patterns of the second generation algorithm.

[0065] The third generation algorithm used by the third 3D asset generation unit 322A-3 prioritizes generation speed, and is an algorithm that generates 3D assets faster than the first and second generation algorithms.

[0066] For example, when the application is background production for virtual production, the 3D asset generation support unit 322A generates a 3D asset using a first generation algorithm. Furthermore, when the application is, for example, film production, film prop production, game background production, game prop production, VR game production, or metaverse production, the 3D asset generation support unit 322A generates a 3D asset using a second generation algorithm. Note that a number of second generation algorithms may be prepared depending on differences in the size, image quality level, editability, and AI tolerance of the 3D asset to be generated, and the 3D asset generation support unit 322A may select an appropriate second generation algorithm depending on the application.

[0067] Furthermore, the 3D asset generation support unit 322A generates 3D assets using a third generation algorithm when the intended use is, for example, digital twin world production, digital twin world prop production, Previz production, or Previz prop production.

[0068] In addition, the 3D asset generation support unit 322A may select an appropriate generation algorithm from the first to third generation algorithms as appropriate when the application is 3D commerce production, 3D exhibition, 3D digital signage, advertising production, corporate video, digital archive, etc.

[0069] For example, the 3D asset generation support unit 322A may select an appropriate generation algorithm based on the object recognition results (scenery, buildings, cars, small items, etc.) of the captured image used for generation.

[0070] The 3D asset generation support unit 322A may also select an appropriate generation algorithm based on the camera position and orientation information of the captured images used for generation and the position and orientation information of the smartphone that performed the shooting navigation while attached to the camera. The 3D asset generation support unit 322A can determine how the user took the images from the camera position and orientation information of each captured image group, and can also estimate the size of the object from the shooting method, and can select an appropriate generation algorithm. For example, if a small item was photographed by surrounding it, the 3D asset generation support unit 322A may use a generation algorithm dedicated to generating a 3D model of the small item. Also, if the image was taken while moving horizontally, the 3D asset generation support unit 322A may use a generation algorithm dedicated to generating a 3D model of a landscape. Also, if the user moved within a space and took detailed photographs of various places, a photogrammetry algorithm may be used.

[0071] The specific examples of the generation algorithms described above are all examples. The generation algorithm selected when generating an asset may change due to technological advances in the generation algorithm. For example, a system update may update information regarding which algorithm to select when generating a certain type of asset. The production support system 30 already has information regarding which generation algorithm is appropriate for which purpose. The user may also arbitrarily change the default setting of the generation algorithm to be used, presented by the production support system 30. The user may also input conditions related to the asset to be generated, and the production support system 30 may select or generate an appropriate generation algorithm in accordance with the user input. Conditions related to the asset to be generated include, for example, size, image quality level, generation speed, editability, and AI tolerance.

[0072] Although three generation algorithms have been described above with reference to Fig. 6, the number of generation algorithms is not limited to this. Furthermore, the properties of each generation algorithm are not limited to the above examples. For example, a generation algorithm using a generation AI may be used.

[0073] The 3D asset generation support unit 322A can select from various generation algorithms a generation algorithm that generates an appropriate asset that matches the planned subsequent processing by obtaining information on subsequent processing, i.e., information on the intended use, such as the content to be generated and the output destination, in advance. This improves the efficiency of asset and content production and enhances user convenience.

[0074] (Utilization Support Department 323) The utilization support unit 323 supports the utilization of assets, specifically, the generation of content by the content generation unit 324 and the output of content by the content output unit 325.

[0075] The content generation unit 324 generates content based on one or more assets generated by the asset generation support unit 322. The content generation unit 324 supports the generation of content corresponding to the usage information. For example, the content generation unit 324 generates content to be used as a background for virtual production, content to be used as a prop for a game, etc.

[0076] 7 is a diagram showing an example of the generation units included in the content generation unit 324 according to this embodiment. As shown in FIG. 7, the content generation unit 324 includes a scene generation unit 324A, a preview generation unit 324B, a 2D video generation unit 324C, an asset and content management unit 324D, and a 3D rendering unit 324E. These are examples of multiple different utilization units.

[0077] The scene generation unit 324A generates a scene (video of a certain duration) using one or more assets. The scene generation unit 324A may generate a scene using LLM (Large Language Models) or AI. The conditions for the scene to be generated may be input by the user as appropriate. The previs generation unit 324B generates a previs to be used in video production using one or more assets. A previs is a video that is generated in video production using simple 3DCG video that is similar to the actual video before actual shooting. The 2D video generation unit 324C generates a 2D video using one or more assets. The conditions for the 2D video to be generated may be input by the user as appropriate.

[0078] The asset and content management unit 324D manages assets and content. For example, the asset and content management unit 324D stores generated assets and content in association with a user ID. The 3D rendering unit 324E performs 3D rendering based on one or more assets. For example, the 3D rendering unit 324E converts a 3D asset (3D model) into a 2D image.

[0079] The content output unit 325 outputs the content generated by the content generation unit 324 to the outside. The content output unit 325 is an example of an utilization unit. For example, the content output unit 325 may output the content to the output device 20. Furthermore, the content output unit 325 may output the content to an external cloud service 40. For example, the external cloud service 40 may be a metaverse creation service, a game creation service, or the like, and the content generation unit 324 may transmit the generated content to the external cloud service 40 in response to a user operation, thereby providing the content for metaverse creation or game creation. Furthermore, the external cloud service 40 may be a distribution service, and the content output unit 325 may distribute the content via the external cloud service 40. Note that the asset may be output as content as is.

[0080] (Plug-in application acquisition unit 326) The plug-in application acquisition unit 326 can acquire plug-in applications from the external plug-in application server 50 and extend the functionality of the production support system 30 .

[0081] (Account Management Department 327) The account management unit 327 manages the accounts of users who use the production support system 30. For example, the account management unit 327 stores a user ID, a login password, an email address, and the like in association with each other.

[0082] (Licensing Department 328) The license granting unit 328 grants the generated assets and content a license that guarantees that the assets and content were generated from data captured by a user (sensing data) and not from AI with unknown rights, i.e., guarantees the origin and authenticity of the digital data. Specifically, the license granting unit 328 may embed metadata into the assets or content as a license, including information such as when, where, and by whom the assets or content were created, the editing content, a digital signature for creator authentication, and a timestamp. Even when the assets and content are distributed, the authenticity of the assets and content can be guaranteed by the license.

[0083] (Sales Management Department 329) The sales management unit 329 manages sales of the generated assets and content. Specifically, the sales management unit 329 may transmit the generated assets and content to a store service server included in the external cloud service 40 and have them sold in a store. The sales management unit 329 may manage a user's sales license in association with a user ID.

[0084] Although the functions of the production support system 30 have been specifically described above, the functional configuration described with reference to FIG. 4 is merely an example, and the present disclosure is not limited thereto. The production support system 30 may further have various other functions. For example, the production support system 30 may generate various UIs, such as a usage selection screen to be displayed on the capture device 10 or the output device 20, and transmit them to the capture device 10 or the output device 20. The production support system 30 may also control setting parameters for the capture device 10 according to usage information. The production support system 30 may also present an image capture navigation to the user on the capture device 10 according to usage information.

[0085] <3. Operation processing> Fig. 8 is a diagram for explaining the overall processing flow according to this embodiment. As shown in Fig. 8, the processing executed in this embodiment roughly includes S1: acquiring captured images (sensing data), S2: generating assets based on the captured images, and S3: utilizing the assets (generating content using the assets and outputting the generated content).

[0086] In this embodiment, as an example, production efficiency can be improved by performing processing in an earlier stage according to a processing schedule in a later stage. That is, for example, in S1, usage information is acquired to acquire captured images according to the assets to be generated. Also, in S2, usage information is acquired to generate assets (using a corresponding generation algorithm) according to the content to be generated and the output destination.

[0087] 9 is a diagram showing an example of the flow of image capture processing according to this embodiment. As shown in Fig. 9, first, the capture device 10 acquires use information through a user operation on the displayed use selection screen (step S103).

[0088] Then, the capture device 10 performs an image capture support process in accordance with the acquired use information (step S106). Specifically, the capture device 10 sets camera parameters, controls the shutter, and presents image capture navigation to the user in accordance with the use information.

[0089] 10 is a diagram showing an example of the flow of production support processing according to this embodiment. As shown in Fig. 10, first, the production support system 30 acquires the captured image and associated data output from the capture device 10 (step S133).

[0090] Next, the production support system 30 acquires use information through a user operation on the displayed use selection screen (step S136). Note that the use information acquired before the image capture by the capture device 10 may be shared.

[0091] Next, the production support system 30 selects, in accordance with the use information, a generation algorithm suitable for generating assets to be used in the subsequent content generation (step S139).

[0092] Next, the production support system 30 generates the asset using the selected generation algorithm (step S142).

[0093] Next, the production support system 30 generates content using the one or more generated assets (step S145).

[0094] Then, the production support system 30 outputs the generated content (step S148).

[0095] The above describes the operation process according to the present embodiment. Note that the above process is an example, and the present disclosure is not limited to this. For example, not all of the above processes may be performed, and the order of some of the above processes may be changed.

[0096] <4.UI example> Next, specific examples of UIs presented to the user in the production support system 30 according to this embodiment will be described. Each UI may be displayed on the capture device 10 or the output device 20, which is an example of a user terminal, under the control of the production support system 30. Furthermore, each UI may be displayed by the production support AP execution units 122 and 222 on the capture device 10 or the output device 20.

[0097] <<4-1. Use Selection Screen>> FIG. 11 is a diagram showing an example of a use selection screen according to this embodiment. The use selection screen 410 shown in FIG. 11 displays options for asset utilization, such as VP (Visual Presentation) exhibition, video production, game production, digital twin, and metaverse. A user can select a use by selecting any option. The use selection screen 410 is displayed on the capture device 10 before acquiring sensing data, or on a user terminal after uploading sensing data and before generating assets in the production support system 30.

[0098] Although the use options on the use selection screen 410 shown in Fig. 11 are expressed in text, the present disclosure is not limited to this, and for example, the use options may be expressed in images. Fig. 12 is a diagram showing another example of a use selection screen according to this embodiment. As shown in Fig. 12, the use selection screen 420 shows the use options in images. Each image may be one that makes it easy to intuitively grasp the use (thumbnail, eye-catching image, icon, etc.).

[0099] Furthermore, the production support system 30 may estimate a use from a captured image uploaded by a user, and when displaying a use selection screen, may present the estimated use to the user in a state in which it is automatically selected as a preset. FIG. 13 is a diagram showing another example of the use selection screen according to this embodiment. As shown in FIG. 13, the use selection screen 430 is in a state in which the use estimated from the captured image (2D image) (in the example shown in FIG. 13, "VP exhibition") is selected. The user may select a use after checking the estimation result by the system.

[0100] Furthermore, the production support system 30 may present one or more uses estimated from the captured image uploaded by the user to the user as use options. FIG. 14 is a diagram showing another example of a use selection screen according to this embodiment. As shown in FIG. 14, the use selection screen 440 displays uses estimated from the captured image (2D image) (in the example shown in FIG. 14, "video production," "game production," and "metaverse"). The user selects any use from these.

[0101] <<4-2. Other selection screen>> In the above example, the production support system 30 generates assets and content according to the usage information, but the present disclosure is not limited to this. For example, if the user has some knowledge or the asset they want to generate is clear, the user may make more specific selections when generating assets.

[0102] Fig. 15 is a diagram showing an example of a size selection screen for a 3D asset to be generated according to this embodiment. As shown in Fig. 15, a selection screen 450 displays options for selecting the size of the 3D asset to be generated, such as a building (assuming a large structure), a landscape (assuming a space), or an object (assuming a small item). The production support system 30 determines a generation algorithm taking into account the size of the 3D asset to be generated selected by the user.

[0103] After the user selects the size of the 3D asset, the production support system 30 may transition to a usage selection screen and accept the usage selection. The production support system 30 then determines an appropriate generation algorithm in consideration of the size and usage of the 3D asset to be generated selected by the user.

[0104] FIG. 16 is a diagram showing an example of a selection screen for a generation algorithm used to generate a 3D asset according to this embodiment. As shown in FIG. 16, the selection screen 460 displays a selection of generation algorithms in text format. Examples of selectable generation algorithms include Photogrammetry, which constructs a 3D mesh of an object from a 2D image; Advanced Photogrammetry, which reproduces the shape of an object photographed from around it with high quality; Instant 3D Gaussian Splatting, which photorealistically converts reality into 3DCG at high speed; and Advanced 3D Gaussian Splatting, which reproduces photorealistic 3DCG of reality with ultra-high image quality. In this way, the user may be able to directly select a generation algorithm to use in generating a 3D asset. Note that the selection of generation algorithms is not limited to the text format shown in FIG. 16 and may be represented by images. The image format may be, for example, a display of a sample asset.

[0105] FIG. 17 is a diagram showing an example of a selection screen for the output format of a 3D asset according to this embodiment. As shown in FIG. 17, a selection screen 470 displays options for the output format of a 3D asset in text notation (e.g., "Mesh" and "3D Gaussian Splatting"). The production support system 30 determines a generation algorithm according to the output format of the 3D asset to be generated selected by the user. Note that the output format options are not limited to the text notation shown in FIG. 17, and may also be image notation such as a sample 3D asset, an eye-catching image, an icon, etc.

[0106] <<4-3. Use of AI and Paid Subscriptions>> It is also anticipated that if a use is selected on the use selection screen, the generation of the corresponding asset may be subject to AI use or a paid fee.

[0107] FIG. 18 is a diagram showing an example of a display screen including uses for which asset generation according to this embodiment will be subject to AI use or paid charges. Among the uses displayed on the selection screen 480 in FIG. 18 , for example, the option for "use as background for virtual production" displays a mark "PRO" indicating that the use will be subject to paid charges. Furthermore, for example, the option for "use in VR SNS" displays a mark "AI Support" indicating that processing using generation AI is included. This allows the user to intuitively understand that the generation of the corresponding asset will be subject to AI use or paid charges. Note that the display manner of each mark is merely an example, and this embodiment is not limited to this.

[0108] FIG. 19 is an example of a warning screen that is displayed when an option marked with the "Generating AI" mark is selected on the usage selection screen according to this embodiment. As shown in FIG. 19, the warning screen 500 displays a confirmation message such as, "This process will generate a 3D asset using a generating AI model provided by Company A. Are you sure?" If the user agrees to use the generating AI model, they tap the "Generate" button; if they do not agree, they tap the "Cancel" button.

[0109] <5. Variations> When generating assets, the production support system 30 may generate multiple assets using multiple generation algorithms, and then present a usage selection screen to accept the usage selection, and output the asset corresponding to the selected usage.

[0110] More specifically, first, the user uploads captured images captured by the capture device 10 to the production support system 30 by operating the upload screen. Next, the user selects one or more captured images to be used for asset generation from a list of uploaded captured images and taps the 3D generation button. When the 3D generation button is tapped, the production support system 30 starts generating multiple assets using multiple generation algorithms without prompting the user for a purpose at this point. After the assets are generated, a purpose selection screen is presented, and the user selects a purpose. The production support system 30 outputs (writes out) assets that correspond to the selected purpose from among the generated assets. Data for assets that were not output is deleted after a certain period of time.

[0111] Fig. 20 is a diagram showing an example of a use selection screen presented after asset generation according to this embodiment. As shown in Fig. 20, the use selection screen 520 displays use options along with a message such as "Please select the use of the 3D asset. We will perform the optimal file conversion depending on the use." In addition, a generation preview screen may be displayed on the use selection screen 520. The user can preview the generated image for each use, making it easier to select.

[0112] Furthermore, one advantage of selecting the purpose at a later stage is that the user can start generating assets immediately after uploading the captured images, which can contribute to efficient production.

[0113] 21 is a diagram showing an example of the flow of production support processing according to a modified example of this embodiment. As shown in Fig. 21, first, the production support system 30 acquires the captured image and associated data output from the capture device 10 (step S203).

[0114] Next, the production support system 30 generates assets using all generation algorithms (step S206).

[0115] Next, the production support system 30 displays a use selection screen and acquires use information based on a user operation (step S209).

[0116] Next, the production support system 30 selects an asset that corresponds to the usage information from the assets that have already been generated (step S212).

[0117] Next, the production support system 30 generates content using the one or more selected assets (step S215).

[0118] Then, the production support system 30 outputs the generated content (step S218).

[0119] <6. Hardware Configuration> An embodiment of the present disclosure has been described above. Next, an example of a hardware configuration used in the capture device 10, the output device 20, or the production support system 30 according to an embodiment of the present disclosure will be described with reference to FIG.

[0120] 22 is a block diagram showing an example hardware configuration of an information processing device 900 according to an embodiment of the present disclosure. The information processing device 900 is an example of a hardware configuration applied to the capture device 10, the output device 20, or the production support system 30 according to this embodiment. Note that the information processing device 900 does not necessarily have to have all of the hardware configuration shown in FIG.

[0121] 22 , the information processing device 900 includes a processing circuit 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925.

[0122] The processing circuit 901 functions as an arithmetic processing device and control device, and controls all or part of the operations within the information processing device 900 in accordance with various programs recorded in the ROM 902, RAM 903, storage device 919, or removable recording medium 927. The ROM 902 stores programs and calculation parameters used by the processing circuit 901. The RAM 903 temporarily stores programs used in the execution of the processing circuit 901 and parameters that change as appropriate during the execution. The processing circuit 901, ROM 902, and RAM 903 are interconnected by a host bus 907 formed by an internal bus. Furthermore, the host bus 907 is connected to an external bus 911, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 909.

[0123] The input device 915 is a device operated by a user, such as a button. The input device 915 may include a mouse, a keyboard, a touch panel, a switch, a lever, or the like. The input device 915 may also include a microphone that detects the user's voice. The input device 915 may be, for example, a remote control device that uses infrared or other radio waves, or an externally connected device 929 such as a mobile phone that supports operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs the signal to the processing circuit 901. The user operates the input device 915 to input various data to the information processing device 900 and to instruct processing operations.

[0124] The input device 915 may also include an imaging device and a sensor. The imaging device is a device that captures real space using various components, such as an imaging element, such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens for controlling the formation of a subject image on the imaging element, to generate a captured image. The imaging device may capture still images or videos. The sensor may be a range sensor, an acceleration sensor, a gyro sensor, a geomagnetic sensor, a vibration sensor, an optical sensor, or a sound sensor. The sensor acquires information about the status of the information processing device 900 itself, such as the orientation of the housing of the information processing device 900, and information about the surrounding environment of the information processing device 900, such as the brightness and noise around the information processing device 900. The sensor may also include a GPS (Global Positioning System) sensor that receives GPS signals to measure the latitude, longitude, and altitude of the device.

[0125] The output device 917 is configured with a device capable of visually or audibly notifying the user of acquired information. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or an audio output device such as a speaker or headphones. The output device 917 may also include a PDP (Plasma Display Panel), a projector, a hologram, a printer, or the like. The output device 917 outputs the results obtained by processing by the information processing device 900 as video such as text or images, or as sound such as voice or audio. The output device 917 may also include a lighting device that brightens the surroundings.

[0126] The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 919 is configured by, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores programs and various data executed by the processing circuitry 901, as well as various data acquired from the outside.

[0127] The drive 921 is a reader / writer for a removable recording medium 927 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and is built into or externally attached to the information processing device 900. The drive 921 reads information recorded on the attached removable recording medium 927 and outputs the information to the RAM 905. The drive 921 also writes information to the attached removable recording medium 927.

[0128] The connection port 923 is a port for directly connecting a device to the information processing device 900. The connection port 923 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. The connection port 923 may also be an RS-232C port, an optical audio terminal, or an HDMI (registered trademark) (High-Definition Multimedia Interface) port. By connecting an external device 929 to the connection port 923, various types of data can be exchanged between the information processing device 900 and the external device 929.

[0129] The communication device 925 is, for example, a communication interface configured with a communication device for connecting to the external network 70. The communication device 925 may be, for example, a communication card for a wired or wireless local area network (LAN), Bluetooth (registered trademark), Wi-Fi (registered trademark), or wireless USB (WUSB). The communication device 925 may also be a router for optical communication, a router for asymmetric digital subscriber line (ADSL), or a modem for various types of communication. The communication device 925 transmits and receives signals, for example, between the Internet and other communication devices using a predetermined protocol such as TCP / IP. The external network 70 connected to the communication device 925 is a network connected by wire or wirelessly, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication.

[0130] For example, when the information processing device 900 functions as the capture device 10, the output device 20, or the production support system 30 according to an embodiment of the present disclosure, the processing circuit 901 of the information processing device 900 executes a program loaded on the RAM 903 to function as each function of the control unit 120, the control unit 220, or the production support system 30 (captured image management unit 321, asset generation support unit 322, utilization support unit 323, plug-in application acquisition unit 326, account management unit 327, license granting unit 328, sales management unit 329). In addition, the storage device 919 stores the information processing program according to the present disclosure and various data stored in the memory unit 150, the memory unit 240, or a memory unit (not shown) included in the production support system 30.

[0131] The processing circuit 901 reads and executes program data from the storage device 919, but as another example, the processing circuit 901 may obtain these programs from another device via the external network 70. That is, the storage device 919 is not limited to being located inside the information processing device 900, but may be located outside the information processing device 900.

[0132] The processing circuit 901 is an example of an integrated circuit, and a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array) can all be considered to be integrated circuits.

[0133] Furthermore, when the information processing device 900 functions as the capture device 10, the output device 20, or the production support system 30 according to an embodiment of the present disclosure, the communication device 925 corresponds to the communication unit 110, the communication unit 210, or a communication unit (not shown) included in the production support system 30. Furthermore, the input device 915 and the output device 917 correspond to the operation display unit 140 or the operation display unit 230. Furthermore, the input device 915 corresponds to the sensor unit 130.

[0134] <7. Supplementary Information> Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the present technology is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0135] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0136] The information processing system according to the present disclosure may be composed of a single device (such as an information processing terminal or a server) or may be composed of multiple devices, which may include, for example, an information processing terminal and one or more servers.

[0137] Furthermore, the above-described embodiments and modifications of the present disclosure can be combined as appropriate within the scope of the processing content. Furthermore, the order of the steps shown in the sequence diagrams or flowcharts of the present embodiments can be changed as appropriate. For example, the steps can be processed in chronological order, repeatedly, or partially in parallel.

[0138] It is also possible to create one or more computer programs for causing hardware such as a CPU, ROM, and RAM built into the capture device 10, output device 20, or production support system 30 to perform the functions of the capture device 10, output device 20, or production support system 30. A computer-readable storage medium storing the one or more computer programs is also provided.

[0139] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0140] The present technology can also be configured as follows. (1) a creation support unit having a plurality of different creation units that create assets; a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support unit; Equipped with The asset used in the utilization unit is an asset corresponding to the utilization unit and generated by any of the generation units. Information processing system. (2) The information processing system according to (1), wherein the plurality of different generation units each use a different generation method for generating the asset. (3) The information processing system described in (2), wherein the plurality of different generation units generate assets that differ in at least one of size, image quality level, generation speed, editability, and AI tolerance. (4) The information processing system described in any one of (1) to (3), wherein the generation support unit determines the generation unit that will generate the corresponding asset based on usage information indicating the intended use of the asset desired by the user, and generates the asset using the determined generation unit. (5) the generation support unit generates a plurality of assets using a plurality of generation units, The information processing system described in any one of (1) to (3), wherein the utilization support unit selects a corresponding asset from the plurality of assets based on usage information indicating the usage purpose desired by the user, and utilizes the selected asset. (6) The information processing system according to any one of (1) to (5), wherein the asset is a 3D asset, a person asset, a motion asset, an audio asset, or a 3D video. (7) the plurality of different generation units use different generation methods for one type of asset among the different types of assets, The information processing system according to (6), wherein each asset of a type generated by each generation unit differs in at least one of size, image quality type, generation speed, editability, and AI tolerance. (8) The information processing system further includes an acquisition unit that acquires sensing data of an object, The information processing system according to any one of (1) to (7), wherein the generation support unit generates the asset using the generation unit based on the sensing data. (9) the sensing data is one or more captured images, The information processing system according to (8), wherein the generation support unit generates the asset based on one or more of the captured images. (10) the information processing system further includes an imaging support unit that supports imaging of the object, the imaging support unit performs a process of supporting imaging to acquire sensing data to be used for generating a corresponding asset in accordance with usage information indicating a usage purpose desired by the user; The information processing system according to (9) above. (11) The information processing system according to (10), wherein the imaging support unit performs camera parameter setting, shutter control, or imaging position and orientation navigation. (12) The information processing system according to any one of (1) to (11), wherein the utilization unit generates content using one or more of the assets generated by the generation unit. (13) The information processing system according to any one of (1) to (12), wherein the utilization unit outputs the asset or content generated using one or more of the assets. (14) The information processing system according to (12) or (13), wherein the utilization support unit determines the utilization unit according to the intended use based on the intended use information indicating the intended use of the utilization desired by the user, and utilizes the asset using the determined utilization unit. (15) The information processing system according to any one of (12) to (14), wherein each of the different types of content generated by each of the different utilization units is a scene, a video for video production simulation, a 2D video, or a 3D model. (16) The information processing system according to any one of (12) to (15), wherein the utilization unit generates the content that satisfies a distribution condition of an output destination indicated by application information. (17) The information processing system according to any one of (1) to (16), wherein the information processing system presents a display screen that accepts a selection input of an application for utilizing the asset. (18) The information processing system according to (17), wherein the use options displayed on the display screen include a mark indicating that the asset generation is subject to AI use or a paid charge. (19) The processor: generating assets by at least one of a plurality of different generators; Utilizing the asset generated by the generation unit by at least one of a plurality of different utilization units; Including, The asset used in the utilization unit is an asset corresponding to the utilization unit and generated by any of the generation units. Information processing methods. (20) Computer, a communication unit that communicates with a system including a generation support unit having a plurality of different generation units that generate assets, and a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support unit, wherein the assets used by the utilization units are assets generated by any of the generation units and corresponding to the utilization units; a control unit that controls the transmission of sensing data used for the generation of the asset by the generation unit to the system, and acquires information on the result of the generation of the asset by the generation unit and the result of utilization by the utilization unit from the system; A program to function as a [Explanation of symbols]

[0141] 10 Capture Devices 110 Communications Department 120 control section 122 Production Support AP Executive Department 130 Sensor unit 140 Operation display section 150 Storage section 20 Output Devices 210 Communications Department 220 Control Unit 222 Production Support AP Executive Department 230 Operation display section 240 Storage section 30 Production Support System 321 Captured Image Management Unit 322 Asset Creation Support Department 322A 3D Asset Creation Support Department 322A-1 First 3D asset generation unit 322A-2 Second 3D asset generation unit 322A-3 Third 3D Asset Generation Unit 323 Utilization Support Department 324 Content Generation Department 325 Content Output Unit 326 Plug-in App Acquisition Department 327 Account Management Department 328 Licensing Department 329 Sales Management Department 40 External Cloud Services 50 External Plug-in Application Server

Claims

1. a creation support unit having a plurality of different creation units that create assets; a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support unit; Equipped with the generation support unit determines the generation unit that generates the asset corresponding to the intended use based on use information indicating the intended use of the asset desired by the user; the utilization support unit determines the utilization unit corresponding to the intended use based on the intended use information. Information processing system.

2. The information processing system according to claim 1 , wherein the plurality of different generators each use a different generation method for generating the asset.

3. The information processing system according to claim 2 , wherein the plurality of different generators generate assets that differ in at least one of size, image quality level, generation speed, editability, and AI tolerance.

4. the generation support unit generates a plurality of assets using a plurality of generation units, The information processing system according to claim 1 , wherein the utilization support unit selects a corresponding asset from the plurality of assets based on the usage information, and utilizes the selected asset.

5. The information processing system of claim 1 , wherein the asset is a 3D asset, a person asset, a motion asset, an audio asset, or a 3D video.

6. the plurality of different generation units use different generation methods for one type of asset among the different types of assets, The information processing system according to claim 5 , wherein the assets of one type generated by each generation unit differ in at least one of size, image quality type, generation speed, editability, and AI tolerance.

7. The information processing system further includes an acquisition unit that acquires sensing data of an object, The information processing system according to claim 1 , wherein the generation support unit generates the asset using the generation unit based on the sensing data.

8. the sensing data is one or more captured images, The information processing system according to claim 7 , wherein the generation support unit generates the asset based on one or more of the captured images.

9. the information processing system further includes an imaging support unit that supports imaging of the object, the imaging support unit performs processing to support imaging so as to acquire sensing data to be used in generating a corresponding asset according to the usage information; The information processing system according to claim 8 .

10. The information processing system according to claim 9 , wherein the imaging support unit performs camera parameter setting, shutter control, or imaging position and orientation navigation.

11. The information processing system according to claim 1 , wherein the utilization unit generates content using one or more of the assets generated by the generation unit.

12. The information processing system according to claim 1 , wherein the utilization unit outputs the asset or content generated using one or more of the assets.

13. The generation support unit generates the asset using the determined generation unit, The information processing system according to claim 11 , wherein the utilization support unit utilizes the asset through the determined utilization unit.

14. The information processing system according to claim 11 , wherein each of the different types of content generated by each of the plurality of different utilization units is a scene, a video for video production simulation, a 2D video, or a 3D model.

15. The information processing system according to claim 11 , wherein the utilization unit generates the content that satisfies a distribution condition for an output destination indicated by the usage information.

16. The information processing system according to claim 1 , wherein the information processing system presents a display screen that accepts a selection input of an application for utilizing the asset.

17. The information processing system according to claim 16, wherein the use options displayed on the display screen include a mark indicating that the asset generation is subject to AI use or is subject to a fee.

18. The processor: generating assets by at least one of a plurality of different generators; Utilizing the asset generated by the generation unit by at least one of a plurality of different utilization units; Including, The processor: determining the generating unit to generate assets corresponding to the intended use based on use information indicating the intended use of the use desired by the user; determining the utilization part according to the purpose based on the purpose information; further comprising: Information processing methods.

19. A generation support unit having a plurality of different generation units that generate assets; a utilization support unit having a plurality of different utilization units that utilize the assets generated by the generation support unit; Equipped with the asset used in the utilization unit is an asset corresponding to the utilization unit and generated by any of the generation units, presenting a display screen for accepting a selection input of an application for utilizing the asset; An information processing system in which the use options displayed on the display screen display a mark indicating that the generation of the asset is subject to AI use or is subject to a paid fee.

Images