Information processing system, information processing method, and program

The information processing system enhances user convenience by generating and displaying 3D assets with associated metadata, addressing the lack of metadata utilization in existing technologies.

WO2026234917A1PCT designated stage Publication Date: 2026-11-12SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2026/012257
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2026-03-26
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing technologies do not effectively utilize metadata obtained from image analysis for enhancing user convenience when viewing generated 3D assets.

Method used

An information processing system that generates 3D assets and displays a list display screen showing the assets along with assigned metadata, allowing users to intuitively understand asset information.

Benefits of technology

Improves user convenience by providing detailed metadata and tags with the 3D assets, facilitating easier navigation and utilization of the assets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2026012257_12112026_PF_FP_ABST
    Figure JP2026012257_12112026_PF_FP_ABST
Patent Text Reader

Abstract

[Problem] To allow an improvement in user convenience in viewing generated assets. [Solution] This information processing system comprises: a generation unit that generates assets of an object; and a display control unit that displays a list display screen of the assets. On the list display screen, the assets and metadata assigned to the assets in a step for generating the assets are displayed.
Need to check novelty before this filing date? Find Prior Art

Description

Information Processing System, Information Processing Method, and Program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

[0002] In recent years, 3D assets are generated from a plurality of 2D images, and the generated 3D assets are used for production of various types of content. Specifically, a plurality of 2D images are analyzed to generate a 3D model.

[0003] Regarding image processing, for example, Patent Document 1 discloses a technology for analyzing an image to acquire metadata, and applying an effect to the image based on the acquired metadata.

[0004] Japanese Unexamined Patent Application Publication No. 2014-6912

[0005] However, in Patent Document 1, the metadata obtained from the image analysis result is used for effect processing, and is not utilized when the image is viewed thereafter.

[0006] Accordingly, the present disclosure proposes an information processing system, an information processing method, and a program that can enhance user convenience when viewing generated assets.

[0007] According to the present disclosure, there is provided an information processing system comprising: a generation unit that generates an asset of a target object; and a display control unit that displays a list display screen of the assets, wherein the list display screen displays the assets and metadata assigned to the assets in the generation process of the assets.

[0008] Further, according to the present disclosure, there is provided an information processing method, by a processor, comprising: generating an asset of a target object; and displaying a list display screen of the assets, wherein the list display screen displays the assets and metadata assigned to the assets in the generation process of the assets.

[0009] Furthermore, according to this disclosure, a computer is provided that functions as a generation unit for generating assets of an object and a display control unit for displaying a list of the assets, wherein the list display screen displays the assets and metadata assigned to the assets during the asset generation process.

[0010] This figure shows an example of the configuration of an information processing system 1 according to one embodiment of the present disclosure. This figure shows an overview of the production support processing flow according to this embodiment. This block diagram shows an example of the functional configuration of a production support system 30 according to this embodiment. This flowchart shows an example of the operation processing flow according to this embodiment. This figure shows an example of a tag selection screen 400 according to this embodiment. This figure shows an example of a preview video generation screen 500 according to this embodiment. This flowchart shows an example of the asset search processing flow according to this embodiment. This figure shows an example of an asset list screen 510 with personalized search results. This flowchart shows an example of a search process including sorting according to this embodiment. This figure shows an example of user information registered during user registration. This block diagram shows an example of the hardware configuration of an information processing device 900 according to one embodiment of the present disclosure.

[0011] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0012] Furthermore, the explanation will be conducted in the following order: 1. Overview 2. Configuration of Production Support System 30 3. Operation Processing 4. Hardware Configuration 5. Supplementary Information

[0013] <1. Overview> An information processing system according to one embodiment of the present disclosure has the function of generating assets based on sensing data obtained by sensing an object, and displaying a list of the generated assets.

[0014] For example, an information processing system generates a 3D asset of an object based on multiple images taken from multiple viewpoints of the object, and displays a list of the 3D assets. Furthermore, by displaying various metadata attached to the 3D assets on the 3D asset list screen, the information processing system makes it easier to intuitively understand information about the 3D assets and improves the convenience of using the 3D assets.

[0015] Figure 1 is a diagram showing an example of the configuration of an information processing system 1 according to one embodiment of the present disclosure. As shown in Figure 1, the information processing system 1 includes a capture device 10 for sensing an object, a production support system 30 for generating assets based on sensing data, and an output device 20 for outputting the assets generated by the production support system 30.

[0016] The capture device 10 is a device that acquires sensing data of an object. Examples of capture devices 10 include an imaging device that takes images of the object, a measuring device that measures the distance to the object and the shape of the object, a motion capture device that tracks the movement of the object, and a sound collection device that acquires sound from the object, its surroundings, and the surrounding space.

[0017] The capture device 10 may be a dedicated device (for example, a mirrorless camera, a ToF (Time Of Flight) camera, a 3D laser scanner, a LiDAR (Light Detection and Ranging) device, a recording device, etc.), or it may be implemented using a general-purpose device such as a smartphone, smartwatch, or tablet terminal.

[0018] Furthermore, the capture device 10 may consist of multiple devices. For example, the capture device 10 may be a device in which a smartphone is attached to a camera using an adapter, and the camera and smartphone are connected via communication. The camera's through-image may be displayed on the smartphone, and camera operation may be guided by an application running on the smartphone.

[0019] The sensing data of the object acquired by the capture device 10 is input to the production support system 30. The sensing data of the object may be uploaded from the capture device 10 to the production support system 30 via a network, or it may be transmitted to the production support system 30 via a wired connection or other device.

[0020] The output device 20 is an example of an information processing terminal that outputs various types of information. The output device 20 can be implemented as, for example, a smartphone, smartwatch, tablet terminal, PC (Personal Computer), HMD (Head Mounted Display), stereoscopic display device, large display device, projector, etc. The output device 20 also has an operation input function. The output device 20 may be used as a user interface when assets are generated and utilized in the production support system 30. Alternatively, the output device 20 may be used as a playback device that plays back content generated by the production support system 30.

[0021] The production support system 30 may be formed from one or more servers. Each function of the production support system 30 may be implemented by one or more applications or one or more cloud services. Multiple applications and multiple cloud services may be linked together as appropriate.

[0022] Specifically, the production support system 30 has the function of generating 3D assets based on sensing data acquired by the capture device 10. The production support system 30 also has the function of outputting a list display screen of the generated 3D assets.

[0023] Furthermore, the production support system 30 may also include a function for utilizing the generated assets. Various uses of assets are envisioned, such as posting to SNS (Social Networking Service), content creation, uploading to external cloud services, and distribution from external distribution platforms. Examples of content creation include virtual production, film production, film prop production, game background production, game prop production, VR game production, metaverse production, digital twin world production, digital twin world prop production, Pre-visualization (Pre-visualization) production, Pre-visualization prop production, 3D commerce production, 3D exhibit production, 3D digital signage production, advertising production, corporate videos, and digital archives. In addition, asset utilization also includes generating new assets (content) by combining multiple assets.

[0024] The production support system 30 communicates with an external cloud service (not shown) and may, as appropriate, utilize services provided by the external cloud service for asset generation and utilization.

[0025] The production support system 30 may communicate with an external plugin application server (not shown) and, as appropriate, acquire and use applications from the external plugin application server for asset generation and utilization. This allows for functional expansion of the production support system 30.

[0026] Figure 2 is a diagram illustrating the overview of the production support process according to this embodiment. As shown in Figure 2, this process is broadly classified into three steps.

[0027] First, in step S1, the capture device 10 senses the target object and acquires sensing data, including captured images. The sensing data acquired here may include still images or videos, as well as supplementary information such as depth data, position and orientation information, and camera parameters.

[0028] Specific examples of supplementary information include lens metadata, body metadata, and accessory metadata. Lens metadata includes, for example, lens product information (such as product serial number), distortion correction information, chromatic aberration correction information, and monochromatic aberration correction information. Body metadata includes, for example, camera product information (such as product serial number), calibration results (imager position, rotation information), f-number, ISO, WB (white balance), color information, IMU data, image stabilization information, focal length, exposure, subject distance, depth data (depth image), and location information (relative positioning, absolute positioning).

[0029] Accessory metadata is information obtained from accessories attached to the camera (e.g., smartphones), specifically including posture information, GPS (Global Positioning System) information, AR processing information (plane detection results, object recognition results, etc.), depth data, material information (surface texture, reflectivity, color, texture, material information, etc.), altitude information, and other environmental information (e.g., illuminance information, rainfall information, wind speed information, sound information). Depth data is obtained from, for example, a TOF or LiDAR installed in a smartphone. More specifically, the distance to the subject obtained in accordance with the depth image and depth data is obtained as accessory metadata.

[0030] Next, in step S2, assets are generated using the acquired sensing data. Examples of the generated assets include 3D assets, human assets, motion assets, or audio assets. The assets may also be generated in a form appropriate to the purpose of use (i.e., application). For example, the data format, size, data capacity, resolution, clarity, or texture of the assets may be generated to suit the application.

[0031] Next, in step S3, the generated assets are utilized. In this step, a user interface is provided through the output device 20, and content is created using the assets, the assets are uploaded to an external cloud service, and the assets are distributed or sold.

[0032] The information processing system 1 according to this embodiment can support a series of processes from acquiring captured images (S1) to generating assets (S2) and further to utilizing those assets (S3). For example, by performing the preceding processing according to the planned processing in the subsequent stages, production efficiency can be improved. For instance, by acquiring information regarding the intended use of the assets in S1, it is possible to acquire captured images useful for generating assets according to their intended use, thereby achieving efficient asset generation.

[0033] The user can view a list of the generated 3D assets via the output device 20. Here, the 3D asset list screen displays thumbnail images of the 3D assets, allowing the user to recognize them by viewing the thumbnails. However, detailed information about the 3D assets may not be obtainable from the thumbnail images alone.

[0034] Therefore, in the information processing system 1 according to this embodiment, the 3D asset list display screen displays the 3D asset and the metadata attached to the 3D asset, thereby improving the convenience of using the 3D asset.

[0035] The following describes the functional configuration and operation of the production support system 30 included in the information processing system 1.

[0036] <2. Configuration of the Production Support System 30> Figure 3 is a block diagram showing an example of the functional configuration of the production support system 30 according to this embodiment. The production support system 30 may be implemented by one or more servers, by an information processing terminal such as a smartphone, tablet terminal, or PC, or by a user terminal such as a smartphone (an example of an information processing terminal) and a server. Furthermore, the configuration of the production support system 30 may be implemented by a server and an output device 20, or by an output device 20 (an example of an information processing terminal).

[0037] As shown in Figure 3, the production support system 30 has the functions of a display control unit 300, an image capture management unit 310, an asset generation unit 320, a metadata assignment unit 330, a tagging processing unit 340, a preview video generation unit 350, and a search unit 360. Each of these functions may be implemented by an application or a cloud service. Furthermore, a single function is not necessarily implemented by a single application or cloud service; each function can be implemented by various combinations, such as a group of applications, a combination of an application and a cloud service, or multiple cloud services.

[0038] (Display Control Unit 300) The display control unit 300 controls the display of various screens to the user. For example, the display control unit 300 transmits information about the display screen to the output device 20.

[0039] The display control unit 300 controls the display of the image upload screen, the operation screen for asset generation, the screen displaying a list of generated assets, and so on.

[0040] (Image Management Unit 310) The Image Management Unit 310 acquires captured images from an external device and manages the captured images, including saving them. The captured images are uploaded to the production support system 30 from, for example, the capture device 10 or a user terminal such as a smartphone used by the user (which may also be the output device 20). Various supplementary information such as depth data and camera parameters at the time of acquisition may be attached to the captured images. The Image Management Unit 310 is an example of an acquisition unit that acquires sensing data.

[0041] (Asset generation unit 320) The asset generation unit 320 generates assets based on sensing data. The types of assets that can be generated by the asset generation unit 320 include 3D models, as well as audio (3D audio), video, motion data (3D motion), etc.

[0042] The asset generation unit 320 may generate an asset corresponding to the usage based on usage information input by a user. For example, the asset generation unit 320 generates an asset that satisfies a predetermined condition set for each usage based on the input usage information. Examples of the predetermined condition include the size of the asset, data capacity, image quality level, editability, and AI tolerance.

[0043] The asset generation unit 320 can generate 3D assets that are 3DCG model data of various objects (landscapes, huge structures, buildings, indoor spaces, vehicles, furniture, apparel goods, small items, food, plants, animals, etc.). The asset generation unit 320 can also generate a human asset to which bone data is assigned as 3DCG model data specialized for humans.

[0044] The asset generation method in the asset generation unit 320 is not particularly limited. For example, the asset generation unit 320 may select a generation algorithm according to the usage, the type and size of the asset to be generated, and the like, to perform asset generation. Appropriate generation algorithms differ depending on the size of the asset to be generated, image quality type, editability (such as whether the asset is editable or not), and the like. In addition, the generation speed and the like may vary depending on the difference in the generation algorithm used. Furthermore, a generation algorithm that uses generative AI is also envisioned. The type (landscape, building, car, small item, etc.) and size of the asset to be generated can be grasped from the object recognition result of the captured image used for generation.

[0045] The generated 3D asset can be stored in a storage unit (not shown) of the production support system 30. The production support system 30 accumulates each generated asset in the storage unit in association with the user.

[0046] (Metadata adding unit 330) The metadata adding unit 330 performs processing for adding metadata to the generated 3D asset. The metadata adding unit 330 may add, as metadata, the attached information of the captured image used for generating the 3D asset. As described above, the attached information of the captured image includes information obtained at the time of image capturing. Further, the metadata adding unit 330 may add, as metadata, the information obtained in the generation step. Hereinafter, specific examples of the metadata will be described.

[0047] The attached information of the captured image is, for example, depth data, position information, orientation information, camera parameters, environment information (for example, sunlight exposure, weather, wind direction), and the like. The camera parameters include, for example, lens metadata and body metadata.

[0048] The information obtained in the generation step is, for example, an asset type, an object recognition result, a scene recognition result, asset usage information, an asset format type, information on a generation algorithm used for generation, the SW (software) version of the present system at the time of generation, XYZ direction information of the asset, floor surface information (position and direction of the floor surface), origin information, and the like.

[0049] The metadata added to the asset is displayed by the display control unit 300 in association with (the thumbnail image of) the asset on the asset list display screen, and is presented to the user. Accordingly, when viewing the list display screen, the user can intuitively grasp the information related to the 3D asset. While a large amount of metadata is added to the asset, the metadata displayed on the list display screen by the display control unit 300 may be metadata with a preset high priority.

[0050] (Tag processing unit 340) The tag processing unit 340 performs processing for tagging the metadata of the asset. In the present specification, tagging means converting an expression of metadata that is difficult for a user to understand into an expression that is easy to recognize. A tag can also be referred to as converted metadata. The tag processing unit 340 adds the newly generated tag to the asset.

[0051] For example, the tagging processing unit 340 generates tags such as addresses and facility names on a map based on GPS coordinates, which are metadata. The tagging processing unit 340 also generates tags such as the lens manufacturer and product name (series name, etc.) or the camera manufacturer and product name (series name, etc.) based on the product serial number, which is metadata. Furthermore, the tagging processing unit 340 converts the filename of the generation algorithm used into a predetermined algorithm name and generates it as a tag. Note that some metadata can be understood by the user without tagging, so tagging may be performed only on some metadata.

[0052] Furthermore, the tagging processing unit 340 may assign tags arbitrarily selected by the user.

[0053] The tags assigned to an asset are displayed on the asset list screen by the display control unit 300 and presented to the user. This allows the user to intuitively grasp information about the 3D asset when viewing the list screen. While an asset may have numerous metadata and tags assigned to it, the metadata and tags displayed on the list screen by the display control unit 300 may be those with a pre-set priority.

[0054] (Preview video generation unit 350) The preview video generation unit 350 generates a preview video of an asset based on the metadata assigned to the asset, using a camera path, field of view, effects, motion, or background effects suitable for the asset. Such a video may be displayed as a thumbnail image on the list display screen.

[0055] The preview video generation unit 350, for example, if the asset is a robot with the usage information "combat scene," will generate a video that focuses on the weapons worn by the robot. Furthermore, if the usage information includes information about the posting medium (SNS), the preview video generation unit 350 may consider the playable duration and aspect ratio on that SNS, and may also generate a video in a format that is popular or likely to gain support on that SNS. Additionally, if the asset has the metadata "AI generated," the preview video generation unit 350 may generate a video that avoids the AI-generated portion, as this portion may be of low quality.

[0056] While this discussion has focused on generating preview videos based on metadata attached to assets, the production support system 30 is not limited to generating preview videos. It may also generate still images as thumbnail images based on the metadata, using appropriate angles, effects, or backgrounds for the assets.

[0057] (Search Unit 360) The search unit 360 searches for assets based on the search words entered by the user. Specifically, the search unit 360 searches for assets that have metadata or tags that match the search words. The search may be performed on the asset list display screen.

[0058] Furthermore, the search unit 360 may perform a search by adding one or more other search words based on user information. This allows for personalization of the search results.

[0059] Although the functions of the production support system 30 according to this embodiment have been described in detail above, the functional configuration described with reference to Figure 3 is just one example, and this disclosure is not limited thereto. The production support system 30 may have various other functions. For example, the production support system 30 may have utilization functions that make use of one or more assets. Utilization functions include, for example, content creation functions, SNS linking functions, distribution functions, and sales functions.

[0060] Furthermore, as described above, the output device 20 may also be used as a user interface when generating and utilizing assets in the production support system 30. Users can also show the asset list screen displayed on the output device 20 to representatives of film production companies, advertising production companies, etc., to conduct business negotiations.

[0061] <3. Operation Processing> Figure 4 is a flowchart showing an example of the operation processing flow according to this embodiment.

[0062] First, in step S103, the asset generation unit 320 of the production support system 30 generates an asset of the target object based on multiple captured images of the target object taken from multiple viewpoints. At this time, the asset generation unit 320 may select a generation algorithm to be used for asset generation from the object recognition results, scene recognition results, etc. of the captured images. Alternatively, the asset generation unit 320 may select a generation algorithm according to the usage information input by the user.

[0063] Next, in step S106, the metadata assignment unit 330 assigns metadata to the asset. For example, the metadata assignment unit 330 assigns supplementary information of the captured image used to generate the asset (depth data, location information, orientation information, environmental information, lens metadata, body metadata, etc.) to the asset. The metadata assignment unit 330 may also assign information obtained during asset generation, object recognition results, scene recognition results, usage information, etc., to the asset.

[0064] Next, in step S109, the metadata assignment unit 330 assigns tags to the asset. For example, the metadata assignment unit 330 assigns tags generated by converting the metadata of the asset to the asset. Alternatively, the metadata assignment unit 330 assigns tags selected by the user to the asset.

[0065] Figure 5 shows an example of the tag selection screen 400 according to this embodiment. Tag candidates are displayed on the tag selection screen 400 shown in Figure 5, and the user can assign a tag to an asset by selecting any tag candidate. The tag candidates displayed on the tag selection screen 400 may be presented by the tagging processing unit 840 based on, for example, the object recognition result of the captured image, the scene recognition result, or the shape analysis result of the asset.

[0066] Next, in step S112, the preview video generation unit 350 generates a preview video of the asset based on the metadata or tags attached to the asset. For example, the preview video generation unit 350 may generate a preview video appropriate for its intended use, depending on the intended use information such as for movies, games, metaverse, distribution sites, and various social networking services. Specifically, the preview video generation unit 350 generates a preview video with an aspect ratio, compression ratio, or data format that matches the intended use. The preview video generation unit 350 may also generate a preview video based on user input.

[0067] Figure 6 shows an example of the preview video generation screen 500 according to this embodiment. As shown in Figure 6, the preview video generation screen 500 includes an asset display screen 502, an instruction selection screen 503, and an SNS posting operation screen 504. The instruction selection screen 503 lists items to be reflected in the preview video as selection candidates. For example, it is possible to specify the genre of the preview video to be generated, such as action, horror, comedy, thriller, or fantasy. The preview video generation unit 350 generates a preview video with camera flow, direction, field of view, etc., according to the genre selected by the user.

[0068] The user's selection options are not limited to genre; for example, they can also select famous photographers or film directors. The preview video generation unit 350 may generate a preview video using the characteristic flash flow, direction, and framing of the photographer or film director selected by the user. Media can also be selected by the user. For example, movies, games, metaverse, streaming sites, and various social networking services can be selected, and the preview video generation unit 350 may generate a preview video corresponding to the media selected by the user.

[0069] On the SNS posting operation screen 504, users can post the generated preview video to any SNS. For example, a user can select one or more SNS from SNS1 to SNS3 displayed on the SNS posting operation screen 504 by toggling on / off switches, and then post the asset's preview video by tapping the share button. It is also possible to post the preview video and comments by entering text in the text field on the SNS posting operation screen 504.

[0070] (Regarding the search process) Next, we will explain the asset search process in the asset list screen according to this embodiment. Figure 7 is a flowchart showing an example of the asset search process flow according to this embodiment.

[0071] As shown in Figure 7, first, in step S153, the search unit 360 acquires a search word, and in step S156, it executes a search process. Specifically, the user enters a search word into the search word input field displayed on the asset list screen, and the asset search is performed according to the search execution operation. The search unit 360 searches for assets to which metadata or tags containing the search word are attached. As described above, various metadata and tags are attached to assets, and assets can be searched by, for example, lens type, camera type, shooting location, purpose, etc.

[0072] In step S159, the display control unit 300 controls the display of the search results from the search unit 360. Specifically, the display control unit 300 displays a list screen of the searched assets. The list screen displays the asset thumbnail image, the asset metadata, and tags. The asset metadata and tags may be displayed in a position corresponding to (for example, nearby) the asset thumbnail image. Furthermore, since the assets are displayed as search results based on the search word, the user can intuitively understand that these assets are those to which the entered search word has been attached as metadata or tags. Note that the displayed metadata and tags are not limited to those containing the search word. The user can intuitively further narrow down the desired assets by referring to metadata and tags other than the displayed search word.

[0073] Furthermore, the search unit 360 may sort the search results. For example, the search unit 360 can personalize the search results by adding other search terms based on user information and prioritizing the display of assets that match those other search terms.

[0074] Figure 8 shows an example of an asset list screen 510 with personalized search results. The asset list screen 510 displays a search term input field 512 where a search term is entered, and a search is performed based on the search term entered in the search term input field 512. In the example shown in Figure 8, "game (or specific game name)" is entered.

[0075] As a result of the search, the asset list screen 510 displays assets that have metadata or tags containing the search term "game". Furthermore, the search results are sorted here based on search terms added according to user information, etc. For example, the search unit 360 may add the camera model name used to capture images used to generate assets that the user frequently uses when creating content, and the lens model name used to capture images that similarly frequently use, as search terms.

[0076] As shown in Figure 8, the asset list screen 510 displays the asset thumbnail image and the searched metadata or tags. There is no particular distinction in how the metadata or tags are displayed. As mentioned above, search words added according to user information are also included. Therefore, for example, assets that match more of the search words entered by the user, such as "Game," and other added search words such as "camera model name" and "lens product name," will be displayed preferentially at the top.

[0077] Furthermore, this system can also be used as a CMS (Content Management System). Users can import assets generated by other tools or systems into this system and perform operations such as compression, decimation, or conversion of data formats. To distinguish assets generated by this system from those generated by other tools or systems, assets generated by this system may be tagged with a source tag. A source tag may include the name of the system, application, or service that generated the asset, such as "Created by ◇◇◇ (system name)" or "Powered by ◇◇◇ (system name)". The search unit 360 may add words such as "Created by ◇◇◇ (system name)" or "Powered by ◇◇◇ (system name)" to the search terms to prioritize the display of assets generated by this system.

[0078] The acquisition of user information used to sort search results will be explained below with reference to Figure 9.

[0079] Figure 9 is a flowchart showing an example of a search process including sorting according to this embodiment. As shown in Figure 9, first, in step S203, the production support system 30 registers the user's occupation, industry, preferences, etc., when the user is registered.

[0080] Figure 10 shows an example of user information registered during user registration. As shown in Figure 10, each user's occupation is registered as VFX creator, 3D modeling creator, cameraman, producer, stage director, etc. In addition, each user's industry is registered as film, music, games, advertising, IT, etc. Furthermore, each user's preferences are registered as their favorite film director, favorite video works, genres of interest, etc.

[0081] By using such information to sort searches, the production support system 30 can prioritize displaying assets that match the user's interests at the top of the search results. For example, the production support system 30 may use words related to the user's occupation, industry, or preferences as additional search terms.

[0082] Next, in step S206, when an image is uploaded, the production support system 30 acquires and stores camera information (camera parameters), lens information (lens metadata), and shooting information (location information, environmental information, etc.) attached to the image. From this information, it is possible to identify frequently used camera model names, lens product names, and frequently used shooting locations.

[0083] The production support system 30 uses this information to sort searches, allowing it to prioritize the display of assets that utilize frequently used cameras and lenses, as well as assets based on images taken in frequently used locations.

[0084] Next, in step S209, the production support system 30 adds generation parameters, usage information, etc., to the 3D asset when generating it. Generation parameters include, for example, the generation algorithm, compression ratio, and number of meshes. The production support system 30 also adds any additional information inherited from the captured image used to generate the 3D asset to the 3D asset.

[0085] Next, in step S212, the production support system 30 obtains the user questionnaire results after the 3D asset is generated. After the 3D asset is generated, a questionnaire with a score is conducted, and the production support system 30 saves the user's preferred processing parameters (generation algorithm, compression ratio, number of meshes, etc.). The production support system 30 also updates the user's past processing history for 3D asset generation after the 3D asset is generated. Past processing history refers to various setting information (processing details) used when generating the 3D asset. For example, this includes the generation algorithm, bounding box size, compression ratio, number of meshes, conversion format, and preview video work history. The type of 3D asset generated by the user may also be included in the past processing history. In addition, the review type (draft, approved, rejected) and Processing status (pending, converting, completed, failed, no asset, terminated) of the generated 3D asset, entered by the user, may also be assigned to the 3D asset and saved as part of the user's past processing history.

[0086] By using this information to sort searches, the production support system 30 can prioritize displaying assets that have undergone processing according to the user's preferences (which the user frequently performs) and assets of frequently generated types at the top of the search results.

[0087] Furthermore, the production support system 30 may register user attributes based on the user's response to a questionnaire prompt after 3D asset generation.

[0088] Specifically, the production support system 30 registers users as interested in the system if they generate assets using the system more than a predetermined number of times each month and also answer the questionnaire. Furthermore, if a user answers the questionnaire but uses the system infrequently, the production support system 30 registers attributes such as low interest in the system or dissatisfaction with the generation results. Additionally, if a user regularly answers the questionnaire, the production support system 30 registers them as a cooperative user. In this case, for example, an invitation for a face-to-face user interview may be provided. Furthermore, if no questionnaire response is received, the production support system 30 registers the user as an uncooperative user and may use this information for improvement activities by the company providing the system. Additionally, qualitative evaluations of the generation results are obtained from users who use the system (generate assets) more than a predetermined number of times each month and answer the questionnaire, and users interested in the system register attributes indicating whether or not they are satisfied with the quality.

[0089] Next, in step S215, the production support system 30 attaches the user's operation log as metadata to the 3D asset. The production support system 30 may also generate metadata from the user's operation log after the 3D asset has been generated, such as viewing the generated 3D asset, and attach it to the target 3D asset.

[0090] For example, the production support system 30 acquires the number of clicks by the user, the number of screen transitions, and the time spent on each screen for each asset, and assigns this information as metadata. This allows the system provider to understand which 3D assets the user is interested in. Furthermore, by using these operation logs to sort searches, it becomes possible to prioritize the display of assets that the user is interested in (assets that the user frequently interacts with or views) at the top of the results.

[0091] Furthermore, the production support system 30 may also attach other users' operation logs as metadata to the 3D assets. For example, if a 3D asset is shared with a user's client, metadata can be attached from the client's operation logs for the 3D asset and used for sorting searches, making it possible to prioritize the display of assets popular with the client.

[0092] Then, in step S218, the production support system 30 performs a search using metadata via the search unit 360. Furthermore, as described above, the production support system 30 can also sort the search results using various user information. The sorting conditions may be changed at the user's discretion.

[0093] <4. Hardware Configuration> An embodiment of the present disclosure has been described above. Next, with reference to Figure 11, an example of the hardware configuration used in the production support system 30 according to an embodiment of the present disclosure will be described.

[0094] Figure 11 is a block diagram showing an example of the hardware configuration of an information processing device 900 according to one embodiment of the present disclosure. The information processing device 900 is an example of a hardware configuration applied to the capture device 10, output device 20, or production support system 30 according to this embodiment. Note that the information processing device 900 does not necessarily have all of the hardware configurations shown in Figure 11.

[0095] As shown in Figure 11, the information processing device 900 includes a processing circuit 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925.

[0096] The processing circuit 901 functions as an arithmetic processing unit and control unit, and controls the overall operation or a part of the operation within the information processing unit 900 according to various programs recorded in the ROM 902, RAM 903, storage device 919, or removable recording medium 927. The ROM 902 stores programs and calculation parameters used by the processing circuit 901. The RAM 903 temporarily stores programs used in the execution of the processing circuit 901 and parameters that change as appropriate during its execution. The processing circuit 901, ROM 902, and RAM 903 are interconnected by a host bus 907, which is composed of an internal bus. Furthermore, the host bus 907 is connected to an external bus 911, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 909.

[0097] The input device 915 is a device operated by the user, such as a button. The input device 915 may also include a mouse, keyboard, touch panel, switch, and lever. The input device 915 may also include a microphone that detects the user's voice. The input device 915 may be, for example, a remote control device that uses infrared or other radio waves, or an external connection device 929 such as a mobile phone that is compatible with the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs it to the processing circuit 901. By operating this input device 915, the user inputs various data to the information processing device 900 or instructs it to perform processing operations.

[0098] The input device 915 may also include an imaging device and sensors. The imaging device is a device that captures real space and generates an image using various components such as an image sensor, such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens for controlling the imaging of a subject onto the image sensor. The imaging device may capture still images or motion images. The sensors are various types of sensors, such as distance sensors, acceleration sensors, gyro sensors, geomagnetic sensors, vibration sensors, light sensors, and sound sensors. The sensors acquire information about the state of the information processing device 900 itself, such as the orientation of the housing of the information processing device 900, and information about the surrounding environment of the information processing device 900, such as the brightness and noise around the information processing device 900. The sensors may also include a GPS sensor that receives GPS (Global Positioning System) signals and measures the latitude, longitude, and altitude of the device.

[0099] The output device 917 is comprised of a device capable of visually or audibly notifying the user of the acquired information. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or an audio output device such as a speaker or headphones. The output device 917 may also include a PDP (Plasma Display Panel), a projector, a hologram, a printer, etc. The output device 917 outputs the results obtained from the processing of the information processing device 900 as text or images, or as sound such as voice or acoustics. The output device 917 may also include a lighting device that brightens the surroundings.

[0100] The storage device 919 is a data storage device configured as an example of the storage unit of the information processing device 900. The storage device 919 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. This storage device 919 stores programs and various data executed by the processing circuit 901, as well as various data acquired from external sources.

[0101] The drive 921 is a reader / writer for removable recording media 927 such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory, and is either built into or external to the information processing device 900. The drive 921 reads information recorded on the installed removable recording media 927 and outputs it to the RAM 905. The drive 921 also writes data to the installed removable recording media 927.

[0102] The connection port 923 is a port for directly connecting equipment to the information processing device 900. The connection port 923 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. Alternatively, the connection port 923 may be an RS-232C port, an optical audio terminal, or an HDMI (High-Definition Multimedia Interface) port. By connecting an external device 929 to the connection port 923, various types of data can be exchanged between the information processing device 900 and the external device 929.

[0103] The communication device 925 is a communication interface, for example, consisting of a communication device for connecting to an external network 70. The communication device 925 may be, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth®, Wi-Fi®, or WUSB (Wireless USB). Alternatively, the communication device 925 may be a router for optical communication, an ADSL (Asymmetric Digital Subscriber Line) router, or a modem for various types of communication. The communication device 925 transmits and receives signals, for example, to the Internet or other communication devices using a predetermined protocol such as TCP / IP. The external network 70 connected to the communication device 925 is a network connected by wire or wireless, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication.

[0104] For example, when the information processing device 900 functions as a production support system 30 according to the embodiment of this disclosure, the processing circuit 901 of the information processing device 900 executes a program loaded onto the RAM 903 to realize various functions that can be executed by a control unit (not shown) included in the production support system 30 (display control unit 300, captured image management unit 310, asset generation unit 320, metadata assignment unit 330, tagging processing unit 340, preview video generation unit 350, and search unit 360). The storage device 919 stores the information processing program according to this disclosure and various data stored by a storage unit (not shown) included in the production support system 30.

[0105] The processing circuit 901 reads and executes program data from the storage device 919, but as an alternative, these programs may be obtained from other devices via the external network 70. In other words, the storage device 919 is not limited to being inside the information processing device 900, but may be located outside the information processing device 900.

[0106] The processing circuit 901 is an example of an integrated circuit, and CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), ASIC (Application Specific Integrated Circuit), and FPGA (Field Programmable Gate Array) can all be considered integrated circuits.

[0107] Furthermore, when the information processing device 900 functions as a production support system 30 according to the embodiment of this disclosure, the communication device 925 corresponds to a communication unit (not shown) included in the production support system 30.

[0108] <5. Supplementary Information> Although preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the present technology is not limited to such examples. It is clear that a person with ordinary skill in the art of the present disclosure may conceive of various modifications or alterations within the scope of the technical idea described in the claims, and these will naturally be understood to fall within the technical scope of the present disclosure.

[0109] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0110] The information processing system described herein may consist of a single device (such as an information processing terminal or a server), or it may consist of multiple devices. Multiple devices may include, for example, an information processing terminal and one or more servers.

[0111] Furthermore, the embodiments and modifications of this disclosure described above can be combined as appropriate in areas where the processing content is not contradictory. Also, the order of each step shown in the sequence diagram or flowchart of this embodiment can be changed as appropriate. For example, each step may be processed chronologically, repeatedly, or partially in parallel.

[0112] Furthermore, one or more computer programs can be created to enable the capture device 10, output device 20, or production support system 30 to perform their functions, using hardware such as the CPU, ROM, and RAM built into the capture device 10, output device 20, or production support system 30. A computer-readable storage medium storing such one or more computer programs is also provided.

[0113] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0114] Furthermore, this technology can also take the following configurations: (1) An information processing system comprising: a generation unit that generates assets of an object; and a display control unit that displays a list display screen of the assets, wherein the list display screen displays the assets and metadata attached to the assets in the asset generation process. (2) The information processing system according to (1), wherein the generation unit generates the assets using sensing data acquired by sensing the object, and the metadata attached to the assets includes information obtained when the sensing data was acquired. (3) The information processing system according to (2), wherein the sensing data is an image, and the metadata includes at least one of depth data, camera position information at the time of imaging, camera attitude information at the time of imaging, camera parameters, and environmental information. (4) The information processing system according to any one of (1) to (3), wherein the metadata includes information obtained in the asset generation process. (5) The information processing system according to (4), wherein the metadata includes usage information of the asset, object recognition results obtained from captured images used to generate the asset, or scene recognition results. (6) The information processing system according to any one of (1) to (5), wherein the asset is tagged with converted metadata obtained by converting the metadata. (7) The information processing system according to any one of (1) to (6), wherein the asset is tagged with a tag specified by the user. (8) The information processing system according to any one of (1) to (7), wherein the metadata attached to the asset further includes an operation log for the generated asset. (9) The information processing system according to any one of (1) to (8), further comprising a search unit that searches for metadata including a search word entered by the user, or an asset to which a tag generated by converting the metadata has been attached. (10) The information processing system according to (9), wherein the display control unit controls the display of the list display screen based on the search results.(11) The information processing system according to (10), wherein the search unit adds one or more other search words based on user information, searches for assets to which metadata or tags including the other search words are attached, and the display control unit preferentially displays assets to which metadata or tags including the other search words are attached as search results. (12) The information processing system according to (11), wherein the user information includes information relating to at least one of the user's occupation, industry, and preferences. (13) The information processing system according to (10) or (11), wherein the search unit rearranges the search results so that assets to which assets relating to the user's interests based on the user's occupation, industry, or preferences, captured images taken using a camera or lens frequently used by the user, or captured images taken in locations frequently used by the user are displayed at the top. (14) The information processing system according to any one of (1) to (13), wherein the information processing system further comprises a video generation unit that generates a preview video or thumbnail image of the asset based on metadata or tags attached to the asset. (15) The information processing system according to (14), wherein the video generation unit generates a preview video or thumbnail image suitable for the intended use based on the intended use information contained in the metadata. (16) The information processing system according to (14), wherein the video generation unit generates a preview video with a characteristic camera flow, direction, or field of view of a genre, cameraman, or film director selected by the user. (17) The information processing system according to (14), wherein the video generation unit generates a preview video to correspond to the media selected by the user. (18) The information processing system according to (1), wherein the asset is assigned a tag indicating its origin, and the list display screen displays the asset and the tag indicating its origin.(19) An information processing method comprising: a processor generating an asset of an object; and displaying a list display screen of the asset, wherein the list display screen displays the asset and metadata attached to the asset in the asset generation process. (20) A program that causes a computer to function as a generation unit for generating an asset of an object; and a display control unit for displaying a list display screen of the asset, wherein the list display screen displays the asset and metadata attached to the asset in the asset generation process. (21) An information processing system comprising: a generation unit for generating an asset of an object; and an attachment unit for attaching metadata to the asset, wherein the attachment unit attaches supplementary information of sensing data used to generate the asset as metadata to the asset.

[0115] 10 Capture device 20 Output device 30 Production support system 300 Display control unit 310 Captured image management unit 320 Asset generation unit 330 Metadata assignment unit 340 Tagging processing unit 350 Preview video generation unit 360 Search unit

Claims

1. An information processing system comprising: a generation unit that generates assets of an object; and a display control unit that displays a list of the assets, wherein the list display screen displays the assets and metadata assigned to the assets during the asset generation process.

2. The information processing system according to claim 1, wherein the generation unit generates the asset using sensing data obtained by sensing the target object, and the metadata attached to the asset includes information obtained when the sensing data was acquired.

3. The information processing system according to claim 2, wherein the sensing data is an captured image, and the metadata includes at least one of depth data, camera position information at the time of imaging, camera attitude information at the time of imaging, camera parameters, and environmental information.

4. The information processing system according to claim 1, wherein the metadata includes information obtained in the asset generation process.

5. The information processing system according to claim 4, wherein the metadata includes usage information of the asset, object recognition results obtained from captured images used to generate the asset, or scene recognition results.

6. The information processing system according to claim 1, wherein the asset is tagged with converted metadata obtained by converting the metadata.

7. The information processing system according to claim 1, wherein the asset is assigned a tag specified by the user.

8. The information processing system according to claim 1, wherein the metadata assigned to the asset further includes an operation log for the generated asset.

9. The information processing system according to claim 1, further comprising a search unit that searches for metadata including search words entered by a user, or assets to which tags generated by converting the metadata have been assigned.

10. The information processing system according to claim 9, wherein the display control unit controls the display of the list display screen based on the search results.

11. The information processing system according to claim 10, wherein the search unit adds one or more other search words based on user information, searches for assets to which metadata or tags containing the other search words are assigned, and the display control unit preferentially displays assets to which metadata or tags containing the other search words are assigned as search results.

12. The information processing system according to claim 11, wherein the user information includes at least one of the user's occupation, industry, and preferences.

13. The information processing system according to claim 10, wherein the search unit rearranges the search results so that assets that utilize assets corresponding to the user's interests based on the user's occupation, industry, or preferences, captured images taken using a camera or lens frequently used by the user, or captured images taken in locations frequently used by the user are displayed at the top.

14. The information processing system according to claim 1, further comprising a video generation unit that generates a preview video or thumbnail image of an asset based on metadata or tags assigned to the asset.

15. The information processing system according to claim 14, wherein the video generation unit generates the preview video or thumbnail image based on the usage information contained in the metadata.

16. The information processing system according to claim 14, wherein the video generation unit generates the preview video based on a genre selected by the user, a cameraman, or a characteristic camera flow, direction, or angle of view of a film director.

17. The information processing system according to claim 14, wherein the video generation unit generates the preview video to correspond to the media selected by the user.

18. The information processing system according to claim 1, wherein the asset is assigned a tag indicating its origin, and the list display screen displays the asset and the tag indicating its origin.

19. An information processing method comprising: a processor generating an asset of an object; and displaying a list display screen of the asset, wherein the list display screen displays the asset and metadata attached to the asset during the asset generation process.

20. A program that causes a computer to function as a generation unit that generates assets of an object, and a display control unit that displays a list of the assets, wherein the list display screen displays the assets and the metadata assigned to the assets during the asset generation process.