Information processing system, information processing method, and program

Abstract

This disclosure relates to information processing systems, methods, and procedures that enable the provision of more valuable user experiences. In this disclosure, a driving simulation system uses a vehicle selected by the user to perform a driving simulation and acquires driving data corresponding to the user's actions on the vehicle and in-simulator images of the vehicle driving in virtual space. A camera system uses a camera to capture images of the vehicle and the user to obtain captured images. A video creation system creates a video configured using at least the in-simulator images and the captured images. For example, this technology can be applied to a content creation system that uses an actual vehicle to perform a driving simulation to create content.

Description

Technical Field

[0001] This disclosure relates to information processing systems, information processing methods and procedures, and more particularly to information processing systems, information processing methods and procedures that can provide a more valuable user experience. Background Technology

[0002] In recent years, various user experiences, such as realistic driving simulations and content appreciation, have been proposed by using display devices and audio facilities, for example, inside and outside electric vehicles in a parked state, and by combining user profiles, external vibration devices, and charging devices. Furthermore, with the rise of virtual production, three-dimensional computer graphics (3DCG) has been used to create promotional videos.

[0003] PTL 1 discloses a racing game device that uses an operating unit equipped with a steering wheel, accelerator, and brakes to perform racing games.

[0004] Reference List

[0005] Patent documents

[0006] PTL 1: JP 2000-229177A Summary of the Invention

[0007] Technical issues

[0008] As mentioned above, while various user experiences have been proposed using parked electric vehicles, there is a need to provide more valuable user experiences, such as incentivizing users to travel to charging stations further away and spend time there.

[0009] This disclosure is made in view of the foregoing and is intended to provide a more valuable user experience by reusing 3DCG created through virtual production, games, etc.

[0010] Solution to the problem

[0011] An information processing system according to one aspect of this disclosure includes: a driving simulation system configured to perform a driving simulation using a vehicle selected by a user, and to acquire driving data corresponding to the user's operation of the vehicle and an in-simulator video depicting the vehicle driving in a virtual space; a camera system configured to capture the vehicle and the user using a camera to acquire captured video; and a motion picture creation system configured to create motion pictures using at least the in-simulator video and the captured video.

[0012] An information processing method or procedure according to one aspect of this disclosure includes: performing a driving simulation using a vehicle selected by a user, and acquiring driving data corresponding to the user's operation of the vehicle and an in-simulator video depicting the vehicle driving in a virtual space; capturing the vehicle and the user using a camera to obtain captured video; and creating a motion image using at least the in-simulator video and the captured video.

[0013] According to one aspect of this disclosure, a driving simulation is performed using a vehicle selected by a user, and driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space are acquired; a camera is used to capture the vehicle and the user to acquire captured video; and motion images are created using at least the in-simulator video and the captured video. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating a configuration example of a content creation system that applies the present technology.

[0015] Figure 2 This is an example diagram used to describe a clip (cut) capturing the state of a user entering a vehicle.

[0016] Figure 3 This is an example diagram used to describe a clip capturing the state of a user driving a vehicle from above.

[0017] Figure 4 This is a diagram used to illustrate an example of capturing the interior of a vehicle.

[0018] Figure 5 This is a block diagram illustrating an example configuration of a motion image creation processing unit.

[0019] Figure 6 This is a block diagram illustrating an example configuration of the simulation processing unit.

[0020] Figure 7 This is a diagram illustrating an example of a clip used to describe the layout of a captured 3DCG model and a vehicle moving in virtual space.

[0021] Figure 8 This is a block diagram illustrating an example configuration of the capture control processing unit.

[0022] Figure 9 It is a sequence diagram used to describe the content creation and processing.

[0023] Figure 10 It is a flowchart used to describe the template clip creation process.

[0024] Figure 11 It is a flowchart used to describe the initial capture process.

[0025] Figure 12 It is a flowchart used to describe the emulator startup process.

[0026] Figure 13 It is a flowchart used to describe the video capture process within the simulator.

[0027] Figure 14 It is a flowchart used to describe the automatic capture process.

[0028] Figure 15 It is a flowchart used to describe the process of creating moving images.

[0029] Figure 16 This is a block diagram illustrating a configuration example of a computer to which the present technology is applied. Detailed Implementation

[0030] In the following text, specific embodiments in which this technology is applied will be described in detail with reference to the accompanying drawings.

[0031] Configuration example of a content creation system

[0032] First, refer to Figures 1 to 8 This section describes a configuration example for a content creation system that utilizes this technology.

[0033] Figure 1 This is a block diagram showing a configuration example of the content creation system 11.

[0034] Suppose that the device is installed and used at a vehicle dealership, for example, a charging station equipped for electric vehicles. Figure 1 The content creation system 11 shown is used to create content similar to existing promotional videos (e.g., commercial footage) for advertising vehicle 43 by performing a driving simulation while a user visits a vehicle dealership to charge an electric vehicle and test drives the vehicle 43 displayed at the dealership.

[0035] For example, in the content creation system 11, the vehicle 43 that the user wants to test drive is moved to a predetermined position in front of the display device 42, and the user selects, for example, the location to be driven in the content and the music to be used in the content, and then the camera 52 begins to capture. Then, while the camera 52 captures clips of the user entering the vehicle 43 and performs a driving simulation based on the user driving the vehicle 43, various clips of the vehicle 43 are captured by the camera 52 against the background of the display device 42 displaying a simulated video of the location selected by the user. In addition, during the driving simulation based on the user driving the vehicle 43, the virtual camera captures the state of the vehicle 43 driving in a virtual space corresponding to the location selected by the user, and generates in-simulation video clips of various types.

[0036] Subsequently, in the content creation system 11, the camera 52 stops capturing footage of the user exiting the vehicle 43 when the driving simulation ends. Then, in the content creation system 11, the captured video of each clip captured by the camera 52, as well as the in-simulator video of each clip generated during the driving simulation, are presented to the user, and the user can select their preferred clips. Therefore, the content creation system 11 can create motion graphics that combine the video of the clips selected by the user, produce content in which music selected by the user is reproduced along with the motion graphics, and provide this content to the user. Furthermore, the content creation system 11 can assign non-fungible tokens (NFTs) to links used for sharing content on social networking services (SNS) and provide those links.

[0037] like Figure 1 As shown, the content creation system 11 includes a motion image creation system 21, a driving simulation system 22, and a camera system 23, and a cloud server 24 is connected to the motion image creation system 21 via a network 25.

[0038] The motion picture creation system 21 includes a motion picture creation processing unit 31 and a user interface (UI) terminal 32.

[0039] UI terminal 32 includes a touch panel, and when a user performs an operation on the touch panel to select items such as the vehicle 43 to be test driven, the location to be driven in the content, and the music used in the content, UI terminal 32 supplies user selection information to motion image creation processing unit 31 to indicate the user's selection content.

[0040] The motion image creation processing unit 31 references each clip of an existing promotional video constituting the vehicle 43 indicated by the user selection information, and creates template clips for instructing camera operation and capture timing, for example, for capturing a state of performing a driving simulation at a location selected by the user. Examples of instructions included in the template clips include instructions for starting and ending the capture timing of each clip, instructions for the arrangement and perspective of the camera 52 required to capture the captured video of each clip, instructions for the arrangement and perspective of the virtual camera required to generate the in-simulator video of each clip, and instructions for including the capture of the user in cases where the driver is depicted in each clip. The motion image creation processing unit 31 then supplies the template clips and user selection information to the driving simulation system 22 and the camera system 23, respectively.

[0041] Subsequently, driving data and in-simulator video from the driving simulation system 22 are supplied to the motion image creation processing unit 31, along with captured images from the camera system 23. Here, the motion image creation processing unit 31 stores multiple pre-prepared virtual production videos as advertising material for the vehicle 43 (e.g., including material not used in promotional videos), and can select virtual production videos similar to the driving simulation based on the driving data. Then, the motion image creation processing unit 31 displays the in-simulator video, captured video, and virtual production video on the UI terminal 32 as options for allowing the user to select the video to be used in the content. When the user selects a video to be used in the content from the options, the motion image creation processing unit 31 creates a motion image by adjusting and editing the user-selected video according to the length of the music selected by the user, and displays the motion image on the UI terminal 32.

[0042] The driving simulation system 22 includes a simulation processing unit 41, a display device 42, and a vehicle 43.

[0043] The simulation processing unit 41 performs a driving simulation in which the vehicle 43 virtually drives in a virtual space corresponding to a location indicated by user selection information supplied from the motion image creation processing unit 31, based on the user's actual driving of the vehicle 43. For example, the vehicle 43 supplies the simulation processing unit 41 with driving data indicating driving content (e.g., tire angle, or degree of acceleration and deceleration) corresponding to the user's operation of the steering wheel, pedals, etc. The simulation processing unit 41 controls, for example, the built-in active suspension, seat vibration device, and external vibration device to generate vibrations to be generated in the vehicle 43 based on the driving data (e.g., left-right tilting vibrations due to turning, and front-back tilting vibrations due to acceleration and deceleration). At this time, the simulation processing unit 41 generates a simulated video linked to the scenery in the simulator corresponding to the virtual space in which the vehicle 43 is driving, and displays the simulated video on the display device 42.

[0044] Then, the simulation processing unit 41 generates an in-simulator video depicting the vehicle 43 driving in virtual space by capturing images using a virtual camera based on template clips supplied from the motion image creation processing unit 31. Afterward, when the virtual driving of the vehicle 43 has been completed for all clips that needed to be captured based on the template clips, the simulation processing unit 41 ends the driving simulation and supplies driving data and in-simulator video to the motion image creation processing unit 31.

[0045] The camera system 23 includes a capture control processing unit 51 and a camera 52.

[0046] The capture control processing unit 51 performs camera control based on a template clip supplied from the motion image creation processing unit 31, and acquires captured video of the vehicle 43 and the user through the camera 52. For example, the capture control processing unit 51 performs initial capture processing to detect the vehicle type of the vehicle 43 and the stopping position of the vehicle 43, and in automatic capture processing to perform camera control based on the template clip, adjusts the position of the camera 52 according to the stopping position of the vehicle 43, so that capture by the camera 52 is performed automatically. At this time, the capture control processing unit 51 can adjust the camera position based on, for example, the position and posture of the subject in the template clip (e.g., the vehicle 43 or the user as the driver). Note that camera position learning data obtained by learning the relationship between the position and posture of the object and the camera position can be used to adjust the camera position.

[0047] When the driving simulation in the driving simulation system 22 ends, the capture control processing unit 51 supplies the motion image creation processing unit 31 with the captured video obtained by the camera 52 capturing the vehicle 43 and the user.

[0048] Cloud server 24 provides services for distributing, for example, existing promotional videos referenced when creating template clips in motion graphics creation processing unit 31, virtual production videos used when creating motion graphics in motion graphics creation processing unit 31, and various assets used when generating videos within the simulator in simulation processing unit 41. Cloud server 24 provides services for publishing NFTs to content created in content creation system 11.

[0049] Notice, Figure 1 The content creation system 11 shown has a connection configuration in which cloud server 24 is connected to motion image creation system 21 via network 25, but it may also have a connection configuration in which motion image creation system 21, driving simulation system 22, camera system 23 and cloud server 24 are connected via network 25. Simulation processing unit 41 may be an in-vehicle infotainment electronic control unit (IVI ECU) installed in vehicle 43, or it may be a multimedia plug-in unit (MPU) connected to the IVI ECU. Camera 52 may be a system utilizing multiple cameras.

[0050] The content creation system 11, configured as described above, can provide a user experience in which the user drives through a virtual space corresponding to an existing promotional video of vehicle 43 and experiences the emotions of the characters in the promotional video. At the end of the driving simulation, the content creation system 11 can provide content created using motion images captured while the user experiences driving in the virtual space through the driving simulation.

[0051] For example, the content creation system 11 can provide services that integrate value for users with promotion for vehicle dealerships, such as services where users form a deep connection with the new vehicle 43 at vehicle dealerships and create first memories of the vehicle 43. Then, for example, the content creation system 11 can provide an even more valuable user experience, where users create content while enjoying driving simulations and share links to NFT-enabled content via social networking services.

[0052] Note that the content creation system 11 can adjust the relative positions of the camera 52, lights, display device 42, and user based on the user's location and the progress of the driving simulation to suit capture in both software and hardware. Furthermore, in addition to providing content composed of moving images, the content creation system 11 can also provide multiple image shots selected from the moving images.

[0053] Figure 2 This is a clip used to describe the state of capturing a user entering vehicle 43.

[0054] For example, in the clip capturing the state of the user entering vehicle 43, such as Figure 2 As shown in Figure A, video clips 1 depicting a user opening the door of vehicle 43 and 2 depicting the user sitting in the seat of vehicle 43 were captured. To perform such captures, template clips were used to indicate the arrangement of camera 52 relative to vehicle 43 when video clip 1 was captured (see Figure A). Figure 2 (B) and the arrangement of camera 52 relative to vehicle 43 when capturing video clip 2 (see B) Figure 2 (C). Therefore, the capture control processing unit 51 can initiate capture by the camera 52 based on the template clipping, and execute camera control to move the camera 52 from... Figure 2 The arrangement shown in B is moved to Figure 2 The arrangement shown in C is used to image the state of the user entering vehicle 43.

[0055] Figure 3 It is a clip used to describe the state of the user driving vehicle 43 captured from above vehicle 43.

[0056] For example, in a clip capturing the state of the user driving vehicle 43 from above, such as Figure 3 As shown in A, video clip 3, depicting the vehicle 43 from above, and video clip 4, depicting the user driving the vehicle 43 through the front window, are captured. To perform this capture, a template clip is used to indicate the arrangement of the camera 52 relative to the vehicle 43 when video clip 3 is captured (see [reference]). Figure 3 (B) and the arrangement of camera 52 relative to vehicle 43 when capturing video clip 4 (see B) Figure 3(C). Therefore, the capture control processing unit 51 can initiate capture by the camera 52 based on the template clipping, and execute camera control to move the camera 52 from... Figure 3 The arrangement shown in B is moved to Figure 3 The arrangement shown in C is used to image the state of the user driving the vehicle 43 from above the vehicle 43.

[0057] Reference Figure 4 Description of the capture inside vehicle 43.

[0058] like Figure 4 As shown, when capturing the interior of vehicle 43, camera 52 is attached to the distal end of bracket 53, and camera 52 is inserted into vehicle 43 through the window of vehicle 43 to perform the capture. Figure 4 Figure A shows the arrangement of camera 52 when it performs a capture from the center of the rear seat toward the front of vehicle 43, and Figure 4 Figure B shows the arrangement of camera 52 when it captures the user driving from the oblique front of the passenger seat side.

[0059] Figure 5 This is a block diagram illustrating an example configuration of the motion image creation processing unit 31.

[0060] like Figure 5 As shown, the motion image creation processing unit 31 includes a UI control unit 71, a storage device 72, a template clip creation unit 73, a communication unit 74, a processing control unit 75, a clip selection unit 76, and a motion image creation unit 77.

[0061] The UI control unit 71 controls data exchange with the UI terminal 32. For example, the UI control unit 71 acquires user selection information supplied from the UI terminal 32 and supplies the user selection information to the template clip creation unit 73, the communication unit 74, the clip selection unit 76, and the motion picture creation unit 77. The UI control unit 71 also supplies the UI terminal 32 with content including video options (in-simulator video, captured video, and virtual production video) supplied from the clip selection unit 76 and motion pictures supplied from the motion picture creation unit 77.

[0062] Storage device 72 holds various types of data, such as existing promotional videos to be referenced in creating template clips, virtual production videos presented to users as options for videos to be used in the content, and music to be used in the content.

[0063] The template clip creation unit 73 retrieves existing promotional videos of vehicle 43 indicated by user selection information supplied from UI control unit 71 from storage device 72, creates template clips by referencing each clip constituting the promotional video, and supplies the template clips to communication unit 74. Furthermore, if the location indicated by the user selection information supplied from UI control unit 71 does not match the location where the existing promotional video of vehicle 43 indicated by the user selection information was captured, the template clip creation unit 73 can create template clips by referencing each clip of existing promotional videos of other vehicles captured at the location indicated by the user selection information. For example, the template clip creation unit 73 can extract main clips at predetermined time intervals in each scene of the promotional video and create template clips based on the extracted clips. The template clip creation unit 73 can use learned recognition criteria when identifying main clips and creating template clips. Note that the recognition criteria for the main clips are pre-learned and maintained in the template clip creation unit 73.

[0064] The communication unit 74 communicates with the simulation processing unit 41 and the capture control processing unit 51. For example, the communication unit 74 sends user selection information supplied from the UI control unit 71 and template clips supplied from the template clip creation unit 73 to the simulation processing unit 41 and the capture control processing unit 51, respectively. The communication unit 74 receives driving data and in-simulator video sent from the simulation processing unit 41 and supplies the driving data and in-simulator video to the clip selection unit 76, and receives captured images sent from the capture control processing unit 51 and supplies the captured images to the clip selection unit 76.

[0065] The processing control unit 75 controls the processing performed by the simulation processing unit 41 and the capture control processing unit 51. For example, when the simulation processing unit 41 notifies the processing control unit 75 of the start of the simulator via the communication unit 74, the processing control unit 75 instructs the simulation processing unit 41 to start the driving simulation. When the simulation processing unit 41 notifies the start of the driving simulation via the communication unit 74, the processing control unit 75 instructs the capture control processing unit 51 to start the automatic capture processing and instructs the simulation processing unit 41 to start the in-simulator video capture processing.

[0066] The editing selection unit 76 supplies the UI control unit 71 with in-simulator video sent from the simulation processing unit 41, captured video sent from the capture control processing unit 51, and virtual production video read from the storage device 72, based on driving data sent from the simulation processing unit 41. For example, the editing selection unit 76 selects a virtual production video that is similar to the driving situation from multiple virtual production videos prepared in advance as material for advertising the vehicle, based on driving data.

[0067] Then, the UI control unit 71 presents the simulator video, captured video, and virtual-generated video as options for use in the content to the user, and provides user selection information, instructing the user to select a video from the videos, to the editing selection unit 76. In response, the editing selection unit 76 provides the video selected by the user from the simulator video, captured video, and virtual-generated video to the moving image creation unit 77.

[0068] The motion picture creation unit 77 creates motion pictures by adjusting and editing the video selected by the user, which is supplied from the clip selection unit 76, according to the length of the music indicated by the user selection information supplied from the UI control unit 71. Then, the motion picture creation unit 77 supplies the UI control unit 71 with content that reproduces the music together with the created motion picture, and the UI control unit 71 supplies the content to the UI terminal 32.

[0069] Figure 6 This is a block diagram showing an example configuration of the simulation processing unit 41.

[0070] like Figure 6 As shown, the simulation processing unit 41 includes a communication unit 81, a simulator startup processing unit 82, a simulation execution unit 83, and a simulator video capture processing unit 84.

[0071] The communication unit 81 communicates with the motion picture creation processing unit 31 and the capture control processing unit 51. For example, the communication unit 81 receives user selection information and template clips sent from the motion picture creation processing unit 31, and supplies the user selection information and template clips to the simulator startup processing unit 82, the simulation execution unit 83, and the simulator in-video capture processing unit 84. The communication unit 81 also sends in-simulator video supplied from the simulator in-video capture processing unit 84 to the motion picture creation processing unit 31.

[0072] When user selection information and template clips are supplied from communication unit 81, simulator startup processing unit 82 executes simulator startup processing to start the simulator. Simulator startup processing unit 82 arranges the 3DCG model data included in the assets retrieved according to the location indicated by the user selection information in the virtual space where the vehicle 43 is virtually driven in the driving simulation. Note that the 3DCG is existing 3DCG created through virtual production, games, etc. For example, when the location indicated by the user selection information is an urban area and a mountainous area, simulator startup processing unit 82 arranges the 3DCG model data constituting the urban area (e.g., building complexes and viaducts) and the 3DCG model data constituting the mountainous area (e.g., mountains and forests) in the virtual space, such as... Figure 7As shown in A. Then, the simulator startup processing unit 82 notifies the motion picture creation processing unit 31 via the communication unit 81 that the simulator has been started.

[0073] When the motion image creation processing unit 31, which notifies the virtual space simulator of its activation via communication unit 81, instructs the simulation execution unit 83 to begin driving simulation, the simulation execution unit 83 begins driving simulation and notifies the motion image creation processing unit 31 that driving simulation has begun. Then, the simulation execution unit 83 performs a driving simulation for virtually driving the vehicle 43 based on driving data supplied from the vehicle 43, enabling the capture of each clip corresponding to a template clip in the virtual space where 3DCG model data has been placed by the simulator activation processing unit 82.

[0074] Subsequently, when all the clips required to capture according to the template clips have been completed for the vehicle 43's driving in virtual space, the simulation execution unit 83 ends the driving simulation. Then, the simulation execution unit 83 creates driving data indicating the driving conditions of the vehicle 43 in the driving simulation (such as its driving position and speed in virtual space) based on the driving data supplied from the vehicle 43 during the driving simulation, and supplies the driving data to the motion picture creation processing unit 31 via the communication unit 81. The simulation execution unit 83 notifies the capture control processing unit 51 via the communication unit 81 that the driving simulation has ended. Furthermore, the simulation execution unit 83 generates a simulation video adjusted to appear as if the vehicle 43 is actually being driven from its position in virtual space during the driving simulation via the display device 42, and displays the simulation video on the display device 42.

[0075] The in-simulator video capture processing unit 84 positions a virtual camera in virtual space at an angle capable of capturing in-simulator video corresponding to template clips supplied from the communication unit 81. Then, when the in-simulator video capture processing unit 84 is instructed by the motion image creation processing unit 31 to begin in-simulator video capture processing in response to a notification from the simulation execution unit 83 via the communication unit 81 to begin driving simulation, the in-simulator video capture processing unit 84 performs in-simulator video capture processing according to the template clips, sequentially capturing the clips to be captured via the virtual camera. For example, when vehicle 43... Figure 7 As shown in A, when the vehicle 43 travels from an urban area to a mountainous area in virtual space, the simulator video capture processing unit 84 performs simulator video capture processing. This simulator video capture processing is used to capture video clips 5 from when the vehicle 43 is virtually traveling in the urban area to video clips 6 from when the vehicle 43 is virtually traveling in the mountainous area using a virtual camera, such as... Figure 7 As shown in B.

[0076] Furthermore, when the clip to be captured includes a user's capture, the video capture processing unit 84 in the simulator can notify the camera system 23 via the communication unit 81 of a prompt to instruct the camera 52 to start capturing based on the template clip.

[0077] Then, when the driving simulation performed by the simulation execution unit 83 ends, the simulator video capture processing unit 84 ends the simulator video capture processing and supplies the simulator video to the motion image creation processing unit 31 via the communication unit 81.

[0078] Figure 8 This is a block diagram illustrating a configuration example of the capture control processing unit 51.

[0079] like Figure 8 As shown, the capture control processing unit 51 includes a communication unit 91, an initial capture processing unit 92, and an automatic capture processing unit 93.

[0080] Communication unit 91 communicates with motion picture creation processing unit 31 and simulation processing unit 41. For example, communication unit 91 receives user selection information and template clips sent from motion picture creation processing unit 31, and supplies the user selection information and template clips to initial capture processing unit 92 and automatic capture processing unit 93. Communication unit 91 sends the captured video supplied from automatic capture processing unit 93 to motion picture creation processing unit 31.

[0081] The initial capture processing unit 92 performs initial capture processing to detect the type of vehicle 43 and the stopping position of vehicle 43 relative to display device 42. For example, in the initial capture processing, the initial capture processing unit 92 moves camera 52 to an arrangement that can capture from a perspective corresponding to the clip at the start of capture, based on a template clip supplied from communication unit 91, and calibrates the arrangement of camera 52, for example, based on the stopping position of vehicle 43 relative to display device 42 and the reflection of camera 52.

[0082] The automatic capture processing unit 93, based on the template clips supplied from the communication unit 91, derives camera operations for controlling the operation of the camera 52 and timing for the camera 52 to perform capture for each clip, and performs camera control so that capture video corresponding to the template clip can be captured by the camera 52. Then, when the motion picture creation processing unit 31 instructs the automatic capture processing unit 93 to start automatic capture processing via the communication unit 91, the automatic capture processing unit 93 starts automatic capture processing. For example, in automatic capture processing, when the automatic capture processing unit 93 is notified via the communication unit 91 of a prompt indicating the start of capture from the video capture processing unit 84 in the simulator, the automatic capture processing unit 93 captures the clip to be captured by the camera 52 at that timing.

[0083] Then, when the automatic capture processing unit 93 notifies the simulation execution unit 83 via the communication unit 81 that the driving simulation has ended, the automatic capture processing unit 93 terminates the automatic capture process. Afterwards, the automatic capture processing unit 93 sends the captured video obtained through the automatic capture process to the motion picture creation processing unit 31 via the communication unit 81.

[0084] As described above, the content creation system 11 includes a motion image creation processing unit 31, a simulation processing unit 41, and a capture control processing unit 51. It can provide content created using motion images obtained by capturing the user's state while the user performs a driving simulation, which involves driving through a virtual space corresponding to an existing promotional video of vehicle 43. Furthermore, the content creation system 11 can provide a more valuable user experience by reusing 3DCG created through virtual production, games, etc.

[0085] Content creation and processing examples

[0086] Reference Figures 9 to 15 Describes the content creation processes performed in the content creation system 11.

[0087] Figure 9 It is a sequence diagram used to describe examples of content creation processes.

[0088] For example, when the content creation system 11 is started, processing begins, and in step S11, in the motion picture creation system 21, the template clip creation unit 73 performs template clip creation processing (see description later). Figure 10 (Flowchart). As described later, in the template clip creation process, the template clip creation unit 73 sends the template clip created in the template clip creation process and the user selection information to the driving simulation system 22 and the camera system 23 respectively via the communication unit 74.

[0089] In step S12, within camera system 23, initial capture processing unit 92 receives the template clipping and user selection information sent from motion picture creation system 21 in step S11 via communication unit 91, and performs initial capture processing (see description later). Figure 11 (Flowchart).

[0090] In step S13, within the driving simulation system 22, the simulator startup processing unit 82 receives, via the communication unit 81, the template clip and user selection information sent from the motion image creation system 21 in step S11. Then, the simulator startup processing unit 82 performs simulator startup processing (see description below). Figure 12The flowchart shows that the motion picture creation system 21 simulator has been started, and the communication unit 81 notifies the system that the simulator has been started.

[0091] In step S14, in the motion image creation system 21, the processing control unit 75, via the communication unit 74, instructs the driving simulation system 22 to start driving simulation in response to the notification of simulator startup in step S13.

[0092] In step S15, in the driving simulation system 22, the simulation execution unit 83 starts the driving simulation according to the instructions in step S14, and notifies the motion picture creation system 21 of the start of the driving simulation via the communication unit 81. Then, the simulation execution unit 83 performs a driving simulation for virtually driving the vehicle 43 based on the driving data supplied from the vehicle 43, so that each clip that needs to be captured according to the template clip can be captured in the virtual space.

[0093] In step S16, in the motion image creation system 21, the processing control unit 75, in response to being notified of the start of the driving simulation in step S15, instructs the camera system 23 to start automatic capture processing and instructs the driving simulation system 22 to start in-simulator video capture processing.

[0094] In step S17, within the driving simulation system 22, the in-simulator video capture processing unit 84 performs in-simulator video capture processing according to the instructions in step S16 (see description below). Figure 13 (Flowchart). As described later, in the in-simulator video capture processing, the in-simulator video capture processing unit 84, for example, notifies the camera system 23 of prompts indicating that the camera 52 should begin capturing and the end of the driving simulation.

[0095] In step S18, in camera system 23, automatic capture processing unit 93 performs automatic capture processing according to the instructions in step S16 (see description later). Figure 14 (Flowchart).

[0096] When the automatic capture process in step S18 ends, the process proceeds to step S19. In step S19, in camera system 23, automatic capture processing unit 93 sends the captured video obtained through the automatic capture process performed in step S18 to motion picture creation system 21 via communication unit 91.

[0097] When the in-simulator video capture processing in step S17 ends, the process proceeds to step S20. In step S20, in the driving simulation system 22, the simulation execution unit 83 sends the driving data acquired through the driving simulation performed in step S15 to the motion picture creation system 21 via the communication unit 81. Furthermore, in step S20, in the driving simulation system 22, the in-simulator video capture processing unit 84 sends the in-simulator video acquired through the in-simulator video capture processing performed in step S17 to the motion picture creation system 21 via the communication unit 81.

[0098] In step S21, the motion picture creation system 21 performs motion picture creation processing (see description below). Figure 15 (Flowchart). Then, after the content is provided to the user in the motion picture creation process executed in step S21, the content creation process ends.

[0099] Figure 10 It is used to describe in Figure 9 The flowchart for the template clip creation process performed in step S11.

[0100] In step S21, when the user operates the UI terminal 32 to select the vehicle 43 to test drive, the location to drive in the content, and the music to be used in the content, the template editing creation unit 73 obtains the user's selection information via the UI control unit 71.

[0101] In step S22, the template editing creation unit 73 retrieves the existing promotional video of the vehicle 43 selected by the user from the storage device 72 based on the user selection information obtained in step S21, and identifies the location where the promotional video was captured. Then, the template editing creation unit 73 determines whether the location selected by the user matches the location where the existing promotional video of the vehicle 43 selected by the user was captured based on the user selection information obtained in step S21.

[0102] In step S22, when the template clip creation unit 73 determines that the location selected by the user does not match the location where the existing promotional video of the vehicle 43 selected by the user was captured (i.e., the locations are different), the process proceeds to step S23.

[0103] In step S23, the template clip creation unit 73 retrieves existing promotional videos of other vehicles captured at locations selected by the user from the storage device 72. Then, the template clip creation unit 73 creates a template clip by referencing each clip that constitutes one of the existing promotional videos of other vehicles captured at locations selected by the user.

[0104] After the processing in step S23, the process proceeds to step S24. Alternatively, in step S22, when the template clipping creation unit 73 determines that the location selected by the user matches the location where the existing promotional video of the vehicle 43 selected by the user was captured, the process proceeds to step S24.

[0105] In step S24, the template clip creation unit 73 creates a template clip by referring to each clip of the existing promotional video that constitutes the vehicle 43 selected by the user.

[0106] In step S25, the template clip creation unit 73 sends the user selection information obtained in step S21 and the template clips created in steps S23 and S24 (only the template clips created in step S24 when step S23 is not executed) to the simulation processing unit 41 and the capture control processing unit 51 via the communication unit 74. After the processing in step S25, the template clip creation process ends.

[0107] Figure 11 It is used to describe in Figure 9 The flowchart shows the initial capture process performed in step S12.

[0108] In step S31, the initial capture processing unit 92 identifies the type of vehicle 43 and detects the stopping position of vehicle 43 relative to display device 42 by performing analysis processing on the captured video obtained by the camera 52 capturing vehicle 43.

[0109] In step S32, the initial capture processing unit 92 receives data from the communication unit 91. Figure 10 In step S25, the user selection information and template clipping are sent from the motion picture creation processing unit 31.

[0110] In step S33, the initial capture processing unit 92, based on the template clip received in step S32, moves the camera 52 to a position that allows it to capture the clip at the start of capture (e.g., as described above). Figure 2 The user enters the clip capture state of vehicle 43 as shown in the figure, and the corresponding view is used for capture.

[0111] In step S34, the initial capture processing unit 92 calibrates the arrangement of the camera 52, which is moved in step S33, based on the stopping position of the vehicle 43 relative to the display device 42 detected in step S31. For example, the initial capture processing unit 92 calibrates the arrangement of the camera 52 to correspond to the clip at the start of the capture, and to ensure that the background when capturing the vehicle 43 and the user is within the display range of the display device 42 (nothing outside the display range of the display device 42 is captured). Note that if reflections of the camera 52 (e.g., reflections only on the body of the vehicle 43) can be eliminated through 3DCG post-processing, this part can be ignored and passed to post-processing.

[0112] In step S35, the initial capture processing unit 92 determines whether there is a reflection of the camera 52 (e.g., a reflection including the body of the vehicle 43, glass, etc.) in the image obtained by the camera 52 that was captured in step S34.

[0113] In step S35, if the initial capture processing unit 92 determines that a reflection of the camera 52 exists, the process proceeds to step S36. In step S36, the initial capture processing unit 92 recalibrates the arrangement of the camera 52 to correspond to the clipping at the start of the capture, so that the background when capturing the vehicle 43 and the user is within the display range of the display device 42, and the reflection of the camera 52 is avoided.

[0114] The initial capture process ends after the processing in step S36, or when it is determined in step S35 that there is no reflection of camera 52.

[0115] Figure 12 It is used to describe in Figure 9 The flowchart of the simulator startup process executed in step S13.

[0116] In step S41, the simulator startup processing unit 82 receives the data via the communication unit 81. Figure 10 In step S25, user selection information is sent from template clipping creation unit 73, and assets corresponding to the location selected by the user are invoked based on the user selection information.

[0117] In step S42, the simulator startup processing unit 82 arranges the 3DCG model data included in the assets read in step S41 in the virtual space (see above). Figure 7 (A) enables the use of a virtual camera to capture in-simulator video corresponding to template clips.

[0118] In step S43, the simulator startup processing unit 82 arranges the virtual camera according to a layout that allows capture from a perspective corresponding to the clip captured when the capture begins in the virtual space, and then the simulator startup processing ends.

[0119] Figure 13 It is used to describe in Figure 9 The flowchart shows the video capture process performed in the simulator in step S17.

[0120] In step S51, the video capture processing unit 84 within the simulator determines whether vehicle 43 has started moving. Figure 9 In step S15, the vehicle 43 drives in the virtual space of the driving simulation and waits until it is determined that the vehicle 43 has started driving in the virtual space. Then, in step S51, when the video capture processing unit 84 in the simulator determines that the vehicle 43 has started driving in the virtual space, the process proceeds to step S52.

[0121] In step S52, the video capture processing unit 84 in the simulator selects the clips to be captured in sequence according to the template clips, and determines whether the vehicle 43 is driving in the virtual space within the capture range of the clips to be captured.

[0122] In step S52, when the video capture processing unit 84 in the simulator determines that the vehicle 43 is driving in the virtual space within the capture range of the clip to be captured, the processing proceeds to step S53.

[0123] In step S53, the video capture processing unit 84 in the simulator determines whether the clip to be captured includes the user based on the template clip.

[0124] In step S53, when the video capture processing unit 84 in the simulator determines that the clip to be captured includes the user's capture, the processing proceeds to step S54.

[0125] In step S54, the video capture processing unit 84 in the simulator notifies the camera system 23 via the communication unit 81 of a prompt to instruct the camera 52 to start capturing.

[0126] After the processing in step S54, or when it is determined in step S53 that the user is not included in the capture (i.e., only vehicle 43 is captured, so that the user is not shown), the processing proceeds to step S55.

[0127] In step S55, the simulator video capture processing unit 84 captures the clip to be captured through a virtual camera according to the template clip, and obtains the simulator video depicting the vehicle 43 driving in the virtual space.

[0128] After the processing in step S55, or when it is determined in step S52 that the vehicle 43 is not driving in the virtual space within the capture range of the clip to be captured (i.e., the vehicle 43 is driving in the virtual space outside the capture range of the clip to be captured), the processing proceeds to step S56.

[0129] In step S56, the video capture processing unit 84 within the simulator determines that vehicle 43 is in Figure 9 Whether the driving in the virtual space of the driving simulation performed in step S15 is completed.

[0130] If the video capture processing unit 84 in the simulator determines in step S56 that the vehicle 43 has not yet completed its journey in the virtual space (i.e., there is a clip showing that the vehicle 43 has not yet completed its journey in the virtual space), the processing returns to step S52 and repeats a similar process.

[0131] On the other hand, in step S56, when the video capture processing unit 84 in the simulator determines that the vehicle 43 has completed its driving in the virtual space, the processing proceeds to step S57. That is, in this case, the driving simulation performed by the simulation execution unit 83 ends.

[0132] In step S57, the simulator video capture processing unit 84 notifies the camera system 23 of the end of the driving simulation, and then the simulator video capture processing ends.

[0133] Figure 14 It is used to describe in Figure 9 The flowchart shows the automatic capture process performed in step S18.

[0134] In step S61, the automatic capture processing unit 93 receives data via the communication unit 91 from the target data. Figure 10 In step S25, a template clip is sent from the motion image creation processing unit 31, and a camera operation for controlling the operation of the camera 52 is derived based on the template clip and for each clip.

[0135] In step S62, the automatic capture processing unit 93, based on the template clips received in step S61, derives the timing for capture by the camera 52 for each clip.

[0136] In step S63, the automatic capture processing unit 93 determines whether a notification indicating the start of capture via camera 52 has been provided via video capture processing unit 84 within the simulator, and waits until it is determined that a notification indicating the start of capture has been provided. For example, in the above... Figure 13In step S54, when the video capture processing unit 84 in the simulator notifies the camera system 23 of a prompt instructing the camera 52 to start capturing, and the automatic capture processing unit 93 receives the prompt via the communication unit 91, the automatic capture processing unit 93 determines that a prompt instructing the camera 52 to start capturing has been provided through notification, and the process proceeds to step S64.

[0137] In step S64, the automatic capture processing unit 93 selects the clips to be captured sequentially according to the template clips, and moves the camera 52 to a position that allows capture from a perspective corresponding to the clips to be captured. Then, the automatic capture processing unit 93, based on the... Figure 11 In step S31, the camera 52 is positioned by calibrating the stop position of the vehicle 43 relative to the display device 42, as detected by the initial capture processing unit 92. For example, the automatic capture processing unit 93 calibrates the camera 52 to correspond to the clip to be captured, and ensures that the background when capturing the vehicle 43 and the user is within the display range of the display device 42 (areas outside the display range of the display device 42 are not captured).

[0138] In step S65, the automatic capture processing unit 93 determines whether there is a reflection of the camera 52 (e.g., a reflection including the body of the vehicle 43, glass, etc.) in the image obtained by the camera 52 that was captured in step S64.

[0139] In step S65, if the automatic capture processing unit 93 determines that a reflection of the camera 52 exists, the process proceeds to step S66. In step S66, the automatic capture processing unit 93 recalibrates the arrangement of the camera 52 to correspond to the clip to be captured, so that the background when capturing the vehicle 43 and the user is within the display range of the display device 42, and prevents the reflection of the camera 52.

[0140] After the processing in step S66, or when it is determined in step S65 that there is no reflection of camera 52, the processing proceeds to step S67.

[0141] In step S67, the automatic capture processing unit 93 determines whether the end of the driving simulation has been provided via notification from the video capture processing unit 84 within the simulator. For example, in the above... Figure 13 In step S57, when the video capture processing unit 84 in the simulator notifies the camera system 23 of the end of the driving simulation and the automatic capture processing unit 93 receives the notification via the communication unit 91, the automatic capture processing unit 93 determines that the end of the driving simulation has been provided through the notification.

[0142] In step S67, if the automatic capture processing unit 93 determines that the end of the driving simulation has not been provided by notification, the process returns to step S63 and a similar process is repeated.

[0143] On the other hand, in step S67, when the automatic capture processing unit 93 determines that the driving simulation has ended via notification, the automatic capture processing ends.

[0144] Figure 15 It is used to describe in Figure 9 The flowchart of the motion picture creation process performed in step S21.

[0145] In step S71, the clip selection unit 76 receives the data from the communication unit 74. Figure 9 In step S19, the captured video is sent from the camera system 23, and the video is also received from the camera system 23. Figure 9 In step S20, driving data and video from the driving simulation system 22 are sent.

[0146] In step S72, the editing selection unit 76 selects a virtual video that is similar to the driving situation of vehicle 43 in the driving simulation as indicated by the driving data received in step S71.

[0147] In step S73, the editing selection unit 76 displays the video received in the simulator and the captured video received in step S71, as well as the virtual production video selected in step S72, as options for allowing the user to select the video to be used in the content, on the UI terminal 32 via the UI control unit 71.

[0148] Then, when the user selects a video to use in the content from the simulator video, captured video, and virtual production video displayed as options, the UI control unit 71 supplies user selection information indicating the video selected by the user to the editing selection unit 76. In response, the editing selection unit 76 supplies the video selected by the user from the simulator video, captured video, and virtual production video to the motion picture creation unit 77, and the process proceeds to step S74.

[0149] In step S74, the motion picture creation unit 77 creates a motion picture by adjusting and editing the video supplied from the clip selection unit 76 in step S73 according to the length of the music indicated by the user selection information.

[0150] In step S75, the motion picture creation unit 77 uses the UI terminal 32 to provide the user with content that reproduces the music together with the motion picture created in step S74 via the UI control unit 71.

[0151] In step S76, the motion picture creation unit 77 provides the user with a link with an NFT for sharing content on an SNS, and then the motion picture creation process ends.

[0152] By performing the content creation process described above, the content creation system 11 can provide content created using motion images obtained by capturing the user's state while the user performs a driving simulation that traverses a virtual space corresponding to an existing promotional video of vehicle 43. Therefore, the content creation system 11 can provide a more valuable user experience.

[0153] Note that, in addition to the vehicle dealerships mentioned above, the content creation system 11 of this embodiment can also be installed in places such as restaurants, shopping malls, and entertainment facilities equipped with charging stations, and can be used to attract customers.

[0154] Computer configuration examples

[0155] Next, the above series of processes (information processing methods) can be performed using hardware or software. When the series of processes are performed using software, the program constituting the software is installed on a general-purpose computer or the like.

[0156] Figure 16 This is a block diagram illustrating a configuration example of a computer equipped with a program for performing the series of processes described above.

[0157] The program can be pre-recorded on a hard disk 105 or ROM 103, which is a recording medium built into the computer.

[0158] Alternatively, the program can be stored (recorded) in a removable recording medium 111 driven by drive 109. Such a removable recording medium 111 can be provided as so-called software packages. Examples of removable recording media 111 include floppy disks, compact disc read-only memory (CD-ROM), magneto-optical (MO) discs, digital multifunction discs (DVDs), magnetic disks, and semiconductor memory.

[0159] Note that the program can be installed on the computer from the aforementioned removable recording medium 111, or it can be downloaded to the computer and installed on the internal hard drive 105 via a communication network or broadcast network. That is, the program can be wirelessly transmitted from the download site to the computer via a satellite used for digital satellite broadcasting, or it can be wired to the computer via a network such as a local area network (LAN) or the Internet.

[0160] The computer has a built-in central processing unit (CPU) 102, and an input / output interface 110 is connected to the CPU 102 via a bus 101.

[0161] When a user inputs a command via the input / output interface 110 to the input unit 107, the CPU 102 executes the program stored in the read-only memory (ROM) 103 according to the command. Alternatively, the CPU 102 loads a program stored in the hard disk 105 into the random access memory (RAM) 104 and executes the program.

[0162] Therefore, CPU 102 executes the processing according to the flowchart above or the processing configured by the block diagram above. Then, as needed, CPU 102 outputs the processing result from output unit 106 via input / output interface 110, sends the processing result from communication unit 108, or records the processing result in hard disk 105.

[0163] Note that input unit 107 may include, for example, a keyboard, a mouse, and a microphone. Output unit 106 may include, for example, a liquid crystal display (LCD) and a speaker.

[0164] Here, in this specification, the processes executed by the computer according to the program are not necessarily executed sequentially in the order described in the flowchart. That is, the processes executed by the computer according to the program include processes executed in parallel or in a separate manner (e.g., parallel processing or processing by object).

[0165] A program can be processed by a single computer (processor) or by multiple computers in a distributed manner. A program can be transferred to a remote computer and executed.

[0166] Furthermore, in this specification, "system" refers to a collection of multiple constituent elements (devices, modules (components), etc.), and it is not important whether all constituent elements are housed in the same housing. Therefore, "system" can refer to both multiple devices housed in a separate housing and connected via a network, and multiple modules housed in a single housing for one device.

[0167] For example, a configuration described as a single device (or processing unit) can be partitioned and configured as multiple devices (or processing units). Conversely, a configuration described above as multiple devices (or processing units) can be integrated into a single device (or processing unit). It goes without saying that configurations other than those described above can be added to the configuration of each device (or each processing unit). When the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) can be included in the configuration of other devices (or other processing units).

[0168] Furthermore, for example, this technology can have a cloud computing configuration, in which a function is shared and processed collaboratively by multiple devices via a network.

[0169] For example, the above procedure can be executed by any device. In this case, the device only needs to have the necessary functions (function blocks, etc.) to obtain the required information.

[0170] For example, in addition to being executed by a single device, each step described in the flowchart above can be executed collaboratively by multiple devices. Furthermore, when a single step includes multiple processing operations, these multiple processing operations can be executed by a single device, or they can be shared and executed by multiple devices. In other words, multiple processing operations included in a single step can be executed as processing for multiple steps. Conversely, processing described as multiple steps can be executed together as a single step.

[0171] Note that a computer-executed program can be configured to execute the steps of the described program sequentially in the order described in this specification, or to execute the processes in parallel or separately at required timings (such as when called). That is, each step can be executed in an order different from the above-described order, provided that no contradiction occurs. Furthermore, the processing of the steps of the described program can be executed in parallel with the processing of other programs, or it can be executed in combination with the processing of other programs.

[0172] Note that, provided there is no contradiction, each of the various techniques described herein can be implemented independently in a single component. It should be understood that the various techniques described herein can be implemented in combination with any one of the various techniques. For example, part or all of the techniques described in any embodiment can be implemented in combination with part or all of the techniques described in other embodiments. Furthermore, part or all of any of the techniques described above can be implemented in combination with other techniques not described above.

[0173] Configuration combination examples

[0174] This technology can also be configured as described below. (1)

[0176] An information processing system, comprising: A driving simulation system configured to perform a driving simulation using a vehicle selected by a user, and to acquire driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space. A camera system configured to use cameras to capture vehicles and users to obtain captured video; and A motion image creation system configured to create motion images using at least the video within the simulator and the captured video. (2)

[0178] According to the information processing system described in (1), wherein, The motion image creation system includes a template clip creation unit that references each clip of an existing promotional video constituting a vehicle selected by the user to create template clips for instructing the capture of the in-simulator video and the captured video. The driving simulation system includes an in-simulator video capture and processing unit, which captures in-simulator video according to the template clipping. The camera system includes an automatic capture processing unit, which performs camera control on the camera capturing the captured video according to the template clipping. (3)

[0180] According to the information processing system described in (2), when the location selected by the user does not match the location where the existing promotional video of the vehicle is captured, the template clip creation unit creates the template clip by referring to each clip constituting the existing promotional video of other vehicles captured at the location selected by the user. (4)

[0182] According to the information processing system described in (2) or (3), the motion image creation system further includes a clip selection unit, which selects a virtual production video similar to the driving situation based on the driving data from a plurality of virtual production videos prepared in advance as material for advertising the vehicle, and allows the user to select a clip to be used in the creation of the motion image from a plurality of clips of the video in the simulator, the captured video and the virtual production video. (5)

[0184] According to the information processing system described in (4), the motion image creation system further includes a motion image creation unit, which creates the motion image by adjusting and editing the video of the plurality of clips selected by the user based on the length of music selected by the user. (6)

[0186] According to the information processing system described in (5), the motion picture creation system provides the user with content that reproduces the music together with the motion picture created by the motion picture creation unit, and provides the content to the user by assigning non-fungible tokens (NFTs) to links for sharing the content on social networking services (SNS). (7)

[0188] The information processing system according to any one of (2) to (6), wherein the driving simulation system further includes a display device that displays a simulated video of a location selected by the user during the execution of the driving simulation. (8)

[0190] According to the information processing system described in (7), the driving simulation system further includes a simulator startup processing unit, which arranges three-dimensional computer graphics (3DCG) model data included in the assets called according to the location selected by the user in a virtual space in which the vehicle is virtually driven in the driving simulation. (9)

[0192] According to the information processing system described in (8), the driving simulation system further includes a simulation execution unit that performs a driving simulation of virtually driving a vehicle based on the driving data, thereby enabling the capture of each clip corresponding to the template clip in a virtual space. (10)

[0194] According to the information processing system described in (9), the simulation execution unit generates a simulated video that is adjusted to appear as if the vehicle is actually being driven from the arrangement position of the vehicle traveling in the virtual space through the display device, and causes the display device to display the simulated video. (11)

[0196] The information processing system according to any one of (7) to (10), wherein the camera system further includes an initial capture processing unit that detects the vehicle type of the vehicle and detects the stopping position of the vehicle relative to the display device. (12)

[0198] According to the information processing system described in (11), the initial capture processing unit moves the camera to a position that allows capture at a viewpoint corresponding to the clip at the start of capture based on the template clip, and calibrates the camera position based on the vehicle's stopping position so that the background captured by the camera is within the display range of the display device. (13)

[0200] According to the information processing system described in (12), wherein, when a reflection of the camera exists, the initial capture processing unit recalibrates the camera arrangement to avoid the reflection. (14)

[0202] The information processing system according to any one of (11) to (13), wherein the automatic capture processing unit moves the camera to an arrangement that can capture at a viewpoint corresponding to the clip to be captured based on the template clip, and calibrates the camera arrangement based on the stopping position of the vehicle, such that the background captured by the camera is within the display range of the display device. (15)

[0204] According to the information processing system described in (14), when a reflection of the camera is present, the automatic capture processing unit recalibrates the camera arrangement to avoid the reflection. (16)

[0206] An information processing method, comprising: The information processing system uses a vehicle selected by the user to perform a driving simulation and acquires driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space. The information processing system uses cameras to capture images of vehicles and users to obtain captured video; and The information processing system uses at least the video within the simulator and the captured video to create motion images. (17)

[0208] A program for causing a computer in an information processing system to perform information processing, the information processing including: The driving simulation is performed using a vehicle selected by the user, and driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space are acquired. Using cameras to capture vehicles and users to obtain captured video; and At least the video within the simulator and the captured video are used to create motion images.

[0209] This embodiment is not limited to the above embodiment, and various types of modifications can be made without departing from the spirit of this disclosure. Furthermore, the effects described in this specification are merely examples and not limitations, and other effects can be obtained.

[0210] Reference tag list

[0211] 11 Content Production System, 21 Motion Picture Creation System, 22 Driving Simulation System, 23 Camera System, 24 Cloud Server, 25 Network, 31 Motion Picture Creation Processing Unit, 32 UI Terminal, 41 Simulation Processing Unit, 42 Display Device, 43 Vehicle, 51 Capture Control Processing Unit, 52 Camera, 53 Bracket, 71 UI Control Unit, 72 Storage Device, 73 Template Clip Creation Unit, 74 Communication Unit, 75 Processing Control Unit, 76 Clip Selection Unit, 77 Motion Picture Creation Unit, 81 Communication Unit, 82 Simulator Startup Processing Unit, 83 Simulation Execution Unit, 84 In-Simulator Video Capture Processing Unit, 91 Communication Unit, 92 Initial Capture Processing Unit, 93 Automatic Capture Processing Unit.

Claims

1. An information processing system, comprising: A driving simulation system configured to perform a driving simulation using a vehicle selected by a user, and to acquire driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space. A camera system configured to use cameras to capture vehicles and users to obtain captured video; as well as A motion image creation system configured to create motion images using at least the video within the simulator and the captured video.

2. The information processing system according to claim 1, wherein, The motion image creation system includes a template clip creation unit that references each clip of an existing promotional video constituting a vehicle selected by the user to create template clips for instructing the capture of the in-simulator video and the captured video. The driving simulation system includes an in-simulator video capture and processing unit, which captures in-simulator video according to the template clipping. The camera system includes an automatic capture processing unit, which performs camera control on the camera capturing the captured video according to the template clipping.

3. The information processing system according to claim 2, wherein, When the location selected by the user does not match the location where the vehicle's existing promotional video was captured, the template clip creation unit creates the template clip by referencing each clip that constitutes an existing promotional video of another vehicle captured at the location selected by the user.

4. The information processing system according to claim 2, wherein, The motion picture creation system also includes a clip selection unit that selects a virtual production video similar to the driving situation based on the driving data from a plurality of virtual production videos prepared in advance as material for advertising the vehicle, and allows the user to select a clip to be used in the creation of the motion picture from a plurality of clips of the video in the simulator, the captured video and the virtual production video.

5. The information processing system according to claim 4, wherein, The motion picture creation system further includes a motion picture creation unit that creates motion pictures by adjusting and editing the multiple video clips selected by the user based on the length of music selected by the user.

6. The information processing system according to claim 5, wherein, The motion picture creation system provides users with content that reproduces the music together with motion pictures created by the motion picture creation unit, and provides the content to users by assigning non-fungible tokens (NFTs) to links for sharing the content on social networking services (SNS).

7. The information processing system according to claim 2, wherein, The driving simulation system also includes a display device that displays simulated videos of locations selected by the user during the execution of the driving simulation.

8. The information processing system according to claim 7, wherein, The driving simulation system also includes a simulator startup processing unit, which arranges three-dimensional computer graphics (3DCG) model data included in assets called according to the location selected by the user in a virtual space where the vehicle is virtually driven in the driving simulation.

9. The information processing system according to claim 8, wherein, The driving simulation system also includes a simulation execution unit, which performs a virtual driving simulation of a vehicle based on the driving data, enabling the capture of each clip corresponding to the template clip in virtual space.

10. The information processing system according to claim 9, wherein, The simulation execution unit generates a simulated video that is tuned to appear as if the vehicle is actually being driven from its position in the virtual space through the display device, and causes the display device to display the simulated video.

11. The information processing system according to claim 7, wherein, The camera system also includes an initial capture processing unit that detects the vehicle type and the vehicle's stopping position relative to the display device.

12. The information processing system according to claim 11, wherein, The initial capture processing unit moves the camera to a position that allows it to capture from a perspective corresponding to the clip at the start of capture, based on the template clip, and calibrates the camera's position based on the vehicle's stopping position, so that the background captured by the camera is within the display range of the display device.

13. The information processing system according to claim 12, wherein, When a camera reflection is present, the initial capture processing unit recalibrates the camera arrangement to avoid the reflection.

14. The information processing system according to claim 11, wherein, The automatic capture processing unit moves the camera to a position that allows it to capture the clip from a perspective corresponding to the clip to be captured, based on the template clip, and calibrates the camera's position based on the vehicle's stopping position, so that the background captured by the camera is within the display range of the display device.

15. The information processing system according to claim 14, wherein, When a camera reflection is present, the automatic capture processing unit recalibrates the camera arrangement to avoid the reflection.

16. An information processing method, comprising: The information processing system uses a vehicle selected by the user to perform a driving simulation and acquires driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space. The information processing system uses cameras to capture images of vehicles and users to obtain captured video. as well as The information processing system uses at least the video within the simulator and the captured video to create motion images.

17. A program for causing a computer of an information processing system to perform information processing, the information processing comprising: The driving simulation is performed using a vehicle selected by the user, and driving data corresponding to the user's operation of the vehicle and in-simulator video depicting the vehicle driving in virtual space are acquired. Use cameras to capture vehicles and users to obtain video footage; as well as At least the video within the simulator and the captured video are used to create motion images.

Images