Production methods, production equipment, and programs

Abstract

This disclosure aims to provide a presentation method that can enhance the sense of immersion in content in a limited space such as the interior of a vehicle. [Solution] A performance method according to one aspect of the present disclosure involves acquiring content data including sound data, inputting the sound data into a performance mode determination model, obtaining a determination result from the performance mode determination model, and having a computer perform actions such as emitting light from a light source or vibrating a vibrating part based on the determination result.

Description

Technical Field

[0001] The present disclosure relates to a rendering method, a rendering device, and a program.

Background Art

[0002] Devices for enhancing rendering in a vehicle cabin have been proposed. Patent Document 1 describes a technology that changes the brightness of lamp illumination, the presence or absence of lighting, and the interval and speed of blinking according to the content of a movie when the movie is played in the vehicle cabin. Further, Patent Document 2 describes a technology that generates sound corresponding to the traveling of a vehicle based on information related to the traveling of the vehicle or operations on the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] Since the spatial expanse in a vehicle cabin is limited, it is required to perform rendering with high affinity for content such as video and audio played in the vehicle cabin so that users can obtain a sense of immersion.

[0005] The device described in Patent Document 1 can control illumination according to the content of a movie. On the other hand, the present inventors have found that even if illumination is controlled based on the content of a movie, a sense of immersion may be difficult to obtain, or an inappropriate added rendering may impair the sense of immersion.

[0006] Further, the device described in Patent Document 2 is intended to improve the sense of immersion in vehicle driving, and has a problem from the viewpoint of improving the sense of immersion in content played in the vehicle cabin.

[0007] One aspect of this disclosure aims to provide a presentation method that can enhance the sense of immersion in content in a limited space such as the interior of a vehicle. [Means for solving the problem]

[0008] (1) A performance method according to one aspect of the present disclosure involves a computer acquiring content data including sound data, inputting the sound data into a performance mode determination model, obtaining a determination result from the performance mode determination model, and, based on the determination result, emitting light from a light source or vibrating a vibrating part.

[0009] (2) In the performance method described in (1) above, the computer may determine whether the content data includes video data, and if the content data includes video data, input the sound data and the video data into a first performance mode determination model and obtain a determination result from the first performance mode determination model. If the content data does not include video data, input the sound data into a second performance mode determination model and obtain a determination result from the second performance mode determination model.

[0010] (3) In the performance method of (2) above, the first performance mode determination model includes a scene discrimination model, and the scene discrimination model may include a machine learning model trained to discriminate a scene of content from the sound data and determine a light emission pattern or vibration pattern based on the scene.

[0011] (4) In the performance method of (2) or (3) above, the first performance mode determination model may determine the light emission position or vibration position based on the video data.

[0012] (5) In any of the performance methods described in (2) to (4) above, the determination result in the first performance mode determination model is obtained by transmitting command data based on the sound data and stream data based on the video data separately and then combining them.

[0013] (6) In any of the performance methods described in (2) to (5) above, the second performance mode determination model includes a song discrimination model and a pattern selection model, wherein the song discrimination model includes a machine learning model trained to discriminate at least one of the genre and style of the content from the sound data, and the pattern selection model is preferably a rule-based model.

[0014] (7) In any of the performance methods described in (1) to (6) above, it is preferable to emit light or vibrate inside the vehicle.

[0015] (8) In the lighting method described in (7) above, it is preferable to illuminate at least one of the following locations in the vehicle interior: monitor, sun visor, center console, floor, door trim, or dashboard.

[0016] (9) In the performance method described in (7) or (8) above, it is preferable to vibrate at least one of the following locations in the vehicle interior: seat, armrest, headrest, floor, steering wheel, or seat belt.

[0017] (10) A performance device according to another aspect of the present disclosure includes an acquisition unit that acquires content data including sound data, a processing unit that inputs the sound data to a performance mode determination model and acquires a determination result from the performance mode determination model, and a light source that emits light based on the determination result, or a vibrating unit that vibrates based on the determination result.

[0018] (11) The performance device of (10) further comprises a determination unit that determines whether or not the content data includes video data, and the processing unit may include a first processing unit that inputs the sound data and the video data to a first performance mode determination model and obtains a determination result from the first performance mode determination model when the content data includes the video data, and a second processing unit that inputs the sound data to a second performance mode determination model and obtains a determination result from the second performance mode determination model when the content data does not include the video data.

[0019] (12) In the performance device of (11), the first performance mode determination model includes a scene discrimination model, which may include a machine learning model trained to discriminate a scene of content from the sound data and determine a light emission pattern or vibration pattern based on the scene.

[0020] (13) In the performance device of (11) or (12) above, the first performance mode determination model may determine the light emission position or vibration position based on the video data.

[0021] (14) In any of the performance devices described in (11) to (13), the determination result in the first performance mode determination model is obtained by transmitting command data based on the sound data and stream data based on the video data separately and then combining them.

[0022] (15) In any of the performance devices described in (11) to (14), the second performance mode determination model includes a song discrimination model and a pattern selection model, wherein the song discrimination model includes a machine learning model trained to discriminate at least one of the genre and style of the content from the sound data, and the pattern selection model is preferably a rule-based model.

[0023] (16) In any of the performance devices described in (10) to (15) above, it is preferable to emit light or vibrate inside the vehicle.

[0024] (17) The rendering device according to the above (16), wherein the light source is preferably arranged on at least one of a monitor, a sun visor, a center console, a floor, a door trim, and a dashboard in the vehicle interior.

[0025] (18) The rendering device according to the above (16) or (17), wherein the vibrating portion is preferably arranged on at least one of a seat, an armrest, a headrest, a floor, a steering wheel, and a seat belt in the vehicle interior.

[0026] (19) A program according to another aspect of the present disclosure causes a computer to execute: acquiring content data including sound data; inputting the sound data into a rendering mode determination model; acquiring a determination result from the rendering mode determination model; and causing a light source to emit light or causing a vibrating portion to vibrate based on the determination result. Advantageous Effects of the Invention

[0027] The rendering method according to one aspect of the present disclosure can improve the sense of immersion in content in a limited space such as a vehicle interior. Brief Description of the Drawings

[0028] [Figure 1] Figure 1 is a flowchart illustrating a rendering method according to one aspect of the present disclosure. [Figure 2] Figure 2 is a block diagram illustrating a rendering device according to one aspect of the present disclosure. [Figure 3] Figure 3 is a diagram for explaining an example of a transmission method of determination results in the rendering mode determination model of Figure 2. [Figure 4] Figure 4 is a block diagram illustrating an example of a music discrimination model in the rendering device of Figure 2. [Figure 5] Figure 5 is a block diagram illustrating an example of a pattern selection model in the rendering device of Figure 2. [Figure 6]Figure 6 is a schematic diagram showing an example of the arrangement of the light source and vibrating part in the performance device shown in Figure 2. [Modes for carrying out the invention]

[0029] The embodiments of this disclosure will be described in detail below, with reference to the drawings as appropriate. In this disclosure, the designations "First" and "Second" are for distinguishing the components to which they are attached and do not limit the number, order, priority, etc.

[0030] [First Embodiment] <Direction method> As shown in Figure 1, a performance method according to one aspect of the present disclosure involves a computer acquiring content data including sound data (S1), inputting the sound data into a performance mode determination model (S3 and S13), obtaining a determination result from the performance mode determination model (S4 and S14), and, based on the determination result, causing a light source to emit light or a vibrating part to vibrate (S5). In this disclosure, "causing a light source to emit light or a vibrating part to vibrate" means performing at least one of the following: emitting light from a light source or vibrating a vibrating part, and includes cases where both emitting light from a light source and vibrating a vibrating part are performed.

[0031] The inventors have found that in limited spaces such as the interior of a vehicle, determining the presentation style based on data other than sound data can make it difficult to achieve a sense of immersion. For example, relying solely on video data may result in bright effects being applied based on the light of a camera flash, even in quiet scenes. In contrast, this presentation method uses the sound data as input data to the presentation style determination model, thereby improving the sense of immersion in the content in limited spaces such as the interior of a vehicle.

[0032] In the said performance method, the computer may determine whether the content data includes video data (S2). In this case, if the content data includes video data, the computer inputs the sound data and the video data into a first performance mode determination model (S3) and obtains a determination result from the first performance mode determination model (S4). If the content data does not include video data, the computer inputs the sound data into a second performance mode determination model (S13) and obtains a determination result from the second performance mode determination model (S14).

[0033] In this presentation method, when the content data includes video data and audio data, the audio data and the video data are used as input data to the first presentation mode determination model, thereby enabling a presentation that harmonizes with the video data and further enhances the sense of immersion in the content.

[0034] <Production device> This performance method can be performed using the performance device 1 shown in Figure 2. The performance device 1 includes a computer. The hardware configuration of the computer is not particularly limited, but may include, for example, a processor, a storage device and a communication device connected to each other via a bus, and may also include a drive device, a user interface device, etc.

[0035] The processor executes each step of the performance method according to the program, parameters, etc., stored in the memory device. The processor may include one or more of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and processing circuitry.

[0036] The storage device stores programs executed by the processor and various data used by the processor. The storage device may include known storage media such as semiconductor storage media and magnetic storage media. The storage device may also be configured so that the processor can read from and write to it via a communication network such as the Internet.

[0037] The communication device is configured to communicate with the light source, the vibrating unit, and the like.

[0038] Before explaining each step of the aforementioned performance method, we will first describe the performance device capable of executing this method.

[0039] As shown in Figure 2, the performance device 1 comprises an acquisition unit 10 that acquires content data including sound data, processing units 30 and 40 that input the sound data to performance mode determination models 31 and 41 and acquire determination results from the performance mode determination models 31 and 41, and a light source that emits light based on the determination result, or a vibrating unit that vibrates based on the determination result (hereinafter, the light source and the vibrating unit are collectively referred to as the "performance unit 50").

[0040] The performance device 1 may further include a determination unit 20 that determines whether or not the content data includes video data. In this case, the processing units 30 and 40 may include a first processing unit 30 and a second processing unit 40. The first processing unit 30 inputs the sound data and the video data to a first performance mode determination model 31 when the content data includes video data, and obtains a determination result from the first performance mode determination model 31. The second processing unit 40 inputs the sound data to a second performance mode determination model 41 when the content data does not include video data, and obtains a determination result from the second performance mode determination model 41.

[0041] In the performance device 1, the functions of the acquisition unit 10, the determination unit 20, the first processing unit 30, and the second processing unit 40 may be realized by the execution of a program stored in the storage device by the processor.

[0042] The performance device 1 can improve the sense of immersion in content in a limited space such as inside a vehicle by using the sound data as data input to the performance mode determination models 31 and 41.

[0043] Furthermore, if the content data includes video data and sound data, the performance device 1 can use the sound data and the video data as data input to the first performance mode determination model 31, thereby enabling performances that are in harmony with the video data and further improving the sense of immersion in the content.

[0044] The performance device 1 is suitable for creating a highly immersive experience in a limited space such as the interior of a vehicle. When the performance device 1 is used to create an experience inside a vehicle, the computer may be built into or externally mounted on an in-vehicle device such as a car navigation system. Below, as an example, we will describe the case where the computer is built into a car navigation system.

[0045] (Acquisition Department) The acquisition unit 10 acquires the content data. The content data may be acquired from a portable or internal storage medium. Alternatively, the content data may be acquired from a communication network such as the Internet.

[0046] The aforementioned content data includes sound data. The aforementioned content data may include video data in addition to the sound data, or it may not include video data. If the aforementioned content data includes video data, the specific content may be, for example, a movie, a sports video, a live performance video, a music video, a game video, or a television program. If the aforementioned content data does not include video data, the specific content may be, for example, audio content such as music or a reading.

[0047] (Judgment Department) If the content data includes video data, the determination unit 20 sends the audio data and the video data to the first processing unit 30. On the other hand, if the content data does not include video data, the determination unit 20 sends the audio data to the second processing unit 40.

[0048] (First Processing Unit) The first processing unit 30 includes a first performance mode determination model 31. The first performance mode determination model 31 also includes a scene discrimination model 32 and a performance signal generation unit 33.

[0049] The scene discrimination model 32 preferably includes a machine learning model trained to discriminate a scene of content from the sound data and determine a light emission pattern or vibration pattern based on the scene. This embodiment allows for a more appropriate improvement in immersion depending on the type of content.

[0050] The scene discrimination model 32 may discriminate scenes without using the video data, or it may discriminate scenes using only the audio data. The scenes discriminated by the scene discrimination model 32 may be scenes expressed in the content (such as conversation scenes or exciting scenes), types of content (such as movies, sports, live performances, music videos, games, or television programs), further subdivisions of the types of content (for example, in the case of music videos, genres of music), or combinations thereof.

[0051] The machine learning algorithm for the scene discrimination model 32 is not particularly limited and may use a neural network or a support vector machine. The scene discrimination model 32 may use a support vector machine as its algorithm and be trained using training data in which specific scenes are labeled in the audio data of the practice content.

[0052] Furthermore, the first performance mode determination model 31 may be configured to determine the light emission position or vibration position based on the video data. This configuration can more appropriately improve the sense of immersion when the content includes video.

[0053] When the first performance mode determination model 31 determines the light emission position or vibration position based on the video data, the first performance mode determination model 31 may divide the video displayed on an in-vehicle device such as a car navigation system into multiple regions. In this case, the first performance mode determination model 31 may assign each of the divided regions to a specific light source or a specific vibration unit. Furthermore, the first performance mode determination model 31 may control the light emission mode (e.g., intensity) of the assigned light source or the vibration mode (e.g., vibration intensity) of the vibration unit based on the optical characteristics (e.g., brightness) of the region.

[0054] The first performance mode determination model 31 may be configured to determine brightness based on the video data. For example, when the scene determination model 32 determines that the scene has changed, the brightness of each region may be set as the target brightness, and the brightness may be gradually changed to reach that brightness.

[0055] The performance signals for the performance unit 50 (signals for causing the light source to emit light or the vibration unit to vibrate) are generated by the performance signal generation unit 33. The performance signal generation unit 33 receives, for example, the video data acquired by the acquisition unit 10 and the discrimination result from the scene discrimination model 32, and generates performance signals based on this data.

[0056] The performance signal generation unit 33, while taking into account the discrimination results of the scene discrimination model 32, assigns each of the divided regions as described above to the specific light source or the specific vibrating part, and determines the emission mode of the light source or the vibration mode of the vibrating part. The performance signal generation unit 33 may, for example, determine the division pattern of the image displayed on the in-vehicle device from the discrimination results of the scene discrimination model 32, or it may determine what kind of performance to perform for each region. The performance signal generation unit 33 may, for example, determine the color or vibration intensity for each region, or it may determine the timing of the emission or vibration. Furthermore, depending on the discrimination results of the scene discrimination model 32, the performance signal generation unit 33 may decide not to perform any performance for one or more regions, or for all regions.

[0057] [Transmission Method] As shown in Figure 3, in the performance device 1, the determination result in the first performance mode determination model 31 is preferably obtained by transmitting command data based on the sound data and stream data based on the video data separately, and then combining them. The command data is, for example, data on the determination result by the scene determination model 32. According to this configuration, video and sound can be easily synchronized to produce appropriate light emission or vibration.

[0058] In the performance device 1, the amount of data differs significantly between the command data based on the sound data and the stream data based on the video data. Therefore, the performance device 1 is required to be able to transmit both simultaneously. In this regard, the determination result in the first performance mode determination model 31 can be transmitted separately as command data based on the sound data and stream data based on the video data, and then combined, thereby enabling appropriate transmission of both. The command data and the stream data may be transmitted separately and then transmitted through a shared harness from the same bus interface 60. In this case, by performing communication from the bus interface 60 using, for example, the automotive audio bus "A2B" (registered trademark), the combined command data and stream data can be easily transmitted.

[0059] (Second Processing Unit) The second processing unit 40 includes a second performance mode determination model 41. The second performance mode determination model 41 also includes a music discrimination model 42, a pattern selection model 43, an interlocking unit 44, and a performance signal generation unit 45. The second performance mode determination model 41 may determine the performance mode using only the sound data.

[0060] The music discrimination model 42 preferably includes a machine learning model trained to discriminate at least one of the genre and style of the content from the sound data. Furthermore, the pattern selection model 43 is preferably a rule-based model. According to this embodiment, immersion can be more appropriately improved for content that does not include video.

[0061] The music discrimination model 42 is preferably configured to discriminate both the genre and style of the content. The music discrimination model 42 may include, for example, a machine learning model trained to discriminate the genre of the content and a rule-based model configured to discriminate the style of the content.

[0062] As shown in Figure 4, the musical characteristics that can be identified by the musical style recognition model 42 include, for example, tempo, chords, modulation, and beat. It is preferable that the musical style recognition model 42 is configured to identify one or more of these musical characteristics, and more preferably that it is configured to identify all of tempo, chords, modulation, and beat.

[0063] The machine learning algorithm for the song discrimination model 42 is not particularly limited and may use a neural network or a support vector machine. The song discrimination model 42 may use a neural network as its algorithm and be trained using training data in which sound data in practice content is labeled with genre (jazz, rock, classical, pop, etc.).

[0064] The pattern selection model 43 selects a pattern of light emission or vibration based on the discrimination result of the music discrimination model 42. The pattern selection model 43 may be a rule-based model as described above. The pattern selection model 43 has a list of pattern combinations corresponding to the discrimination result of the music discrimination model 42, as shown in Figure 5, for example. This combination list may be stored in the storage device. The pattern selection model 43 selects an appropriate pattern from the combination list stored in the storage device according to the discrimination result of the music discrimination model 42. The combination list may be a combination of, for example, a light emission or vibration arrangement pattern (a pattern of which light source to emit light or which vibrating part to vibrate) and a color pattern for light emission or an intensity pattern for vibration.

[0065] The interlocking unit 44 determines the timing for changing the light emission or vibration based on the discrimination result of the music discrimination model 42. The interlocking unit 44 may also change the rate of change in the light emission or vibration in conjunction with the tempo or other characteristics of the music determined by the music discrimination model 42. Specifically, the interlocking unit 44 may change the position of the light source to emit light in conjunction with the tempo or other characteristics of the music determined by the music discrimination model 42. Furthermore, the rate of change in the light emission may be the rate of change in brightness of each light source, and the interlocking unit 44 may gradually change the brightness to achieve a target brightness.

[0066] The performance signals for the performance unit 50 are generated by the performance signal generation unit 45. The performance signal generation unit 45 receives, for example, the selection result from the pattern selection model 43 and the decision result from the interlocking unit 44, and generates performance signals based on this data.

[0067] (Production Department) The display device 1 is preferably illuminated or vibrated inside the vehicle. The display device 1 is suitable for improving the sense of immersion inside the vehicle.

[0068] Refer to Figure 6 to see an example of the arrangement of the light source and vibrating unit in the performance device 1.

[0069] The light source may be, for example, an LED light source. The light source may be located in at least one of the following locations in the vehicle interior: monitor 70a, sun visor 70b (including the sun visor 70b itself and the area around the sun visor 70b), center console 70c, floor 70d, door trim 70e, or dashboard 70f. This configuration further enhances the sense of immersion in the vehicle interior. When the light source is located on monitor 70a, it may be located on the back of monitor 70a, on the side of monitor 70a, or on the edge of monitor 70a.

[0070] The vibrating part is preferably located in at least one of the following in the vehicle interior: seat 80a, armrest 80b, headrest 80c, floor 70d, steering wheel 80d, or seat belt (not shown). This embodiment further enhances the sense of immersion in the vehicle interior.

[0071] Next, we will explain in detail each step of the aforementioned performance method.

[0072] (S1) S1 (hereinafter also referred to as "content data acquisition step S1") is performed by the acquisition unit 10. In content data acquisition step S1, content data including sound data and video data may be acquired, or content data including sound data but not video data may be acquired.

[0073] (S2) Step S2 (hereinafter also referred to as the "determination step") is performed by the determination unit 20. In determination step S2, if the content data includes the video data, the sound data and video data included in the content data are sent to the first processing unit 30. On the other hand, in determination step S2, if the content data does not include the video data, the sound data included in the content data is sent to the second processing unit 40.

[0074] (S3) Step S3 (hereinafter also referred to as "first input step S3") is performed by the first processing unit 30. In the first input step S3, the sound data and video data sent from the determination step S2 are input to the first performance mode determination model 31.

[0075] (S4) Step S4 (hereinafter also referred to as "first decision result acquisition step S4") is performed by the first performance mode determination model 31. In the first decision result acquisition step S4, the sound data input in the first input step S3 is input to the scene discrimination model 32, and a discrimination result is obtained from this scene discrimination model 32. In addition, in the first decision result acquisition step S4, the light emission position or vibration position is determined based on the video data input in the first input step S3. If the performance device 1 has light sources at multiple positions, in the first decision result acquisition step S4, a specific region divided in the video displayed on an in-vehicle device such as a car navigation system may be assigned to a specific light source or a specific vibration part, and the light emission of the assigned light source or the vibration of the vibration part may be controlled based on the optical characteristics (e.g., brightness) of the region.

[0076] Furthermore, in the first decision result acquisition step S4, a performance signal is generated for the performance unit 50. The performance signal is generated by the performance signal generation unit 33. In the first decision result acquisition step S4, for example, the performance signal is generated based on the video data input to the first performance mode determination model 31 and the determination result by the scene discrimination model 32.

[0077] (S13) S13 (hereinafter also referred to as "second input step S13") is performed by the second processing unit 40. In the second input step S13, the sound data sent from the determination step S2 is input to the second performance mode determination model 41.

[0078] (S14) Step S14 (hereinafter also referred to as "second decision result acquisition step S14") is performed by the second performance mode determination model 41. In the second decision result acquisition step S14, the sound data input in the second input step S13 is input to the music discrimination model 42, and the discrimination result is obtained from this music discrimination model 42. In addition, in the second decision result acquisition step S14, the discrimination result from the music discrimination model 42 is input to the pattern selection model 43, and the selection result is obtained from this pattern selection model 43. Furthermore, in the second decision result acquisition step S14, the discrimination result from the music discrimination model 42 is input to the interlocking unit 44, and information on the timing of changing the color or vibration is obtained.

[0079] Furthermore, in the second decision result acquisition step S14, a performance signal is generated for the performance unit 50. The performance signal is generated by the performance signal generation unit 45. In the second decision result acquisition step S14, for example, the performance signal is generated based on the selection result by the pattern selection model 43 and the information from the interlocking unit 44.

[0080] (S5) Step S5 (hereinafter also referred to as "performance step S5") is performed by the performance unit 50. In performance step S5, the performance signal generated in the first decision result acquisition step S4 or the second decision result acquisition step S14 is received, and one or more of the light sources are made to emit light, or one or more of the vibrating parts are made to vibrate.

[0081] <Program> The performance method may be implemented by a computer program being executed by a processor. A program according to one aspect of the present disclosure causes a computer to perform the following actions: acquire content data including sound data; input the sound data into performance mode determination models 31 and 41; acquire a determination result from the performance mode determination models 31 and 41; and, based on the determination result, cause a light source to emit light or a vibrating part to vibrate.

[0082] The program may cause the computer to determine whether the content data includes video data, input the sound data and the video data into the first performance mode determination model 31 if the content data includes video data, and obtain a determination result from the first performance mode determination model 31 if the content data does not include video data, and input the sound data into the second performance mode determination model 41 and obtain a determination result from the second performance mode determination model 41.

[0083] According to this program, by using the aforementioned sound data as input data to the performance mode determination models 31 and 41, it is possible to improve the sense of immersion in the content in a limited space such as inside a vehicle.

[0084] Furthermore, according to the program, if the content data includes video data and audio data, by using the audio data and the video data as input data to the first performance mode determination model 31, it is possible to perform performances that are in harmony with the video data, thereby further improving the sense of immersion in the content.

[0085] [Other embodiments] The embodiments described above do not limit the configuration of the present invention. Therefore, the embodiments may omit, substitute, or add components of each part of the embodiments based on the description herein and common technical knowledge, and all such additions should be interpreted as falling within the scope of the present invention.

[0086] In the above embodiment, a method was described in which a computer determines whether or not video data is included and uses a first performance mode determination model and a second performance mode determination model depending on the result of this determination. However, in this disclosure, if, for example, it is determined whether or not content data includes video data, it is also possible to adopt a configuration in which only one of the first performance mode determination model and the second performance mode determination model is included as the performance mode determination model.

[0087] In the above embodiment, the case in which the scene discrimination model is a machine learning model was described. However, in this disclosure, the scene discrimination model may be, for example, a rule-based model. Furthermore, in this disclosure, the above-mentioned music discrimination model, pattern selection model, interlocking unit, etc. may be included as part of the first performance mode determination model. In addition, the first performance mode determination model may be configured to adjust the light emission pattern and vibration pattern based on the video data.

[0088] In the above embodiment, the case in which the song discrimination model includes a machine learning model and the pattern selection model is a rule-based model was described. However, in this disclosure, the song discrimination model may not include a machine learning model, or it may consist only of a machine learning model. Also, the pattern selection model may include a machine learning model. Furthermore, in this disclosure, the above-mentioned scene discrimination model may be included as part of the second performance mode determination model.

[0089] In the above embodiment, a manner in which the image is displayed on an in-vehicle device such as a car navigation system was described. However, in this disclosure, the image may also be displayed on a mobile device such as a smartphone or tablet located inside the vehicle. In this case, the image may be displayed only on the mobile device, or on both the mobile device and the in-vehicle device. When the image is displayed on the mobile device, the mobile device constitutes a monitor on which the image is displayed. Furthermore, when the image is displayed on the mobile device, the in-vehicle device does not need to be located inside the vehicle.

[0090] Furthermore, in this disclosure, the performance method may be controlled so that it is not performed while the vehicle is in operation. This control may be performed by a control device on the vehicle side.

[0091] The display method and display apparatus of this disclosure are suitable for emitting light or vibrating within a vehicle interior. On the other hand, the display method and display apparatus of this disclosure can be suitably used in certain limited spaces other than vehicle interiors. Examples of such limited spaces include private booths in internet cafes, karaoke boxes, and soundproof rooms in homes. [Explanation of Symbols]

[0092] 1 Performance device 10 Acquisition Department 20 Judgment section 30 Processing Unit (First Processing Unit) 31. Model for Determining the Performance Style (First Model for Determining the Performance Style) 32 Scene Discrimination Models 33. Performance signal generation unit 40 Processing Unit (Second Processing Unit) 41. Production Style Determination Model (Second Production Style Determination Model) 42. Song Recognition Model 43 Pattern Selection Models 44 Interlocking part 45. Performance signal generation unit 50 Production Department 60 bus interface 70a monitor 70b Sun visor 70c Center Console 70d floor 70e door trim 70f Dashboard 80a sheet 80b Armrest 80c headrest 80d steering wheel

Claims

1. Retrieve content data including sound data, The aforementioned sound data is input into the performance mode determination model. The decision result is obtained from the aforementioned performance mode determination model, Based on the aforementioned determination result, the light source is made to emit light, or the vibrating part is made to vibrate. The computer executes Direction method.

2. The aforementioned computer, Determine whether the aforementioned content data includes video data. If the content data includes the video data, the sound data and the video data are input to the first performance mode determination model, and a determination result is obtained from the first performance mode determination model. The performance method according to claim 1, wherein if the content data does not include the video data, the sound data is input to a second performance mode determination model, and a determination result is obtained from the second performance mode determination model.

3. The first performance mode determination model includes a scene discrimination model, The performance method according to claim 2, wherein the scene discrimination model includes a machine learning model trained to discriminate a scene of content from the sound data and determine a light emission pattern or vibration pattern based on the scene.

4. The first performance mode determination model determines the light emission position or vibration position based on the video data, according to claim 2.

5. The performance method according to claim 2, wherein the determination result in the first performance mode determination model is obtained by transmitting command data based on sound data and stream data based on video data separately and then combining them.

6. The second performance mode determination model includes a song discrimination model and a pattern selection model. The aforementioned music discrimination model includes a machine learning model trained to determine at least one of the genre and style of the content from the audio data. The presentation method according to claim 2, wherein the pattern selection model is a rule-based model.

7. A method of creating a visual effect according to any one of claims 1 to 6, which involves emitting light or vibrating within the interior of a vehicle.

8. The method of displaying an effect according to claim 7, wherein the light is emitted from at least one of the following locations within the vehicle interior: monitor, sun visor, center console, floor, door trim, or dashboard.

9. The presentation method according to claim 7, wherein the vibration occurs at least at one of the following locations within the vehicle interior: seat, armrest, headrest, floor, steering wheel, or seat belt.

10. An acquisition unit that acquires content data including sound data, A processing unit that inputs the aforementioned sound data into a performance mode determination model and obtains a determination result from the performance mode determination model, A light source that emits light based on the aforementioned determination result, or a vibrating part that vibrates based on the aforementioned determination result. A performance device equipped with the following features.

11. The system further includes a determination unit that determines whether or not the content data includes video data. The aforementioned processing unit, A first processing unit inputs the sound data and the video data to a first performance mode determination model and obtains a determination result from the first performance mode determination model when the content data includes the video data, The performance device according to claim 10, further comprising a second processing unit that inputs the sound data to a second performance mode determination model and obtains a determination result from the second performance mode determination model when the content data does not include the video data.

12. The first performance mode determination model includes a scene discrimination model, The performance device according to claim 11, wherein the scene discrimination model includes a machine learning model trained to discriminate a scene of content from the sound data and determine a light emission pattern or vibration pattern based on the scene.

13. The performance device according to claim 11, wherein the first performance mode determination model determines the light emission position or vibration position based on the video data.

14. The performance device according to claim 11, wherein the determination result in the first performance mode determination model is obtained by transmitting command data based on sound data and stream data based on video data separately and then combining them.

15. The second performance mode determination model includes a song discrimination model and a pattern selection model. The aforementioned music discrimination model includes a machine learning model trained to determine at least one of the genre and style of the content from the audio data. The presentation device according to claim 11, wherein the pattern selection model is a rule-based model.

16. An entertainment device according to any one of claims 10 to 15 that emits light or vibrates inside a vehicle.

17. The lighting device according to claim 16, wherein the light source is located in at least one of the monitor, sun visor, center console, floor, door trim, or dashboard inside the vehicle.

18. The performance device according to claim 16, wherein the vibrating part is located in at least one of the seats, armrests, headrests, floors, steering wheels, or seat belts inside the vehicle.

19. Acquiring content data including sound data, The aforementioned sound data is input into the performance mode determination model, Obtaining the decision result from the aforementioned performance mode determination model, Based on the above determination result, the light source is made to emit light, or the vibrating part is made to vibrate. Make the computer execute it. program.

Images