Content playback program, content playback device, content playback method and content playback system

Abstract

Provided are a content playback program, a content playback device, a content playback method, and a content playback system that can play back sounds closer to the sounds intended by a game maker by selecting a sound file and the settings of an equalizer, a mixer, etc. according to a sound output destination. The content playback processing unit checks the audio device and performs settings suitable for the audio device when it is started and when it is restored from a background state to a foreground state. In addition, when the audio device is changed in the foreground state, the content playback processing unit also performs settings suitable for the changed audio device. This makes it possible to play back sounds closer to the sounds intended by the game maker according to the audio device.

Description

Technical Field

[0001] The present invention relates to a content playback program, a content playback device, a content playback method and a content playback system for performing audio presentation. Background Art

[0002] The applicant develops and distributes game programs and game services. In recent years, the demand for game programs or game services for smart phones has increased compared to the demand for game programs for dedicated game consoles. On the one hand, such a game constitutes a content playback service having a composite content playback function characterized by integrated elements, which include not only action game elements in which game characters (hereinafter referred to as "characters") perform actions according to user operations, but also novel game elements in which the story progresses through dialogues between characters, and functional elements exemplified by animated images, web browsers, etc.

[0003] As games for smartphones become more mature, many users begin to recognize the "sound" of games (which includes background music (BGM) (other similar terms include: soundtrack, soundtrack, etc.) and effect sounds) as an important performance element that attracts users into the game world. In addition, in order to meet this demand, many game manufacturers including the applicant are striving to seek higher quality game sounds.

[0004] Patent Document 1 discloses a program, an information processing apparatus, and an information processing method capable of improving the efficiency of audio playback processing even when the load of calculation processing performed by a CPU increases as a result of repeatedly generating and destroying audio playback objects.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent 6482712 Summary of the invention

[0008] Problem that the invention aims to solve

[0009] Nowadays, smartphones are increasingly used compared to dedicated game consoles, and are becoming very important as a game execution platform replacing dedicated game consoles from the viewpoint of market size. However, the built-in speakers of smartphones (which have computing power comparable to that of personal computers while achieving small size and high density) have a small diameter, so it is undeniable that such built-in speakers have limited playback capabilities compared to large-diameter speakers and headphones.

[0010] On the other hand, since smartphones have a plurality of components for allowing connection of headphones and external audio devices, sound quality-oriented users enjoy games by voluntarily connecting headphones, etc. to smartphones. More specifically, due to differences in the width of the high-pitched range and the low-pitched range of the played-back sound and the reproducibility of the subtle differences of the sound, etc., the game sound played back using the built-in speaker provides a significantly different impression in terms of appeal and perception of real sound compared to the game sound played back using an externally attached acoustic device (external sound device) such as headphones, etc.

[0011] According to the above, it is preferable that, in a smartphone used as a content playback platform, a sound environment suitable for a built-in speaker is prepared for a built-in speaker, and a sound environment suitable for an externally attached acoustic device is prepared for an externally attached acoustic device.

[0012] The present invention is conceived in view of the above-mentioned problems, and an object of the present invention is to provide a content playback program, a content playback device, a content playback method and a content playback system that can play back sounds closer to the game maker's intention by selecting a sound file according to the sound output destination and the settings of an equalizer, a mixer, etc.

[0013] Solutions for solving problems

[0014] In order to solve the above problems, a content playback program according to the present invention is used to enable a computer to implement: a sound rendering control function, which is used to read at least one sound file, decode the at least one sound file, perform signal processing for equalizing and mixing the at least one sound file, and output an audio data stream; and an audio data path control function, which is used to selectively send the audio data stream output by the sound rendering control function to a desired audio device.

[0015] The content playback program also enables the computer to implement: an audio device change detection function for detecting the occurrence of a change in the connection status of the audio device; an audio device confirmation processing function for obtaining audio device information related to the audio device to which the audio data stream is sent based on an inquiry request received from the outside; and an audio device control function for enabling the sound presentation control function to read a definition file corresponding to the audio device information.

[0016] Effects of the Invention

[0017] The present invention can provide a content playback program, a content playback device, a content playback method and a content playback system that can play back sounds closer to the sounds intended by a game maker by selecting sound files according to the sound output destination and setting up an equalizer, a mixer, etc.

[0018] Problems, structures, and effects other than those described above will be clarified through the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram showing a content playback device and a surrounding environment according to a first embodiment of the present invention.

[0020] Figure 2 is a block diagram showing the hardware structure of a content playback device.

[0021] Figure 3 is a block diagram showing the software functions of a content playback device.

[0022] Figure 4 : is a flowchart showing the flow of processing in the content playback processing unit when the game is started and restored to the foreground.

[0023] Figure 5 is a flowchart showing the flow of processing in the content playback processing unit when an audio device change trigger is detected.

[0024] Figure 6 : is a sequence diagram showing the flow of processing in the content playback processing unit when the game is started and restored to the foreground.

[0025] Figure 7 : is a sequence diagram showing the flow of processing in the content playback processing unit when the audio device is switched while the game is running in the foreground.

[0026] Figure 8 : is a sequence diagram showing the flow of processing in the content playback processing unit when the audio device is switched while the game is running in the foreground.

[0027] Fig. 9 A schematic diagram showing an example of equalization setting in the case where the audio device is an external audio device and a built-in speaker in the form of a graphic equalizer is included.

[0028] Fig.10 is a schematic diagram showing a content playback device and a surrounding environment according to a second embodiment of the present invention. DETAILED DESCRIPTION

[0029] In the game, multiple sound sources emit sounds at the same time, such as playing back game effect sounds and character voices while generating background music. This means that the sound processing in the game requires decoding multiple digital audio files into digital audio stream data, and balancing and further mixing the digital audio stream data appropriately. The time and cost required to implement such complex processing in game development increase. On the other hand, since sound processing is a standardized process, many game manufacturers including the applicant entrust sound processing to sound middleware produced by a third party, as disclosed in Patent Document 1.

[0030] Conventional consumer games are made assuming that an excellent content playback platform such as a TV is fixedly provided, and do not assume a content playback platform that allows selective use of built-in speakers and externally attached acoustic devices as specifically seen in terminals such as smartphones and tablet PCs, etc. In addition, there are no games made for smartphones, tablet PCs, etc. that have a function for automatically selecting a sound file and automatically switching an equalizer when a switch between a built-in speaker and an externally attached acoustic device is detected.

[0031] It is assumed in the present invention that the game system monitors the sound playback environment (built-in speakers or externally attached acoustic devices) and selects appropriate sound files and appropriate DSP settings (such as equalizer settings and mixer settings, etc.) according to the final sound output destination, thereby making it possible to play back sounds that are closer to the sounds intended by the game maker.

[0032] According to the present invention, for example, equalizer settings for when headphones are used (e.g., settings that emphasize heavy bass) and equalizer settings for when built-in speakers are used (e.g., settings that emphasize mid- and high-pitched sounds) are prepared separately. When a user uses a built-in speaker, a sound playback equalization that can be easily heard even in a noisy environment can be selected by emphasizing mid- and high-pitched sounds in the entire acoustic range. In addition, when a user uses headphones or earphones, a sound playback equalization that emphasizes heavy bass for BGM can be dynamically switched.

[0033] Similarly, a sound source file for the case of using headphones (for example, a file containing a heavy bass) and a sound source file for the case of using a built-in speaker (for example, a file whose melody consists of only high-pitched sounds) are prepared separately. By doing so, when the user uses the built-in speaker, a sound file that can be easily heard even in a noisy environment by emphasizing high-pitched sounds in the entire sound range, and an equalizer setting and a mixer setting for emphasizing the voice of a character can be selected.

[0034] {First Embodiment: Content Playback Device 101: Overall Structure}

[0035] Figure 1 is a schematic diagram showing a content playback device 101 and a surrounding environment according to a first embodiment of the present invention.

[0036] The smartphone reads the game program and functions as the content playback device 101. The user enjoys the game by operating the touch screen display 101a of the content playback device 101.

[0037] like Figure 1 As shown, the content playback device 101 downloads the game program from the server 102 via the Internet 103 and the wide area wireless communication base station 104 or the wireless LAN router 105, and executes the game program.

[0038] When the game program is executed, the content playback device 101 emits sounds (such as audio and music) in the game content from the built-in speaker 106. In addition, when an external audio device such as a headphone 107, a speaker or an amplifier is connected with a cable (wiring) 108 or wirelessly connected, the sound is also emitted from the external audio device in the same manner.

[0039] {First Embodiment: Content Playback Device 101: Hardware Structure}

[0040] Figure 2 2 is a block diagram showing the hardware structure of the content playback device 101 .

[0041] The content playback device 101 as a well-known computer includes: a CPU 201; a ROM 202; a RAM 203; a display unit 204 as an LCD or the like; an operation unit 205 as an electrostatic touch panel; a wide area wireless communication unit 206; a wireless LAN interface 207; and a nonvolatile memory 208. These units and devices are connected to a bus 209.

[0042] In addition to the above, a DSP 210, a D / A converter 211, a serial interface (hereinafter referred to as "serial I / F") 212 (such as a USB or etc.) and a short-range wireless communication unit 213 (such as (Bluetooth) etc.) are connected to bus 209.

[0043] A plurality of sound files including game BGM, game effect sounds, character voices, etc. are stored in the nonvolatile memory 208. These plurality of sound files are read into the DSP 210 and / or the CPU 201 and decoded. The decoded sound data stream is subjected to digital signal processing such as equalization and mixing, and then sent to the currently active audio device.

[0044] If the currently active audio device is, for example, the built-in speaker 106, a plurality of sound files are sent to the D / A converter 211. If the currently active audio device is the external analog headphone 214, a plurality of sound files are sent to the D / A converter 211, and an analog audio signal is output from the external audio terminal 215. Note that a switch (not shown in the figure) is built into the external audio terminal 215 (stereo mini jack) so that the on / off state of the switch is detected by a program such as an active OS via the bus 209.

[0045] In addition, if the currently active audio device is an external digital headset 216, multiple sound files are sent to the external digital headset 216 as a USB or The external digital headphone 216 has a D / A converter 217 built in, and the D / A converter 217 drives a headphone unit 218.

[0046] If the currently active audio device is a short-range wireless audio device 219 (such as Headphones or Audio interface, etc.), multiple sound files are sent to the short-range wireless communication unit 213. The short-range wireless audio device 219 has a built-in short-range wireless communication unit 220 and a built-in D / A converter 221. The short-range wireless communication unit 220 demodulates digital audio data from the electromagnetic waves received from the short-range wireless communication unit 213. The demodulated digital audio data is converted into an analog signal by the D / A converter 221, and the D / A converter 221 drives the headphone unit 222.

[0047] {First Embodiment: Content Playback Device 101: Software Function}

[0048] Figure 3 is a block diagram showing the software functions of the content playback device 101 .

[0049] The input / output control unit 301 as the OS interprets operation information from the operation unit 205 , and passes the operation information to the content playback processing unit 302 as necessary.

[0050] In addition, the input / output control unit 301 transfers the screen display data output by the content playback processing unit 302 to the display unit 204 .

[0051] The input / output control unit 301 manages the foreground / background execution of multiple applications, and when a program is designated as a foreground program, provides a trigger indicating that the program is designated as a foreground program to the program. Here, it is noted that the program in the foreground state means an application displayed on the touch screen display and can be operated by the user, and the program in the background state means an application other than the foreground application that is executed simultaneously with a different application and cannot be directly operated.

[0052] The content playback processing unit 302 is the main core of the game program.

[0053] The content playback processing unit 302 displays various contents on the display unit 204 or plays back the contents according to the user's operation, etc. For example, the content playback processing unit 302 displays game characters on the display unit 204 and controls the actions of the game characters based on scene data, etc., plays back motion image files, plays back sound files, and displays text data.

[0054] The content playback processing unit 302 includes an initialization processing unit 303 and an audio device control unit 304 .

[0055] The initialization processing unit 303 is activated in response to a trigger from the input / output control unit 301 as the OS when the content playback processing unit 302 is activated and restored to the foreground, and executes predetermined processing.

[0056] Upon receiving a trigger from the initialization processing unit 303 and a trigger from the audio device change detection unit 305 which is one of the functions of the input / output control unit 301 , the audio device control unit 304 confirms the audio device and specifies a definition file suitable for the sound rendering control unit 306 .

[0057] exist Figure 3 In FIG. 1 , the first definition file 310 is a definition file used when the currently active audio device is the built-in speaker 106 . The second definition file 311 is a definition file used when the currently active audio device is a device other than the built-in speaker 106 .

[0058] The input / output control unit 301 includes an audio device change detection unit 305 and an audio device confirmation processing unit 307 as part of its functions.

[0059] The audio device change detection unit 305 detects a change of an audio device by polling the audio data path control unit 308 , and generates a trigger.

[0060] The audio device confirmation processing unit 307 responds to the request for audio device information sent from the audio device control unit 304 of the content playback processing unit 302, accesses the audio data path control unit 308 and obtains the audio device information related to the currently active audio device, and sends the audio device information to the audio device control unit 304 that issued the request.

[0061] The sound presentation control unit 306 as sound middleware reads the definition file specified by the content playback processing unit 302, and reads a required sound file from the sound file group 309 according to a command issued from the content playback processing unit 302. Then, the sound presentation control unit 306 decodes and plays back the read sound file, and equalizes the sound stream data. These processes are performed by using Figure 2 The above-mentioned processing is realized by the computing resources of the CPU 201 and the DSP 210. The sound presentation control unit 306 generates the integrated sound stream data as a result of these computing processes.

[0062] The integrated sound stream data output by the sound presentation control unit 306 is sent to the audio data path control unit 308 which is one of the functions of the OS.

[0063] The audio data path control unit 308 selectively transmits the integrated sound stream data to the currently active audio device. When no external audio device is connected, the integrated sound stream data is transmitted to the built-in speaker 106 through the D / A converter 211. However, it should be noted that in the case where muting is set by means of the function of the OS, the integrated sound stream data is not output to the built-in speaker 106.

[0064] like Figure 3 As shown, a plurality of external audio devices may be connected to the audio data path control unit 308 .

[0065] The external analog headphone 214 is connected to the external audio terminal 215 implemented by a stereo mini jack. The integrated audio stream data output from the audio data path control unit 308 is sent to the D / A converter 211 and converted into an analog audio signal. The analog audio signal output from the D / A converter 211 is output to the external analog headphone 214 through the external audio terminal 215. The external audio terminal 215 has a switch built in, and the on / off state of the switch is sensed by the audio data path control unit 308.

[0066] The external digital headphone 216 is connected to the serial I / F 212. The integrated audio stream data output from the audio data path control unit 308 is sent to the D / A converter 217 built in the external digital headphone 216 through the serial I / F 212, and is converted into an analog audio signal. The analog audio signal output from the D / A converter 217 is output to the analog headphone unit 218.

[0067] Such as Headphones or The short-range wireless audio device 219 of the audio interface or the like establishes bidirectional communication with the short-range wireless communication unit 213 through wireless communication. After that, the integrated sound stream data output from the audio data path control unit 308 is supplied from the short-range wireless communication unit 213 to the D / A converter 221 via the short-range wireless communication unit 220 built in the short-range wireless audio device 219, and is converted into an analog audio signal by the D / A converter 221. The analog audio signal output from the D / A converter 221 is output to the analog headphone unit 222 or an audio amplifier (not shown in the figure), etc.

[0068] Such as or The smartphone OS of , etc. is a multitasking OS, and can execute a plurality of applications at the same time. On the other hand, the application programs that can be directly operated by the user are displayed on the touch screen display, while other programs (with some exceptions) are not displayed and therefore cannot be directly operated.

[0069] As already explained, an application displayed on the entire surface of the touch screen display and operable is referred to as an application in a foreground state, and other applications executed simultaneously and not directly operable are referred to as applications in a background state.

[0070] The content playback processing unit 302 for implementing the content playback apparatus 101 according to an embodiment of the present invention is the main core of the game program, and outputs sound in the foreground state where various operations can be performed. However, in the background state where various operations cannot be performed, the content playback processing unit 302 basically does not output sound, except for some exceptions such as a game program equipped with a BGM appreciation function.

[0071] In addition, the content playback processing unit 302 switches from a state of not outputting sound to a state of outputting sound at startup and when returning to the foreground state from the background state. Therefore, the content playback processing unit 302 needs to confirm the audio device that performs sound output in this situation. It is assumed that the content playback processing unit 302 checks the audio device at startup and when returning to the foreground state, and performs settings of the sound presentation control unit 306 by using a sound file suitable for the audio device based on the definition file. In addition, when the audio device is changed in the foreground state, the content playback processing unit 302 also performs settings of the sound presentation control unit 306 by using a sound file suitable for the changed audio device based on the definition file.

[0072] {First Embodiment: Content Playback Device 101: Flow of the Whole Process}

[0073] Figure 4 is a flowchart showing the flow of processing in the content playback processing unit 302 when the game is started and restored to the foreground.

[0074] When the process starts ( S401 ), the content playback processing unit 302 starts the initialization processing unit 303 and performs a predetermined initialization process ( S402 ).

[0075] After that, the content playback processing unit 302 starts up the audio device control unit 304 through the initialization processing unit 303 .

[0076] In order to confirm the currently active audio device, the audio device control unit 304 inquires the audio device confirmation processing unit 307 of the input / output control unit 301 about audio device information related to the currently active audio device (S403).

[0077] If, as a result of the inquiry, the currently active audio device is the built-in speaker 106 (Yes in S404 ), the content playback processing unit 302 instructs the sound presentation control unit 306 to read the first definition file ( S405 ).

[0078] If, as a result of the inquiry, the currently active audio device is a device other than the built-in speaker 106 (No in S404), the content playback processing unit 302 instructs the sound presentation control unit 306 to read the second definition file (S406).

[0079] This is the end of a series of processes performed when the game is started and restored to the foreground (S407).

[0080] Figure 5 is a flowchart showing the flow of processing in the content playback processing unit 302 when an audio device change trigger is detected.

[0081] When the audio device control unit 304 detects a trigger issued by the audio device change detection unit 305 ( S501 ), the content playback processing unit 302 activates the audio device control unit 304 .

[0082] In order to confirm the currently active audio device, the audio device control unit 304 inquires the audio device confirmation processing unit 307 of the input / output control unit 301 about audio device information related to the currently active audio device (S502).

[0083] If, as a result of the inquiry, the currently active audio device is the built-in speaker 106 (Yes in S503 ), the content playback processing unit 302 instructs the sound presentation control unit 306 to read the first definition file ( S504 ).

[0084] If, as a result of the inquiry, the currently active audio device is a device other than the built-in speaker 106 (No in S503), the content playback processing unit 302 instructs the sound presentation control unit 306 to read the second definition file (S505).

[0085] This is the end of a series of processes performed when an audio device change trigger is detected (S506). Figure 5 The processing of steps S502 to S505 is the same as that performed when the game is started and restored to the foreground. Figure 4 In other words, in addition to the initialization process shown in step S402, the process ( Figure 4 ) and the processing performed when an audio device change trigger is detected ( Figure 5 )same.

[0086] Figure 6 is a sequence diagram showing the flow of processing in the content playback processing unit 302 when the game is started and restored to the foreground. Figure 4 In addition, Figure 6 This includes, for example, and The processing performed.

[0087] When the user operates the operation unit 205 ( S601 ), the operation information generated by the operation unit 205 causes a game start process or a restoration to foreground process through the input / output control unit 301 as the OS ( S602 ).

[0088] A trigger generated by the game start processing or the return to foreground processing in the input / output control unit 301 is sent to the content playback processing unit 302. The trigger is sent to the initialization processing unit 303 in the content playback processing unit 302. Furthermore, the trigger is transmitted from the initialization processing unit 303 to the audio device control unit 304.

[0089] Upon receiving a trigger from the input / output control unit 301 through the initialization processing unit 303 , the audio device control unit 304 instructs the audio device confirmation processing unit 307 of the input / output control unit 301 to perform an audio device confirmation process ( S603 ).

[0090] The audio device confirmation processing unit 307 inquires of the audio data path control unit 308 .

[0091] The audio device confirmation processing unit 307 performs audio device confirmation processing based on the audio device information acquired from the audio data path control unit 308, and returns audio device information indicating the currently active audio device to the audio device control unit 304 (S604).

[0092] The audio device control unit 304 determines the definition file to be used with the aid of the audio device information as the return value reported from the audio device confirmation processing unit 307. Then, the audio device control unit 304 instructs the sound presentation control unit 306 to read the designated definition file (S605).

[0093] The sound presentation control unit 306 reads the definition file specified by the audio device control unit 304 and performs sound file name setting, equalizer setting, mixer setting, etc. (S606). Then, the sound presentation control unit 306 starts sound processing according to a request from the audio device control unit 304 (S607).

[0094] Figure 7 3 is a timing chart showing the flow of processing in the content playback processing unit 302 when the audio device is switched while the game is running in the foreground. However, it should be noted that Figure 7 Including smartphone OS such as situation.

[0095] The content playback processing unit 302, which is the main core of the game program, continues to perform the content playback processing. At this time, the audio device control unit 304 of the content playback processing unit 302 continues to issue a sound processing command to the sound presentation control unit 306 together with the content playback processing (S701). In response to the sound processing command, the sound presentation control unit 306 continues to perform the sound processing (S702).

[0096] In this case, the audio device change detection unit 305 of the input / output control unit 301 detects a change such as attachment and detachment of an acoustic device in a reactive manner with respect to the audio data path control unit 308 ( S703 ).

[0097] The stereo mini-plug of the analog headphones is inserted into the stereo mini-jack.

[0098] An external digital headset 216 is connected to the serial I / F 212 .

[0099] The short-range wireless audio device 219 completes the pairing process.

[0100] These external audio devices have been disassembled.

[0101] In the case where the audio data path control unit 308 performs an audio device change process as a result of performing one of the operations listed above (S704), the audio device change detection unit 305 detects the change process and transmits an audio device change trigger to the audio device control unit 304 of the content playback processing unit 302 (S705). Thereafter, the audio device change detection unit 305 detects changes such as attachment and detachment of an acoustic device again in a reactance manner (S706).

[0102] Upon receiving a trigger from the audio device change detection unit 305 , the audio device control unit 304 of the content playback processing unit 302 instructs the audio device confirmation processing unit 307 of the input / output control unit 301 to perform an audio device confirmation process ( S707 ).

[0103] The audio device confirmation processing unit 307 inquires the audio data path control unit 308 (S708). The audio data path control unit 308 responds to the audio device confirmation processing unit 307 with the audio device name (S709).

[0104] Upon acquiring the audio device name from the audio data path control unit 308 , the audio device confirmation processing unit 307 returns the audio device name to the audio device control unit 304 of the content playback processing unit 302 ( S710 ).

[0105] The audio device control unit 304 determines the definition file to be used with the aid of the audio device information as the return value reported from the audio device confirmation processing unit 307, and instructs the sound presentation control unit 306 to read the specified definition file (S711).

[0106] When reading the definition file specified by the audio device control unit 304, the sound rendering control unit 306 performs resetting of the sound file name, equalizer, mixer, etc. (S712). When the resetting is completed, the sound rendering control unit 306 performs sound processing based on the new setting (S713).

[0107] Figure 8 3 is a timing chart showing the flow of processing in the content playback processing unit 302 when the audio device is switched while the game is running in the foreground. However, it should be noted that Figure 8 Including smartphone OS such as situation.

[0108] The content playback processing unit 302, which is the main core of the game program, continues to perform content playback processing. At this time, the audio device control unit 304 continues to issue a sound processing command to the sound presentation control unit 306 together with the content playback processing (S801). In response to the sound processing command, the sound presentation control unit 306 continues to perform sound processing (S802).

[0109] In this case, the audio device change detection unit 305 of the input / output control device 301 detects a change such as attachment and detachment of an acoustic device in a reactance manner with respect to the audio data path control unit 308 (S803).

[0110] The stereo mini-plug of the analog headphones is inserted into the stereo mini-jack.

[0111] An external digital headset 216 is connected to the serial I / F 212 .

[0112] The short-range wireless audio device 219 completes the pairing process.

[0113] These external audio devices have been disassembled.

[0114] In the case where the audio data path control unit 308 performs an audio device change process as a result of performing one of the operations listed above (S804), the audio device change detection unit 305 detects the change process and transmits an audio device change trigger and audio device information to the audio device control unit 304 (S805). Thereafter, the audio device change detection unit 305 detects changes such as attachment and detachment of an acoustic device again in a reactance manner (S806).

[0115] When receiving the audio device change trigger and audio device information from the audio device change detection unit 305, the audio device control unit 304 of the content playback processing unit 302 determines the definition file to be used with the help of the audio device information which is the return value reported from the audio device change detection unit 305 (S807), and instructs the sound presentation control unit 306 to read the specified definition file (S808).

[0116] Upon reading the definition file specified by the audio device control unit 304, the sound rendering control unit 306 performs resetting of the sound file name, equalizer, mixer, etc. (S809). Upon completion of the resetting, the sound rendering control unit 306 performs sound processing based on the new setting (S810).

[0117] {First Embodiment: Content Playback Device 101: Example of Settings}

[0118] Fig. 9 A is a schematic diagram showing an example of equalization setting in the form of a graphic equalizer in a case where the audio device is an external audio device.

[0119] For example, when using headphones, Fig. 9 As shown in FIG. 1A , the equalizer is set with a gain for emphasizing mid-bass sounds at about 125 Hz and emphasizing sounds over 8 kHz. These equalizer settings enable sounds from high-pitched sounds to low-pitched sounds to be produced favorably.

[0120] Fig. 9 B is a diagram showing an example of equalization setting in the form of a graphic equalizer in a case where the audio device is the built-in speaker 106 .

[0121] On the other hand, when the built-in speaker 106 is used, Fig. 9 B, the equalizer is set with a gain for emphasizing the mid-range sound at about 500 Hz and 1 kHz to emphasize the voice of the character. These settings make it possible to faithfully reproduce the charm of the voice of the character even with the built-in speaker 106.

[0122] In addition, although not shown, it is preferable to use different mixer settings for the case where the audio device is an external audio device and the case where the audio device is the built-in speaker 106.

[0123] In many cases, the performance of the built-in speaker 106 in a smartphone as a general-purpose device is lower than that of a dedicated device dedicated to audio playback, such as a headset, etc. For this reason, due to differences in the width of the high-pitched range and the low-pitched range of the played-back sound and the reproducibility of the nuances of the sound, etc., the game sound played back using the built-in speaker 106 of the smartphone provides a significantly different impression in terms of appeal and perception of real sound compared to the game sound played back using an externally attached acoustic device such as a headset.

[0124] In a battle game, there is a situation where, when multiple characters start to attack an enemy character with a skill, the multiple characters shout out the name of the skill in a loud voice. The main character is made to speak his / her voice loudly, while the other characters are made to speak their voices less loudly. At this time, the game can provide a sense of real sound by changing the volume of the other characters' voices according to the distance of the characters in the battlefield on the game.

[0125] However, with the built-in speaker 106, not only is it difficult to reproduce this perception of real sound, but there is a high possibility that the character's voice will be buried in other sounds such as BGM if the volume of the character's voice is too low.

[0126] For this reason, when the audio device is an external audio device, the volume of the voice of the secondary character is changed according to the distance of the secondary character on the battlefield. On the other hand, when the audio device is the built-in speaker 106, the volume of the voice of the secondary character is not determined according to the distance of the secondary character on the battlefield, but is always fixed to, for example, about 70% of the volume of the voice of the main character.

[0127] {Second Embodiment: Content Playback Device 101: Overall Structure}

[0128] For example, when the standard of wide area wireless communication evolves from the current 4G system to the fifth generation mobile communication system called 5G, it is expected that not only the amount of communication data per unit time will increase but also the communication delay (latency) will decrease. When such a communication environment is realized, the game application in the smartphone is likely to become compact, most of the processing is likely to be performed in the server 102, and the calculation results such as screen data are likely to be transmitted from the server 102 to the smartphone in real time.

[0129] In such a game environment, it will be possible to move part of the functions of the content playback device 101 according to the first embodiment of the present invention (such as the function of the audio device control unit 304 ) to the server 102 .

[0130] Fig.10 1 is a schematic diagram showing a content playback device 1001 and a surrounding environment according to a second embodiment of the present invention.

[0131] The smartphone functions as the content playback device 1001 by reading the game program.

[0132] The game program is downloaded from the content playback control server 1002 via the Internet 103 and the wide area wireless communication base station 104 or the wireless LAN router 105 and then executed.

[0133] The smartphone emits sounds such as audio and music in game contents from the built-in speaker 106. In addition, sounds are emitted from external headphones in the same manner.

[0134] like Fig.10 As shown, it is assumed that the content playback control server 1002 includes the audio device control unit 304 .

[0135] Upon receiving a trigger indicating that the audio device change detection unit 305 has detected the occurrence of a change in the connection state of the audio device from the content reproduction apparatus 1001, the audio device control unit 304 transmits an inquiry request to the audio device confirmation processing unit 307 of the content reproduction apparatus 1001. Then, the audio device control unit 304 transmits a control instruction for causing the sound presentation control unit 306 to read a definition file corresponding to the audio device information received from the audio device confirmation processing unit 307 to the content playback apparatus 1001.

[0136] Since the audio device control unit 304 exists in the content playback control server 1002, the content playback device 1001 cannot fully realize the content playback function alone. By communicating with the content playback control server 1002, the content playback device 1001 functions as a content playback system 1003 having a content playback function.

[0137] With the above-described configuration, the content playback system 1003 can produce effects similar to those produced by the content playback apparatus 101 according to the first embodiment.

[0138] The content playback device 101 described above in the first embodiment and the content playback system 1003 described above in the second embodiment each include: a sound presentation control unit 306; an audio data path control unit 308; an audio device change detection unit 305; an audio device confirmation processing unit 307; and an audio device control unit 304. The sound presentation control function implemented by the sound presentation control unit 306, the audio data path control function implemented by the audio data path control unit 308, the audio device change detection function implemented by the audio device change detection unit 305, the audio device confirmation processing function implemented by the audio device confirmation processing unit 307, and the audio device control function implemented by the audio device control unit 304 are implemented by a content reproduction program constituting the content reproduction device 101 in a computer such as a smartphone.

[0139] In addition, the content playback processing unit 302 implements a content playback processing function for playing back desired content, and the input / output control unit 301 implements an input / output control function for transmitting screen information generated by the content playback processing function to the display unit, and for executing a start process, a stop process, a foreground execution process, and a background execution process of the content playback processing function when receiving operation information. In addition, the initialization processing unit 303 implements an initialization processing function for transmitting a trigger to the audio device control function by detecting that the input / output control function has executed a start process and / or a foreground execution process of the content playback processing function.

[0140] Although the content playback device 101 and the content playback system 1003 according to the embodiment of the present invention select sound files and switch equalizer settings and mixer settings according to the audio device, the settings to be changed are not limited to the above settings. For example, sound effects such as echo, delay and chorus can also be changed according to the audio device.

[0141] In addition, although Figure 3 A first definition file 310 for the built-in speaker 106 and a second definition file 311 for a speaker other than the built-in speaker 106 are shown, but a greater number of definition files may be provided. For example, the embodiment may be configured to prepare definition files corresponding to the frequency characteristics of commercially available headphones, and when the device name of the headphones is acquired by any means, the best definition file is selected according to the device name.

[0142] In addition, although the content playback device 101 and the content playback system 1003 according to the embodiment of the present invention have been described by assuming that a smartphone is used, the device that can implement the present invention is not limited to the smartphone. The present invention can be implemented in a portable game console, a personal computer, etc., as long as they are digital devices to which the audio playback device can be connected.

[0143] In the embodiment of the present invention, the content playback apparatus 101 and the content playback system 1003 , and the content playback program for implementing the content playback apparatus 101 and the content playback system 1003 are described.

[0144] The content playback processing unit 302 checks the audio device when it is started and when it is restored from the background state to the foreground state, thereby making settings suitable for the audio device. In addition, when the audio device is changed in the foreground state, the content playback processing unit 302 also makes settings suitable for the changed audio device. By doing so, it is possible to play back sounds closer to the sounds intended by the game maker according to the audio device.

[0145] Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. The present invention includes other modifications and applications without departing from the spirit of the present invention.

[0146] Description of Reference Numerals

[0147] 101 Content playback device

[0148] 101a Touch Screen Monitor

[0149] 102 Server

[0150] 103 Internet

[0151] 104 Base Station

[0152] 105 Wireless LAN Router

[0153] 106 built-in speakers

[0154] 107 Headphones

[0155] 108 Cables

[0156] 201 CPU

[0157] 202 ROM

[0158] 203 RAM

[0159] 204 Display unit

[0160] 205 Operating unit

[0161] 206 Wide Area Wireless Communication Unit

[0162] 207 Wireless LAN interface

[0163] 208 Non-volatile memory

[0164] 209 Bus

[0165] 210 DSP

[0166] 211 D / A converter

[0167] 212 Serial interface

[0168] 213 Short-range wireless communication unit

[0169] 214 External analog headphones

[0170] 215 External audio terminal

[0171] 216 External digital headphones

[0172] 217 D / A converter

[0173] 218 Headphone Unit

[0174] 219 Short-range wireless audio device

[0175] 220 Short-range wireless communication unit

[0176] 221 D / A converter

[0177] 222 Headphone Unit

[0178] 301 Input / output control unit

[0179] 302 Content playback processing unit

[0180] 303 Initialize processing unit

[0181] 304 Audio device control unit

[0182] 305 Audio device change detection unit

[0183] 306 Sound Presentation Control Unit

[0184] 307 Audio device confirmation processing unit

[0185] 308 Audio Data Path Control Unit

[0186] 309 Sound File Group

[0187] 310 First definition file

[0188] 311 Second definition file

[0189] 1001 Content playback device

[0190] 1002 Content playback control server

[0191] 1003 Content playback system

Claims

1. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, a content playback method is implemented, wherein the content playback method comprises: A sound rendering step for reading at least one sound file, decoding the at least one sound file, performing signal processing for equalizing and mixing the at least one sound file, and outputting an audio data stream; an audio data path control step for selectively sending the audio data stream output in the sound rendering step to a desired audio device; an audio device change detection step for detecting the occurrence of a change in the connection state of the audio device; an audio device confirmation processing step for acquiring audio device information related to the audio device to which the audio data stream is transmitted according to an inquiry request received from the outside; as well as An audio device control step, for reading a first definition file adjusted for the built-in speaker when the currently active audio device in the audio device information is the built-in speaker, and for reading a second definition file adjusted for the external audio device when the currently active audio device is a device other than the built-in speaker, and selecting a sound file suitable for the currently active audio device from a sound file group based on the first definition file or the second definition file at startup, when restoring to a foreground state, and when changing the audio device in the foreground state, so that the sound file is read in the sound presentation step.

2. The computer program product according to claim 1, in, In the audio device change detection step, the audio data path control step is monitored and occurrence of a change in the connection state of the audio device to which the audio data stream is transmitted is detected, and In the audio device control step, a trigger indicating that a change in the connection status of the audio device has been detected in the audio device change detection step is received, the inquiry request is transmitted to the audio device confirmation processing step, and the first definition file or the second definition file corresponding to the audio device information received in the audio device confirmation processing step is read in the sound presentation step.

3. The computer program product according to claim 2, configured to cause the computer to further implement: A content playback processing step for playing back the desired content; an input and output control step for transmitting the screen information generated in the content playback processing step to a display unit, and executing a start process, a stop process, a foreground execution process, and a background execution process of the content playback processing step upon receiving operation information; and An initialization processing step is used to detect that a start process and / or a foreground execution process of the content playback processing step is executed through the input and output control step, and transmit the trigger to the audio device control step.

4. A content playback device, comprising: a sound presentation control unit, configured to read at least one sound file, decode the at least one sound file, perform signal processing for equalizing and mixing the at least one sound file, and output an audio data stream; an audio data path control unit, configured to selectively send the audio data stream output by the sound presentation control unit to a desired audio device; an audio device change detection unit for monitoring the audio data path control unit and detecting an occurrence of a change in a connection state of the audio device connected to the audio data path control unit; an audio device confirmation processing unit for acquiring audio device information related to the active audio device connected to the audio data path control unit according to an inquiry request received from the outside; as well as an audio device control unit configured to receive a trigger indicating that the audio device change detection unit has detected an occurrence of a change in the connection state of the audio device, transmit an inquiry request to the audio device confirmation processing unit, and cause the sound presentation control unit to read a definition file corresponding to the audio device information received from the audio device confirmation processing unit, Wherein, when the currently active audio device is a built-in speaker, the sound presentation control unit reads a first definition file adjusted for the built-in speaker; when the currently active audio device is a device other than the built-in speaker, the sound presentation control unit reads a second definition file adjusted for the external audio device, and when starting, when restoring to the foreground state, and when changing the audio device in the foreground state, the sound file suitable for the currently active audio device is selected from the sound file group based on the first definition file or the second definition file, and the audio data stream is output.

5. A content playback method, comprising: A sound rendering step for reading at least one sound file, decoding the at least one sound file, performing signal processing for equalizing and mixing the at least one sound file, and outputting an audio data stream; an audio data path control step for selectively sending the audio data stream output in the sound rendering step to a desired audio device; an audio device change detection step for detecting the occurrence of a change in the connection state of the audio device; an audio device confirmation processing step for acquiring audio device information related to the audio device to which the audio data stream is transmitted according to an inquiry request received from the outside; as well as A step for receiving a trigger indicating the occurrence of a change in the connection status of the audio device and the audio device information, and executing the sound rendering step by using a definition file corresponding to the received audio device information, wherein, in the sound rendering step, when the currently active audio device is a built-in speaker, a first definition file adjusted for the built-in speaker is read, and when the currently active audio device is a device other than the built-in speaker, a second definition file adjusted for an external audio device is read, and at startup, when restoring to a foreground state, and when changing the audio device in the foreground state, a sound file suitable for the currently active audio device is selected from a sound file group based on the first definition file or the second definition file, and the audio data stream is output.

6. A content playback system, comprising: a content playback device; as well as a content playback control server, configured to control the content playback device, Wherein, the content playback device comprises: a sound presentation control unit, configured to read at least one sound file, decode the at least one sound file, perform signal processing for equalizing and mixing the at least one sound file, and output an audio data stream, an audio data path control unit, configured to selectively send the audio data stream output by the sound presentation control unit to a desired audio device, an audio device change detection unit for monitoring the audio data path control unit and detecting the occurrence of a change in the connection state of the audio device connected to the audio data path control unit, and an audio device confirmation processing unit for acquiring audio device information related to the active audio device connected to the audio data path control unit according to an inquiry request received from the outside, and The content playback control server includes an audio device control unit configured to receive a trigger from the content playback device indicating that the audio device change detection unit has detected an occurrence of a change in the connection state of the audio device, transmit an inquiry request to the audio device confirmation processing unit of the content playback device, and cause the sound presentation control unit to read a definition file corresponding to the audio device information received from the audio device confirmation processing unit. Among them, the sound presentation control unit of the content playback device reads a first definition file adjusted for the built-in speaker when the currently active audio device is the built-in speaker, and reads a second definition file adjusted for the external audio device when the currently active audio device is a device other than the built-in speaker, and selects a sound file suitable for the currently active audio device from the sound file group based on the first definition file or the second definition file when starting, when restoring to the foreground state, and when changing the audio device in the foreground state, and outputs the audio data stream.

Images