Audio transmission method and device, computer, storage medium and program product

Abstract

The embodiment of the invention discloses an audio transmission method and device, a computer, a storage medium and a program product, and the method comprises the steps: generating a loop start instruction when audio data in a cloud game application is transmitted to business equipment, and sequentially transmitting the loop start instruction and a loaded first audio clip to the business equipment; when the duration of the sent first audio clip is the one-way audio length, sending a data ending instruction to the service equipment; the data ending instruction is used for circularly playing the first audio clip when the service equipment does not receive a loop ending instruction; and when detecting that the loop is ended, sending a loop ending instruction to the service equipment, so that the service equipment ends playing the first audio clip based on the loop ending instruction. According to the invention, the traffic generated by audio transmission can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an audio transmission method, device, computer, storage medium, and program product. Background Art

[0002] In cloud gaming scenarios, cloud servers need to continuously send small segments of audio data from the game to service devices in chronological order for playback. This ensures that users can hear the game audio in its original form. To reduce network transmission pressure, the audio data is typically encoded and sent to the service devices, which then decode the encoded data to play the audio data. However, this network transmission introduces latency to playback and increases network traffic. To address this, service devices typically pre-download audio data. When playback is needed, they directly retrieve the required audio segments from the pre-downloaded audio data. However, this approach requires that all possible audio resources must be pre-downloaded to the service devices before the game starts. This slows down game launches and consumes significant resources to update and maintain audio data. Furthermore, the accumulation of audio data on service terminals consumes significant storage space on service devices, compromising the lightweight nature of cloud gaming. Summary of the Invention

[0003] The embodiments of the present application provide an audio transmission method, apparatus, computer, storage medium, and program product, which can reduce the traffic generated by audio transmission while reducing the storage usage of business equipment.

[0004] On the one hand, an embodiment of the present application provides an audio transmission method, which is performed by a cloud service device and includes:

[0005] When transmitting audio data in the cloud gaming application to a service device, generating a loop start instruction, and sequentially sending the loop start instruction and the loaded first audio segment to the service device; the audio data includes the first audio segment;

[0006] When the duration of the first audio clip that has been sent is the one-way audio length, sending a data end instruction to the service device; the data end instruction is used to enable the service device to play the first audio clip in a loop if no loop end instruction is received;

[0007] When the end of the loop is detected, a loop end instruction is sent to the service device, so that the service device ends playing the first audio segment based on the loop end instruction.

[0008] On the one hand, an embodiment of the present application provides an audio transmission method, which is performed by a service device and includes:

[0009] Receiving a loop start instruction sent by the cloud service device, and storing a first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction;

[0010] When the data end instruction is received, the first audio segment is played in a loop until a loop end instruction sent by the cloud service device is received, and the playing of the first audio segment is stopped.

[0011] In one aspect, an embodiment of the present application provides an audio transmission device, which is applied to a cloud service device and includes:

[0012] A data sending module, configured to generate a loop start instruction when transmitting audio data in a cloud gaming application to a service device, and sequentially send the loop start instruction and the loaded first audio segment to the service device; the audio data includes the first audio segment;

[0013] An instruction sending module is used to send a data end instruction to the service device when the length of the first audio clip that has been sent is the one-way audio length; the data end instruction is used to make the service device play the first audio clip in a loop if no loop end instruction is received;

[0014] The end sending module is used to send a loop end instruction to the service device when detecting that the loop ends, so that the service device ends playing the first audio segment based on the loop end instruction.

[0015] The device further comprises:

[0016] A protocol building module is used to send an association request to the game management device associated with the cloud game application, and establish a game transmission protocol with the game management device based on the association request;

[0017] The data sending module is specifically used to:

[0018] Receive a loop start message sent by the game management device through the game transmission protocol, start the audio transmission process, and generate a loop start instruction based on the loop start message during the audio transmission process; the audio transmission process is used to transmit the audio data in the cloud game application to the business device;

[0019] The first audio segment is loaded based on the loop start message, and the loop start instruction and the first audio segment are sequentially sent to the service device.

[0020] When the first audio clip is loaded based on the loop start message, the data sending module is specifically configured to:

[0021] Determine the audio identifier to be transmitted and the transmission order based on the loop start message, and sequentially load the first audio segment indicated by the audio identifier to be transmitted from the audio data stored in the cloud service device according to the transmission order; the audio data refers to audio associated with the cloud gaming application; or

[0022] The first audio clip is loaded in real time from the game management device based on the loop start message.

[0023] The device further comprises:

[0024] A hook adding module is used to obtain the audio related functions in the cloud game application and add a hook detection program for the audio related functions;

[0025] The data sending module is specifically used to:

[0026] When the hook detection program detects that the audio loading function in the audio associated function is called, a loop start instruction is generated;

[0027] Obtaining the function parameter transmitted by the audio loading function through the hook detection program, and obtaining the first audio segment loaded within the one-way audio length indicated by the function parameter;

[0028] A loop start instruction and a first audio segment are sequentially sent to the service device.

[0029] The device further comprises:

[0030] an end determination module, configured to determine that the loop ends when a hook detection program detects that an audio stop function in an audio associated function is called; or,

[0031] The end determination module is further used to obtain the number of loops and the duration of the first audio segment in the function parameters, determine the loop duration based on the number of loops and the duration of the first audio segment, and determine that the loop ends when the loop duration is reached.

[0032] The device further comprises:

[0033] An audio deduplication module is configured to obtain audio data from a cloud gaming application, perform duplicate segment detection on audio segments included in the audio data, obtain a first duplicate segment, and obtain a second duplicate segment in the first duplicate segment whose segment length is greater than or equal to a duplicate splitting threshold;

[0034] The audio deduplication module is further configured to split the audio segment into N audio sub-segments based on the second repeated segment, perform segment encoding on the N audio sub-segments, and obtain a segment code sequence; identical audio sub-segments in the N audio sub-segments have the same corresponding segment codes in the segment code sequence; N is a positive integer;

[0035] The audio deduplication module is further used to perform deduplication processing on the N audio sub-segments to obtain simplified segments corresponding to the audio segments;

[0036] The data sending module is specifically used to:

[0037] When transmitting audio data in the cloud gaming application to the service device, a loop start instruction is generated, a first audio segment to be transmitted is determined, a segment encoding sequence to be transmitted corresponding to the first audio segment to be transmitted is obtained, and a simplified segment to be transmitted corresponding to the audio segment to be transmitted is determined as the first audio segment;

[0038] A loop start instruction, a coding sequence of segments to be transmitted, and a first audio segment are sequentially sent to the service device.

[0039] The device further comprises:

[0040] The audio splitting module is used to obtain the audio data of the cloud gaming application and detect the number of audio streams corresponding to the audio segments included in the audio data;

[0041] The audio splitting module is further configured to split the audio segment into M initial audio streams if the number of audio streams is not one, and obtain audio repetition breakpoints corresponding to the M initial audio streams respectively; M is a positive integer;

[0042] The audio splitting module is further configured to convert the M initial audio streams into M deduplicated audio streams based on the audio duplication breakpoints corresponding to the M initial audio streams;

[0043] The data sending module is specifically used to:

[0044] When transmitting audio data in the cloud gaming application to the service device, a loop start instruction is generated, a second audio segment to be transmitted is determined, and M deduplicated audio streams corresponding to the second audio segment to be transmitted are determined as the first audio segment;

[0045] A loop start instruction and a first audio segment are sequentially sent to the service device.

[0046] The device further comprises:

[0047] A transmission trigger module is used to respond to a loading request for a game scene from a business device and send the loading request for the game scene to a game management device, so that the game management device starts transmitting the audio data of the cloud game application in the game scene to the business device; or

[0048] The transmission trigger module is also used to respond to the trigger operation of the business device on the audio playback component, and send the trigger operation on the audio playback component to the game management device, so that the game management device starts to transmit the audio data corresponding to the audio playback component in the cloud game application to the business device.

[0049] On the one hand, an embodiment of the present application provides an audio transmission device, which is applied to a service device and includes:

[0050] a data receiving module, configured to receive a loop start instruction sent by the cloud service device and store the first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction; the loop start instruction is generated by the cloud service device when transmitting audio data in the cloud gaming application to the service device;

[0051] The audio playback module is used to play the first audio clip in a loop when a data end instruction is received, until a loop end instruction sent by the cloud service device is received, and the first audio clip is stopped from playing; the data end instruction is generated by the cloud service device when the length of the first audio clip that has been sent is the one-way audio length.

[0052] The first audio segment includes a simplified segment to be transmitted; and the data receiving module is specifically configured to:

[0053] Receive a loop start instruction sent by the cloud service device, and obtain a code sequence of the segment to be transmitted sent by the cloud service device;

[0054] storing the simplified segments to be transmitted sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction;

[0055] The simplified segment to be transmitted is decoded into a first audio segment to be transmitted based on the encoding sequence of the segment to be transmitted, and the first audio segment to be transmitted is played.

[0056] The first audio clip includes M deduplicated audio streams; M is a positive integer;

[0057] The data receiving module is specifically used for:

[0058] Receiving a loop start instruction sent by the cloud service device, and storing M deduplicated audio streams sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction;

[0059] M deduplicated audio streams are mixed and played.

[0060] On the one hand, an embodiment of the present application provides a computer device, including a processor, a memory, and an input and output interface;

[0061] The processor is connected to the memory and the input and output interface respectively, wherein the input and output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device including the processor executes the audio transmission method in one aspect of an embodiment of the present application.

[0062] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the audio transmission method in one aspect of the embodiment of the present application.

[0063] In one aspect, an embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional embodiments of the present application. In other words, when the computer instructions are executed by the processor, the methods provided in various optional embodiments of the present application are implemented.

[0064] Implementing the embodiments of this application will have the following beneficial effects:

[0065] In an embodiment of the present application, when the cloud service device transmits the audio data in the cloud gaming application to the business device, it generates a loop start instruction, and sends the loop start instruction and the loaded first audio clip to the business device in sequence; when the duration of the first audio clip that has been sent is the one-way audio length, a data end instruction is sent to the business device; the data end instruction is used to enable the business device to play the first audio clip in a loop when the loop end instruction is not received; when the end of the loop is detected, a loop end instruction is sent to the business device, so that the business device ends playing the first audio clip based on the loop end instruction. In this way, different instructions are inserted into the audio transmission, and the loop start instruction to the data end instruction is used to determine the audio segment that needs to be played in a loop, and the loop end instruction is used to determine the time to end the loop playback, so that when the audio segment needs to be played repeatedly, it only needs to send the audio segment to the business device once, and the loop start instruction and the data end instruction enable the business device to determine the audio segment that needs to be played in a loop, and end the loop playback of the audio segment based on the loop end instruction, so that the traffic generated by the audio transmission from the cloud service device to the business device can be reduced, and the traffic overhead can be reduced; at the same time, the business device does not need to preload the audio data, and only needs to store a repeated audio segment each time the audio is transmitted to achieve the playback of the audio data, so that the transmission of audio data takes up less storage of the business device, and there is no need to update and maintain the audio segments stored in the business device, thereby saving resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0067] Figure 1 This is a network interaction architecture diagram for audio transmission provided by an embodiment of the present application;

[0068] Figure 2 This is a schematic diagram of an audio transmission scenario provided by an embodiment of the present application;

[0069] Figure 3 This is a flow chart of an audio transmission method provided by an embodiment of the present application;

[0070] Figure 4 This is a schematic diagram of a loop processing scenario provided by an embodiment of the present application;

[0071] Figure 5 This is a schematic diagram of another loop processing scenario provided by an embodiment of the present application;

[0072] Figure 6 This is a schematic diagram of a simplified audio scenario provided by an embodiment of the present application;

[0073] Figure 7 This is a schematic diagram of a multi-audio stream transmission scenario provided by an embodiment of the present application;

[0074] Figure 8 This is a flow chart of another audio transmission method provided by an embodiment of the present application;

[0075] Figure 9 This is a schematic diagram of an audio transmission device provided in an embodiment of the present application;

[0076] Figure 10 This is a schematic diagram of another audio transmission device provided in an embodiment of the present application;

[0077] Figure 11 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0079] Among them, if it is necessary to collect object (such as user, etc.) data in this application, a prompt interface or pop-up window will be displayed before or during the collection. The prompt interface or pop-up window is used to remind the user that certain data is currently being collected. Only after the user confirms the prompt interface or pop-up window, the relevant steps for data acquisition will be started, otherwise it will end. Moreover, the acquired user data will be used in reasonable and legal scenarios or purposes. Optionally, in some scenarios where user data needs to be used but the user has not authorized it, authorization can be requested from the user, and the user data can be used when the authorization is passed. In other words, the use of user data by this application complies with the relevant provisions of laws and regulations.

[0080] In the examples of this application, see Figure 1 , Figure 1 This is a network interaction architecture diagram of an audio transmission provided by an embodiment of the present application, such as Figure 1 As shown, the game management device 101 can transmit audio data to the cloud service device 102, and the cloud service device 102 can send the audio data to the business device, such as business device 103a, business device 103b and business device 103c. In other words, the present application can be applied to cloud gaming scenarios, especially to scenarios where audio data in cloud gaming applications is transmitted to business devices, wherein cloud gaming refers to a gaming mode in which a server is used to render the game screen, and the user client (i.e., the client integrated in the business device) is only used for display. Cloud gaming (Cloud gaming), also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not run on the user client, but on the cloud service device, and the cloud service device renders the game scene into a video and audio stream, which is transmitted to the user client over the network. The user client does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input instructions and send them to the cloud server. As Figure 1As shown, game management device 101 can send first game data to cloud service device 102. Cloud service device 102 can process the game data to obtain second game data, and then send the second game data to the business devices participating in the cloud gaming application. Game management device 101 refers to the device associated with the cloud gaming application, and is used to manage data related to the cloud gaming application. Cloud service device 102 can be a cloud gaming server, etc., which is used to load the relevant data of the cloud gaming application, process the game data obtained from game management device 101 to obtain game data that can be run by the business devices, and send the processed game data to the business devices.

[0081] For details, see Figure 2 , Figure 2 This is a schematic diagram of an audio transmission scenario provided by an embodiment of the present application. Figure 2 As shown, when the cloud service device transmits the audio data in the cloud game application to the business device, it is assumed that the data to be sent includes audio segment 1, audio segment 1, ... and audio segment 2 in sequence. Each audio segment can include one or more fragment segments. An audio segment can be considered as an audio, and each fragment segment can be considered as a small segment that makes up the audio, and can be considered as a segment that is transmitted once during audio transmission. The cloud service device can generate a loop start instruction and send the loop start instruction and the loaded first audio segment to the business device in sequence. The first audio segment can be considered as the audio segment that needs to be sent in a loop, such as Figure 2 The audio segment 1 shown in , when the length of the first audio segment that has been sent is the one-way audio length, a data end instruction is sent to the business device, so that the business device plays the first audio segment in a loop when the loop end instruction is not received. The one-way audio length refers to the length of the audio segment that needs to be played in a loop in this audio transmission. When the business device receives the loop start instruction, it will start playing the fragmented segments received from the loop start instruction, and at the same time record the fragmented segments received from the loop start instruction until the data end instruction is received. At this time, the fragmented segments received by the business device between the loop start instruction and the data end instruction can be determined as the first audio segment. Further, the business device plays the first audio segment in a loop until it receives the loop end instruction, and ends playing the first audio segment. As Figure 2 As shown in , the service device receives the loop start instruction, audio segment 1 and data end instruction in sequence. At this time, audio segment 1 can be considered as the first audio segment. The service device can play audio segment 1 in a loop until it receives the loop end instruction.

[0082] In simple terms, the cloud service device can insert a loop start instruction (which can be recorded as a loopstart instruction) before the first fragment of the audio segment that is about to start looping (i.e., the first audio segment), and send it to the business device along with the first fragment of the audio segment. When the business device receives the loopstart instruction, it starts playing and stores the received audio segments in the order of reception for subsequent automatic loop playback. The cloud service device can continue to send audio segments until the last fragment, and then insert a data end instruction (which can be recorded as a loopend instruction) and send it to the business device along with the last fragment. After receiving the loopend instruction, the business device has stored all the fragments of the audio segment that is about to be looped, so it can loop the fragments by itself. From the time the data end instruction is sent until the loop playback of the audio segment needs to end, the cloud service device does not need to transmit the audio segment to the business device. If the audio played by the cloud gaming application on the cloud service device changes, it indicates that the loop has ended. At this time, a loop end instruction (which can be recorded as a loopout instruction) can be inserted before the first fragment of the new audio segment (which can be called the second audio segment), and the loopout instruction can be sent to the business device along with the first fragment of the new audio segment. Optionally, if the new audio segment also needs to be played in a loop, the loopout instruction and the loopstart instruction can be sent to the business device along with the first fragment of the new audio segment, which can be recorded as "loopout loopstart the first fragment of the new audio segment" and so on. When the business device receives the loopout instruction, it ends playing the first audio segment. When other audio segments need to be played in a loop subsequently, the processing method is the same as above. By knowing that the cloud gaming application is playing the same audio segment in a loop, the original data is only transmitted during the transmission of the first complete audio segment. No data needs to be transmitted in the subsequent loop playback stage. The business device can loop the audio segment that has been obtained by itself, which can avoid the transmission of duplicate data, reduce the traffic generated by audio transmission, and reduce traffic overhead.

[0083] It is understandable that the business equipment and game management equipment mentioned in the embodiments of the present application can be a computer device, and the computer device in the embodiments of the present application includes but is not limited to a terminal device or a server. In other words, the computer device can be a server or a terminal device, or it can be a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, PDAs, vehicle-mounted equipment, augmented reality / virtual reality (AR / VR) equipment, helmet displays, smart TVs, wearable devices, smart speakers, digital cameras, cameras and other mobile Internet devices (mobile internet device, MID) with network access capabilities, or terminal devices in scenarios such as trains, ships, and flights. As Figure 1 As shown in , the terminal device can be a laptop (as shown by the service device 103b), a mobile phone (as shown by the service device 103c) or a vehicle-mounted device (as shown by the service device 103a), etc. Figure 1 Only some of the devices are listed as examples. Optionally, the business device 103a refers to a device located in the vehicle 104. The business device 103a can be used to run cloud games and play audio data 1031. The servers mentioned above can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0084] Optionally, the data involved in the embodiments of the present application can be stored in a computer device, or the data can be stored based on cloud storage technology or a blockchain network, which is not limited here.

[0085] Further, see Figure 3 , Figure 3 This is a flow chart of an audio transmission method provided by an embodiment of the present application. Figure 3 As shown, the method is executed by a cloud service device. Specifically, the audio transmission process includes the following steps:

[0086] Step S301: When transmitting audio data in a cloud gaming application to a service device, a loop start instruction is generated, and the loop start instruction and the loaded first audio segment are sequentially sent to the service device.

[0087] In an embodiment of the present application, when the cloud service device transmits the audio data in the cloud gaming application to the business device, it generates a loop start instruction, loads the first audio segment to be played in a loop, and sends the loop start instruction and the first audio segment to the business device in sequence, wherein the audio data in the cloud gaming application includes the first audio segment, and the cloud service device refers to a device for providing cloud gaming services. Optionally, the cloud service device can load the first audio segment to be played in a loop while sending the loaded first audio segment to the business device to achieve cloud rendering of the cloud gaming application. The first audio segment can include k fragments, load the first fragment, and send the loop start instruction and the first fragment to the business device, load and send the second fragment to the business device, until the kth fragment is sent to the business device, k is a positive integer. At this time, the duration of the first audio segment that has been sent is the one-way audio length, and step S302 is executed.

[0088] Specifically, in one case, the cloud service device can send an association request to the game management device associated with the cloud gaming application, and establish a game transmission protocol with the game management device based on the association request. In other words, there is an association relationship, i.e., a cooperative relationship, between the cloud gaming application and the cloud gaming service (i.e., the cloud service device). The cloud service device can receive a loop start message sent by the game management device via the game transmission protocol and start the audio transmission process. During the audio transmission process, a loop start instruction is generated based on the loop start message. The audio transmission process is used to transmit the audio data in the cloud gaming application to the service device; a first audio clip is loaded based on the loop start message, and the loop start instruction and the first audio clip are sequentially sent to the service device.

[0089] For details, see Figure 4 , Figure 4 This is a schematic diagram of a loop processing scenario provided by an embodiment of the present application. Figure 4As shown, cloud service device 401 may include a game process and a cloud gaming service. The game process is used to run a cloud gaming application, and the cloud gaming service is used to provide gaming services to users, including but not limited to rendering game data for the cloud gaming application. Cloud service device 401 sends a loop start message, or a start notification, to the cloud gaming service through the game process. The cloud gaming service generates a loop start instruction, which is then continuously sent to the service device based on the obtained audio clips, or a continuous audio stream. When all the audio clips for loop playback have been sent to the service device, a data end instruction is inserted and sent to the service device. The game management device adds audio playback association notifications to the game logic. These audio playback association notifications include an audio playback start notification and an audio playback end notification. The audio playback start notification is used to generate a loop start message, and the audio playback end notification is used to generate a loop end message. Specifically, the game management device can run the cloud gaming application in the cloud service device through the game process. Cloud service device 401 can run the game logic of the cloud gaming application through the game process. When the audio playback start notification is received, a loop start message is sent to the cloud gaming service, and the cloud service device generates a loop start instruction through the cloud gaming service.

[0090] Among them, when loading the first audio clip based on the loop start message, the cloud service device can determine the audio identifier to be transmitted and the transmission order based on the loop start message, and load the first audio clip indicated by the audio identifier to be transmitted from the audio data stored in the cloud service device in sequence according to the transmission order; audio data refers to audio associated with the cloud game application; cloud service device refers to the terminal that generates the loop start instruction. Alternatively, the first audio clip is loaded in real time from the game management device based on the loop start message. In other words, when there is an association relationship between the cloud game application and the cloud game service, the audio playback association notification can be directly integrated into the game logic according to the agreed game transmission protocol, and the audio loop start notification can be actively sent to the cloud service device through the audio playback association notification to realize the marking and transmission of the audio clip that needs to be looped, so that when the audio clip needs to be looped, only a complete audio clip for looping needs to be sent, thereby improving the loop transmission efficiency.

[0091] In one case, there is no association relationship between the cloud gaming application and the cloud gaming service. The cloud service device can obtain the audio association function in the cloud gaming application and add a hook detection program to the audio association function, that is, to detect (hook) the audio association function. The audio association function may include an audio loading function and an audio stop function, etc. The hook detection program is used to detect the operating logic of the added audio association function, and can be used to obtain the data generated when the audio association function is running. Further, when the hook detection program detects that the audio loading function in the audio association function is called, a loop start instruction is generated. The function parameters transmitted by the audio loading function are obtained through the hook detection program, and the first audio segment loaded in the one-way audio length indicated by the function parameters is obtained. The loop start instruction and the first audio segment are sent to the service device in sequence.

[0092] For details, see Figure 5 , Figure 5 This is another schematic diagram of a loop processing scenario provided by an embodiment of the present application. Figure 5 As shown, the cloud service device 501 can run the game logic through the game process, integrate a hook detection program in the game logic, and when the game logic is run through the game process, including loading audio clips, playing audio clips, and stopping audio clips, the hook detection program detects the audio-related functions in the game logic for detection. When it is detected that the audio loading function in the audio-related function is called, a loop start instruction is generated, and the function parameters transmitted by the audio loading function are obtained through the hook detection program, and the first audio clip loaded in the one-way audio length indicated by the function parameters is obtained. That is, when there is no association between the cloud game application and the cloud game service, the hook detection program can be used directly to detect the audio-related functions related to the cloud game application in the processes of playing audio, loading audio, starting playback, and stopping playback, so as to realize a single transmission of the audio that needs to be looped, and thus the service device can realize loop playback of the audio.

[0093] Among them, the cloud service device can obtain audio clips from the game management device in real time, and can also load the audio data of the cloud game application from the game management device. Optionally, the cloud service device can obtain the audio data of the cloud game application, perform duplicate segment detection on the audio clips included in the audio data, obtain a first duplicate segment, and obtain a second duplicate segment in the first duplicate segment whose segment length is greater than or equal to the duplicate splitting threshold. Split the audio clip into N audio sub-segments based on the second duplicate segment, and perform segment encoding on the N audio sub-segments to obtain a segment encoding sequence; the same audio sub-segments in the N audio sub-segments have the same corresponding segment encoding in the segment encoding sequence; N is a positive integer. De-duplication processing is performed on the N audio sub-segments to obtain a simplified segment corresponding to the audio clip. For example, see Figure 6 , Figure 6 This is a simplified audio scenario diagram provided by the embodiment of the present application. Figure 6 As shown, repeated segment detection can be performed on audio segment 601 to obtain first repeated segments, such as first repeated segment 6011 and first repeated segment 6012. Repeated segments refer to segments that appear multiple times in the audio segment. A second repeated segment within the first repeated segment can be obtained, whose length is greater than or equal to a repeated segmentation threshold. If the length of first repeated segment 6011 is greater than the repeated segmentation threshold, and the length of first repeated segment 6012 is less than the repeated segmentation threshold, first repeated segment 6011 is determined to be a second repeated segment. Based on the second repeated segment, audio segment 601 is split into N audio sub-segments 602. The N audio sub-segments include the second repeated segment and the segment between two adjacent second repeated segments. Segment encoding is performed on the N audio sub-segments to obtain a segment encoding sequence. Deduplication is then performed on the N audio sub-segments to obtain simplified segments corresponding to the audio segment. Optionally, each audio sub-segment can be determined as a fragmented segment.

[0094] At this time, when transmitting audio data from the cloud gaming application to the service device, the cloud service device can generate a loop start instruction, determine the first audio segment to be transmitted, obtain the segment code sequence to be transmitted corresponding to the first audio segment to be transmitted, and determine the simplified segment to be transmitted corresponding to the audio segment to be transmitted as the first audio segment. The loop start instruction, the segment code sequence to be transmitted, and the first audio segment are sequentially sent to the service device.

[0095] Optionally, the cloud service device can obtain the audio data of the cloud gaming application and detect the number of audio streams corresponding to the audio segments included in the audio data. If the number of audio streams is not one, the audio segment is split into M initial audio streams. Optionally, the audio repetition breakpoints corresponding to the M initial audio streams can be obtained; M is a positive integer; based on the audio repetition breakpoints corresponding to the M initial audio streams, the M initial audio streams are converted into M deduplicated audio streams. At this time, when the audio data in the cloud gaming application is transmitted to the business device, a loop start instruction is generated, and the loop start instruction and the loaded first audio segment are sent to the business device in sequence. Specifically, when the audio data in the cloud gaming application is transmitted to the business device, a loop start instruction is generated, the second audio segment to be transmitted is determined, and the M deduplicated audio streams corresponding to the second audio segment to be transmitted are determined as the first audio segment, or the M initial audio streams corresponding to the second audio segment to be transmitted can be determined as the first audio segment; the loop start instruction and the first audio segment are sent to the business device in sequence. Wherein, a loop start instruction can be added before the first fragment of each of the M deduplication audio streams. In this case, each deduplication audio stream may include one or more fragments. For example, see Figure 7 , Figure 7 This is a schematic diagram of a multi-audio stream transmission scenario provided by an embodiment of the present application. Figure 7 As shown, the cloud service device 701 can obtain M deduplication audio streams, add loop instructions to each deduplication audio stream, such as loop start instructions and data end instructions, and send each deduplication audio stream with loop instructions added to the service device 702; the service device 702 can parse the M deduplication audio streams with loop instructions added to obtain M deduplication audio streams, and mix and play the M deduplication audio streams. Figure 7 In the process, M deduplicated audio streams 703 are mixed and played. Through this process, when there are multiple audio streams in an audio segment, different audio streams can be looped separately, so that different audio streams can be looped with different loop lengths, that is, the one-way audio lengths of different audio streams can be different, thereby minimizing the length of the complete segment that needs to be transmitted when transmitting the audio segment, thereby reducing the traffic generated by audio transmission and reducing traffic overhead.

[0096] Optionally, the cloud service device can use any method to initiate audio transmission. For example, it can respond to a business device's request to load a game scene and send the request to load the game scene to the game management device, so that the game management device begins to transmit the audio data of the cloud gaming application in the game scene to the business device. Alternatively, it can respond to a business device's trigger operation for an audio playback component and send the trigger operation for the audio playback component to the game management device, so that the game management device transmits the audio data corresponding to the audio playback component in the cloud gaming application to the business device, etc. The specific method is determined by the audio playback trigger method provided by the cloud gaming application and is not limited here.

[0097] Step S302: When the duration of the first audio segment that has been sent is the one-way audio length, a data end instruction is sent to the service device.

[0098] In an embodiment of the present application, a data end instruction is used to enable the business device to play the first audio clip in a loop when no loop end instruction is received. Optionally, when there is an association relationship between the cloud gaming application and the cloud gaming service, the one-way audio length sent by the game management device can be directly obtained; or, when there is no association relationship between the cloud gaming application and the cloud gaming service, the one-way audio length can be obtained through a hook detection program, such as obtaining the one-way audio length from the function parameters transmitted by the audio loading function. When the duration of the first audio clip that has been sent is the one-way audio length, that is, the audio clip that needs to be played in a loop has been transmitted to the business device, a data end instruction can be sent to the business device.

[0099] Taking the k fragments shown in step S301 as an example, the cloud service device can send "loop start instruction + fragment 1, fragment 2, ... fragment 3 + data end instruction" to the business device in sequence, such as Figure 2 The transmission scenario of the first audio clip 1 is shown in FIG.

[0100] Step S303: When the loop is detected to be finished, a loop end instruction is sent to the service device, so that the service device ends playing the first audio clip based on the loop end instruction.

[0101] In an embodiment of the present application, when there is an association relationship between the cloud gaming application and the cloud gaming service, the cloud service device can run the game logic of the cloud gaming application through the game process. When the audio playback ends, a loop end message is sent to the cloud gaming service. The cloud service device generates a loop end instruction through the cloud gaming service, such as Figure 4 When there is no association between the cloud gaming application and the cloud gaming service, the loop may be terminated when the hook detection program detects that the audio stop function in the audio association function is called; or, the loop count and the duration of the first audio segment are obtained from the function parameters, and the loop duration is determined based on the loop count and the duration of the first audio segment. When the loop duration is reached, the loop is terminated, as shown in FIG. Figure 5 The loop end instruction shown in .

[0102] In an embodiment of the present application, when the cloud service device transmits the audio data in the cloud gaming application to the business device, it generates a loop start instruction, and sends the loop start instruction and the loaded first audio clip to the business device in sequence; when the duration of the first audio clip that has been sent is the one-way audio length, a data end instruction is sent to the business device; the data end instruction is used to enable the business device to play the first audio clip in a loop when the loop end instruction is not received; when the end of the loop is detected, a loop end instruction is sent to the business device, so that the business device ends playing the first audio clip based on the loop end instruction. In this way, different instructions are inserted into the audio transmission, and the loop start instruction to the data end instruction is used to determine the audio segment that needs to be played in a loop, and the loop end instruction is used to determine the time to end the loop playback, so that when the audio segment needs to be played repeatedly, it only needs to send the audio segment to the business device once, and the loop start instruction and the data end instruction enable the business device to determine the audio segment that needs to be played in a loop, and end the loop playback of the audio segment based on the loop end instruction, so that the traffic generated by the audio transmission from the cloud service device to the business device can be reduced, and the traffic overhead can be reduced; at the same time, the business device does not need to preload the audio data, and only needs to store a repeated audio segment each time the audio is transmitted to achieve the playback of the audio data, so that the transmission of audio data takes up less storage of the business device, and there is no need to update and maintain the audio segments stored in the business device, thereby saving resources.

[0103] Further, see Figure 8 , Figure 8 This is a flow chart of another method for audio transmission provided by an embodiment of the present application. Figure 8 As shown, the audio transmission process is performed by the service device and may specifically include the following steps:

[0104] Step S801: receiving a loop start instruction sent by a cloud service device, and storing a first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction.

[0105] In an embodiment of the present application, a loop start instruction is generated by a cloud service device when transmitting audio data from a cloud gaming application to a service device. Specifically, the service device can receive the loop start instruction sent by the cloud service device, store, and play the first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction. Optionally, the first audio segment includes a simplified segment to be transmitted. The service device can receive the loop start instruction sent by the cloud service device, obtain the encoding sequence of the segment to be transmitted sent by the cloud service device, store the simplified segment to be transmitted sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction, decode the simplified segment to be transmitted into a first audio segment to be transmitted based on the encoding sequence of the segment to be transmitted, and play the first audio segment to be transmitted. For example, if the segment to be transmitted is encoded as "1, 2, 2, 3, 4, 1" and the simplified segment to be transmitted is "audio sub-segment 1, audio sub-segment 2, audio sub-segment 3, audio sub-segment 4," the simplified segment to be transmitted can be converted into the first audio segment "audio sub-segment 1, audio sub-segment 2, audio sub-segment 2, audio sub-segment 3, audio sub-segment 4, audio sub-segment 1."

[0106] Optionally, the first audio clip includes M deduplicated audio streams, where M is a positive integer. The service device may receive a loop start instruction sent by the cloud service device, store the M deduplicated audio streams sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction, and mix and play the M deduplicated audio streams.

[0107] Step S802: When a data end instruction is received, the first audio segment is played in a loop until a loop end instruction sent by the cloud service device is received, and the first audio segment is stopped.

[0108] In the embodiment of the present application, when a data end instruction is received, the first audio clip is played in a loop until a loop end instruction sent by the cloud service device is received, and the first audio clip is stopped. Figure 7 As shown in , the M deduplicated audio streams are looped separately and mixed and played in sequence.

[0109] Further, see Figure 9 , Figure 9 Schematic diagram of an audio transmission device provided in an embodiment of the present application. The audio transmission device can be a computer program (including program code, etc.) running on a computer device. For example, the audio transmission device can be an application software; the device can be used to execute the corresponding steps of the method provided in an embodiment of the present application. Figure 9 As shown, the audio transmission device 900 can be used Figure 3The computer device in the corresponding embodiment is applied to a cloud service device. Specifically, the device may include: a data sending module 11, an instruction sending module 12 and an end sending module 13.

[0110] The data sending module 11 is configured to generate a loop start instruction when transmitting audio data in the cloud gaming application to the service device, and sequentially send the loop start instruction and the loaded first audio segment to the service device; the audio data includes the first audio segment;

[0111] The instruction sending module 12 is used to send a data end instruction to the service device when the length of the first audio clip that has been sent is the one-way audio length; the data end instruction is used to cause the service device to play the first audio clip in a loop if no loop end instruction is received;

[0112] The end sending module 13 is configured to send a loop end instruction to the service device when detecting that the loop ends, so that the service device ends playing the first audio segment based on the loop end instruction.

[0113] The device 900 further includes:

[0114] The protocol construction module 14 is used to send an association request to the game management device associated with the cloud game application, and establish a game transmission protocol with the game management device based on the association request;

[0115] The data sending module 11 is specifically used for:

[0116] Receive a loop start message sent by the game management device through the game transmission protocol, start the audio transmission process, and generate a loop start instruction based on the loop start message during the audio transmission process; the audio transmission process is used to transmit the audio data in the cloud game application to the business device;

[0117] The first audio segment is loaded based on the loop start message, and the loop start instruction and the first audio segment are sequentially sent to the service device.

[0118] When the first audio clip is loaded based on the loop start message, the data sending module 11 is specifically configured to:

[0119] Determine the audio identifier to be transmitted and the transmission order based on the loop start message, and sequentially load the first audio segment indicated by the audio identifier to be transmitted from the audio data stored in the cloud service device in the transmission order; the audio data refers to the audio associated with the cloud gaming application; the cloud service device refers to the terminal that generates the loop start instruction; or,

[0120] The first audio clip is loaded in real time from the game management device based on the loop start message.

[0121] The device 900 further includes:

[0122] A hook adding module 15 is used to obtain the audio related function in the cloud game application and add a hook detection program for the audio related function;

[0123] The data sending module 11 is specifically used for:

[0124] When the hook detection program detects that the audio loading function in the audio associated function is called, a loop start instruction is generated;

[0125] Obtaining the function parameter transmitted by the audio loading function through the hook detection program, and obtaining the first audio segment loaded within the one-way audio length indicated by the function parameter;

[0126] A loop start instruction and a first audio segment are sequentially sent to the service device.

[0127] The device 900 further includes:

[0128] The end determination module 16 is configured to determine that the loop ends when the hook detection program detects that the audio stop function in the audio association function is called; or

[0129] The end determination module 16 is further configured to obtain the number of loops and the duration of the first audio segment in the function parameters, determine the duration of the loop based on the number of loops and the duration of the first audio segment, and determine that the loop ends when the duration of the loop is reached.

[0130] The device 900 further includes:

[0131] An audio deduplication module 17 is configured to obtain audio data from a cloud gaming application, perform duplicate segment detection on audio segments included in the audio data, obtain a first duplicate segment, and obtain a second duplicate segment in the first duplicate segment whose segment length is greater than or equal to a duplicate splitting threshold;

[0132] The audio deduplication module 17 is further configured to split the audio segment into N audio sub-segments based on the second repeated segment, perform segment encoding on the N audio sub-segments, and obtain a segment code sequence; identical audio sub-segments in the N audio sub-segments have the same corresponding segment codes in the segment code sequence; N is a positive integer;

[0133] The audio deduplication module 17 is further configured to perform deduplication processing on the N audio sub-segments to obtain simplified segments corresponding to the audio segments;

[0134] The data sending module 11 is specifically used for:

[0135] When transmitting audio data in the cloud gaming application to the service device, a loop start instruction is generated, a first audio segment to be transmitted is determined, a segment encoding sequence to be transmitted corresponding to the first audio segment to be transmitted is obtained, and a simplified segment to be transmitted corresponding to the audio segment to be transmitted is determined as the first audio segment;

[0136] A loop start instruction, a coding sequence of segments to be transmitted, and a first audio segment are sequentially sent to the service device.

[0137] The device 900 further includes:

[0138] The audio splitting module 18 is used to obtain the audio data of the cloud gaming application and detect the number of audio streams corresponding to the audio segments included in the audio data;

[0139] The audio splitting module 18 is further configured to split the audio segment into M initial audio streams if the number of audio streams is not one, and obtain audio repetition breakpoints corresponding to the M initial audio streams, respectively; M is a positive integer;

[0140] The audio splitting module 18 is further configured to convert the M initial audio streams into M deduplicated audio streams based on the audio duplication breakpoints corresponding to the M initial audio streams;

[0141] The data sending module 11 is specifically used for:

[0142] When transmitting audio data in the cloud gaming application to the service device, a loop start instruction is generated, a second audio segment to be transmitted is determined, and M deduplicated audio streams corresponding to the second audio segment to be transmitted are determined as the first audio segment;

[0143] A loop start instruction and a first audio segment are sequentially sent to the service device.

[0144] The device 900 further includes:

[0145] The transmission trigger module 19 is used to respond to the loading request of the business device for the game scene, and send the loading request for the game scene to the game management device, so that the game management device starts to transmit the audio data of the cloud game application in the game scene to the business device; or

[0146] The transmission trigger module 19 is also used to respond to the trigger operation of the business device on the audio playback component, and send the trigger operation on the audio playback component to the game management device, so that the game management device starts to transmit the audio data corresponding to the audio playback component in the cloud game application to the business device.

[0147] Further, see Figure 10 , Figure 10Schematic diagram of another audio transmission device provided in an embodiment of the present application. The audio transmission device can be a computer program (including program code, etc.) running on a computer device. For example, the audio transmission device can be an application software; the device can be used to execute the corresponding steps of the method provided in an embodiment of the present application. Figure 10 As shown, the audio transmission device 1000 can be used for Figure 8 The computer device in the corresponding embodiment is applied to business equipment. Specifically, the device may include: a data receiving module 21 and an audio playing module 22 .

[0148] The data receiving module 21 is configured to receive a loop start instruction sent by the cloud service device and store the first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction; the loop start instruction is generated by the cloud service device when transmitting audio data in the cloud gaming application to the service device;

[0149] The audio playback module 22 is used to play the first audio clip in a loop when a data end instruction is received, until a loop end instruction sent by the cloud service device is received, and the first audio clip is stopped from playing; the data end instruction is generated by the cloud service device when the length of the first audio clip that has been sent is the one-way audio length.

[0150] The first audio segment includes a simplified segment to be transmitted; the data receiving module 21 is specifically configured to:

[0151] Receive a loop start instruction sent by the cloud service device, and obtain a code sequence of the segment to be transmitted sent by the cloud service device;

[0152] storing the simplified segments to be transmitted sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction;

[0153] The simplified segment to be transmitted is decoded into a first audio segment to be transmitted based on the encoding sequence of the segment to be transmitted, and the first audio segment to be transmitted is played.

[0154] The first audio clip includes M deduplicated audio streams; M is a positive integer;

[0155] The data receiving module 21 is specifically configured to:

[0156] Receiving a loop start instruction sent by the cloud service device, and storing M deduplicated audio streams sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction;

[0157] M deduplicated audio streams are mixed and played.

[0158] An embodiment of the present application provides an audio transmission device, which can run in a cloud service device. When transmitting audio data in a cloud gaming application to a business device, the device can generate a loop start instruction, and send the loop start instruction and the loaded first audio clip to the business device in sequence; when the duration of the first audio clip that has been sent is the one-way audio length, a data end instruction is sent to the business device; the data end instruction is used to enable the business device to play the first audio clip in a loop when the loop end instruction is not received; when the end of the loop is detected, a loop end instruction is sent to the business device, so that the business device ends playing the first audio clip based on the loop end instruction. In this way, different instructions are inserted into the audio transmission, and the loop start instruction to the data end instruction is used to determine the audio segment that needs to be played in a loop, and the loop end instruction is used to determine the time to end the loop playback, so that when the audio segment needs to be played repeatedly, it only needs to send the audio segment to the business device once, and through the loop start instruction and the data end instruction, the business device can determine the audio segment that needs to be played in a loop, and end the loop playback of the audio segment based on the loop end instruction, so that the traffic generated by the cloud service device transmitting the audio data in the cloud game application to the business device can be reduced, thereby reducing traffic overhead; at the same time, the business device does not need to preload audio data, and only needs to store a repeated audio segment each time the audio data in the cloud game application is transmitted to the business device to achieve the playback of the audio data, so that the transmission of audio data takes up less storage of the business device, and there is no need to update and maintain the audio segments stored in the business device, thereby saving resources.

[0159] See also Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 11 As shown, the computer device in the embodiment of the present application may include: one or more processors 1101, memory 1102, and input / output interface 1103. The processor 1101, memory 1102, and input / output interface 1103 are connected via a bus 1104. The memory 1102 is used to store computer programs, which include program instructions. The input / output interface 1103 is used to receive and output data, such as for data exchange between a game management device and a cloud service device, or for data exchange between a cloud service device and a business device. The processor 1101 is used to execute the program instructions stored in the memory 1102.

[0160] When the processor 1101 is integrated into a cloud service device, it can perform the following operations:

[0161] When transmitting audio data in the cloud gaming application to the service device, a loop start instruction is generated, and the loop start instruction and the loaded first audio clip are sequentially sent to the service device;

[0162] When the duration of the first audio clip that has been sent is the one-way audio length, sending a data end instruction to the service device; the data end instruction is used to enable the service device to play the first audio clip in a loop if no loop end instruction is received;

[0163] When the end of the loop is detected, a loop end instruction is sent to the service device, so that the service device ends playing the first audio segment based on the loop end instruction.

[0164] When the processor 1101 is integrated into a service device, it can perform the following operations:

[0165] Receiving a loop start instruction sent by the cloud service device, and storing a first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction;

[0166] When the data end instruction is received, the first audio segment is played in a loop until a loop end instruction sent by the cloud service device is received, and the playing of the first audio segment is stopped.

[0167] In some feasible implementations, the processor 1101 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0168] The memory 1102 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1101 and the input / output interface 1103. A portion of the memory 1102 may also include a non-volatile random access memory. For example, the memory 1102 may also store device type information.

[0169] In a specific implementation, the computer device can execute the following operations through its built-in functional modules: Figure 3 or Figure 8 For details on the implementation methods provided in each step, please refer to the Figure 3or Figure 8 The implementation methods provided in each step are not repeated here.

[0170] The embodiment of the present application provides a computer device, including: a processor, an input and output interface, and a memory, wherein the processor obtains a computer program in the memory and executes the computer program. Figure 3 or Figure 8 The various steps of the method shown in the embodiment of the present application are used to transmit the audio data in the cloud gaming application to the business device for operation. The embodiment of the present application realizes that when the audio data in the cloud gaming application is transmitted to the business device, a loop start instruction is generated, and the loop start instruction and the loaded first audio clip are sequentially sent to the business device; when the duration of the first audio clip that has been sent is the one-way audio length, a data end instruction is sent to the business device; the data end instruction is used to make the business device play the first audio clip in a loop when the loop end instruction is not received; when the end of the loop is detected, a loop end instruction is sent to the business device, so that the business device ends playing the first audio clip based on the loop end instruction. In this way, different instructions are inserted into the audio transmission, and the loop start instruction to the data end instruction is used to determine the audio segment that needs to be played in a loop, and the loop end instruction is used to determine the time to end the loop playback, so that when the audio segment needs to be played repeatedly, it only needs to send the audio segment to the business device once, and through the loop start instruction and the data end instruction, the business device can determine the audio segment that needs to be played in a loop, and end the loop playback of the audio segment based on the loop end instruction, so that the traffic generated by the cloud service device transmitting the audio data in the cloud game application to the business device can be reduced, thereby reducing traffic overhead; at the same time, the business device does not need to preload audio data, and only needs to store a repeated audio segment each time the audio data in the cloud game application is transmitted to the business device to achieve the playback of the audio data, so that the transmission of audio data takes up less storage of the business device, and there is no need to update and maintain the audio segments stored in the business device, thereby saving resources.

[0171] The present invention also provides a computer-readable storage medium storing a computer program suitable for being loaded and executed by the processor. Figure 3 or Figure 8 The audio transmission method provided in each step is detailed in this Figure 3 or Figure 8The implementation methods provided in each step will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the computer program can be deployed to be executed on one computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0172] The computer-readable storage medium may be the audio transmission device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0173] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 3 or Figure 8The method provided in the various optional methods realizes the insertion of different instructions in audio transmission. The loop start instruction to the data end instruction are used to determine the audio segment that needs to be played in a loop, and the loop end instruction is used to determine the time to end the loop playback, so that when the audio segment needs to be played repeatedly, it only needs to send the audio segment to the business device once, and through the loop start instruction and the data end instruction, the business device can determine the audio segment that needs to be played in a loop, and end the loop playback of the audio segment based on the loop end instruction, so that the traffic generated by the cloud service device transmitting the audio data in the cloud game application to the business device can be reduced, thereby reducing traffic overhead; at the same time, the business device does not need to preload audio data, and only needs to store a repeated audio segment each time the audio data in the cloud game application is transmitted to the business device to achieve the playback of the audio data, so that the transmission of audio data takes up less storage of the business device, and there is no need to update and maintain the audio segments stored in the business device, thereby saving resources.

[0174] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0175] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0176] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0177] The methods and related devices provided by the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided by the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable audio transmission device to generate a machine, so that the instructions executed by the processor of the computer or other programmable audio transmission device generate instructions for implementing the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer readable memory that can guide a computer or other programmable audio transmission device to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including an instruction device, which implements the function specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable audio transmission device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0178] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0179] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0180] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An audio transmission method, characterized in that: The method is performed by a cloud service device, and includes: When transmitting audio data in the cloud gaming application to a service device, generating a loop start instruction, and sequentially sending the loop start instruction and the loaded first audio segment to the service device; the audio data includes the first audio segment; When the duration of the first audio clip that has been sent is the one-way audio length, sending a data end instruction to the service device; the data end instruction is used to enable the service device to play the first audio clip in a loop unless a loop end instruction is received; When the end of the loop is detected, the loop end instruction is sent to the service device, so that the service device ends playing the first audio segment based on the loop end instruction.

2. The method according to claim 1, wherein The method further comprises: Sending an association request to a game management device associated with the cloud gaming application, and establishing a game transmission protocol with the game management device based on the association request; When transmitting the audio data in the cloud gaming application to the service device, generating a loop start instruction, and sequentially sending the loop start instruction and the loaded first audio clip to the service device, including: Receiving a loop start message sent by the game management device via the game transmission protocol, starting an audio transmission process, and generating a loop start instruction based on the loop start message during the audio transmission process; the audio transmission process is used to transmit audio data in the cloud gaming application to the service device; The first audio segment is loaded based on the loop start message, and the loop start instruction and the first audio segment are sequentially sent to the service device.

3. The method according to claim 2, wherein The loading of the first audio clip based on the loop start message includes: Determine the audio identifier to be transmitted and the transmission order based on the loop start message, and sequentially load the first audio segment indicated by the audio identifier to be transmitted from the audio data stored in the cloud service device according to the transmission order; the audio data refers to audio associated with the cloud gaming application; or The first audio clip is loaded in real time from the game management device based on the loop start message.

4. The method according to claim 1, wherein The method further comprises: Obtain an audio-related function in a cloud gaming application, and add a hook detection program to the audio-related function; When transmitting the audio data in the cloud gaming application to the service device, generating a loop start instruction, and sequentially sending the loop start instruction and the loaded first audio clip to the service device, including: When the hook detection program detects that the audio loading function in the audio association function is called, a loop start instruction is generated; Obtaining, through the hook detection program, a function parameter transmitted by the audio loading function, and obtaining a first audio segment loaded within a one-way audio length indicated by the function parameter; The loop start instruction and the first audio segment are sequentially sent to the service device.

5. The method according to claim 4, wherein The method further comprises: When the hook detection program detects that the audio stop function in the audio associated function is called, determining that the loop ends; or, The number of loops and the duration of the first audio segment in the function parameters are obtained, a loop duration is determined according to the number of loops and the duration of the first audio segment, and the loop is determined to be ended when the loop duration is reached.

6. The method according to claim 1, wherein The method further comprises: Acquire audio data of a cloud gaming application, perform repeated segment detection on audio segments included in the audio data to obtain a first repeated segment, and obtain a second repeated segment in the first repeated segment whose segment length is greater than or equal to a repeated splitting threshold; Splitting the audio segment into N audio sub-segments based on the second repeated segment, performing segment encoding on the N audio sub-segments to obtain a segment code sequence; identical audio sub-segments among the N audio sub-segments have identical corresponding segment codes in the segment code sequence; N is a positive integer; Deduplication processing is performed on the N audio sub-segments to obtain a simplified segment corresponding to the audio segment; When transmitting the audio data in the cloud gaming application to the service device, generating a loop start instruction, and sequentially sending the loop start instruction and the loaded first audio clip to the service device, including: When transmitting audio data in a cloud gaming application to a service device, a loop start instruction is generated, a first audio segment to be transmitted is determined, a segment encoding sequence to be transmitted corresponding to the first audio segment to be transmitted is obtained, and a simplified segment to be transmitted corresponding to the audio segment to be transmitted is determined as the first audio segment; The loop start instruction, the encoding sequence of the segment to be transmitted, and the first audio segment are sequentially sent to the service device.

7. The method according to claim 1, wherein The method further comprises: Obtaining audio data of a cloud gaming application, and detecting the number of audio streams corresponding to audio segments included in the audio data; If the number of the audio streams is not one, split the audio clip into M initial audio streams, and obtain the audio repetition breakpoints corresponding to the M initial audio streams respectively; M is a positive integer; Converting the M initial audio streams into M deduplicated audio streams based on the audio duplication breakpoints corresponding to the M initial audio streams; When transmitting the audio data in the cloud gaming application to the service device, generating a loop start instruction, and sequentially sending the loop start instruction and the loaded first audio clip to the service device, including: When transmitting audio data in the cloud gaming application to the service device, a loop start instruction is generated, a second audio segment to be transmitted is determined, and M deduplicated audio streams corresponding to the second audio segment to be transmitted are determined as the first audio segment; The loop start instruction and the first audio segment are sequentially sent to the service device.

8. The method according to claim 1, wherein The method further comprises: In response to the service device's request to load the game scene, the service device sends the request to load the game scene to the game management device, so that the game management device starts transmitting the audio data of the cloud game application in the game scene to the service device; or In response to the triggering operation of the audio playback component by the business device, the triggering operation of the audio playback component is sent to the game management device, so that the game management device starts to transmit the audio data corresponding to the audio playback component in the cloud game application to the business device.

9. An audio transmission method, characterized in that: The method is performed by a service device, and includes: Receiving a loop start instruction sent by a cloud service device, and storing a first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction; the loop start instruction is generated by the cloud service device when transmitting audio data in the cloud gaming application to the business device; When the data end instruction is received, the first audio clip is played in a loop until the loop end instruction sent by the cloud service device is received, and the first audio clip is stopped; the data end instruction is generated by the cloud service device when the length of the first audio clip that has been sent is the one-way audio length.

10. The method according to claim 9, wherein The first audio segment includes a simplified segment to be transmitted; the receiving loop start instruction sent by the cloud service device, and storing the first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction, comprises: Receiving a loop start instruction sent by a cloud service device, and obtaining a code sequence of a segment to be transmitted sent by the cloud service device; storing the simplified segments to be transmitted sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction; The simplified segment to be transmitted is decoded into a first audio segment to be transmitted based on the encoding sequence of the segment to be transmitted, and the first audio segment to be transmitted is played.

11. The method according to claim 9, wherein The first audio segment includes M deduplicated audio streams; M is a positive integer; The receiving a loop start instruction sent by the cloud service device and storing the first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction includes: Receiving a loop start instruction sent by a cloud service device, and storing M deduplicated audio streams sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction; The M deduplicated audio streams are mixed and played.

12. An audio transmission device, characterized in that: The device is applied to a cloud service device, and includes: a data sending module, configured to generate a loop start instruction when transmitting audio data in a cloud gaming application to a service device, and sequentially send the loop start instruction and a loaded first audio segment to the service device; the audio data including the first audio segment; an instruction sending module, configured to send a data end instruction to the service device when the duration of the first audio segment that has been sent reaches the one-way audio length; the data end instruction is configured to cause the service device to play the first audio segment in a loop unless a loop end instruction is received; The end sending module is configured to send the loop end instruction to the service device when detecting that the loop ends, so that the service device ends playing the first audio segment based on the loop end instruction.

13. An audio transmission device, characterized in that: The device is applied to a service device and includes: a data receiving module, configured to receive a loop start instruction sent by a cloud service device and store a first audio segment sent by the cloud service device between receiving the loop start instruction and receiving the data end instruction; the loop start instruction is generated by the cloud service device when transmitting audio data in a cloud gaming application to the service device; An audio playback module is used to play the first audio clip in a loop when the data end instruction is received, until a loop end instruction sent by the cloud service device is received, and the playback of the first audio clip is stopped; the data end instruction is generated by the cloud service device when the length of the first audio clip that has been sent is the one-way audio length.

14. A computer device, characterized in that: Includes processor, memory, input and output interfaces; The processor is connected to the memory and the input / output interface, respectively, wherein the input / output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1 to 8, or executes the method described in any one of claims 9 to 11.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 8, or executes the method according to any one of claims 9 to 11.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented, or the method according to any one of claims 9 to 11 is executed.

Images