A method for playing video ringback tone
By stripping the video ringtone decoding process from the playback process, recording it into a file, and directly sending RTP messages during playback, the problem of video ringtone decoding consumes CPU resources, and the load capacity of the ringtone server is improved and the playback is achieved. stability.
Patent Information
- Application Number
- CN202311353681.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-19
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-10-19
AI Technical Summary
The decoding process of video ringtone consumes a lot of CPU resources, resulting in poor load capacity of the ringtone server.
The ring ring server is divided into a preprocessing server and a playback server. The decoding server decodes the video file into an RTP stream and records it into a file. The player reads and sends RTP messages from the file, allowing you to define the number of times SPS/PPS messages to be sent. To ensure smooth playback.
It reduces CPU consumption during ringtone playback, prevents playback failure, and optimizes the sending rules of RTP packets to ensure smooth playback of ringtones.
Smart Images

Figure CN117319555B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video color ring back tone, and in particular to a method for playing video color ring back tone. Background Art
[0002] Video ringback tone is a service customized by the called user to provide the calling user with a pleasant, colorful or interesting video media to replace the ordinary ringback tone. During the playback of the video ringback tone, the decoder reads the video file of the video ringback tone, decodes it, and then converts it into an RTP message, which is sent to the caller through the RTP server. Among them, the decoding process consumes the most CPU resources, resulting in poor load capacity of the ringback tone server.
[0003] Compared with audio and video calls, video ringback tone has the following characteristics: the same content, when the same ringback tone is played in the same format, its RTP content is exactly the same; it is one-way, the ringback tone server sends an RTP message to the caller, and the caller does not need to reply to RTCP (Real-time Transport Control Protocol, a control protocol for real-time transport protocol RTP) to perform complex feedback such as TMMBR and NACK; loss is allowed, as an auxiliary ringing method, RTP message loss does not require complex operations such as retransmission; at the same time, if the SPS / PPS message is lost during playback, the mobile phone cannot decode the video, resulting in the failure of ringback tone playback.
[0004] Therefore, according to the characteristics of video ring back tone, a method for playing video ring back tone can be provided to solve the problem that during the playing process of video ring back tone, a decoding process consumes a large amount of CPU resources, resulting in poor load capacity of the ring back tone server. Summary of the invention
[0005] The present invention aims at the deficiencies in the prior art and provides a method for playing video ringback tone to solve the problem that a decoding process consumes a large amount of CPU resources during the playing of the video ringback tone, resulting in poor load capacity of the ringback tone server.
[0006] To achieve the above object, the present invention adopts the following technical solutions:
[0007] A method for playing video ring back tone, characterized in that: a ring back tone server is divided into a preprocessing server and a playing server, the preprocessing server includes a decoding server and a recorder, the playing server includes a ring back tone audio file, a ring back tone video file and a player, the decoding server is used to decode the video file and output it as an RTP stream of audio and video, the decoding server pushes the output audio and video RTP streams to the recorder by listening to the fixed IP and audio and video ports of the recorder; the input audio and video RTP streams are respectively recorded by the recorder and stored in the corresponding ring back tone audio file and ring back tone video file; the player selects the corresponding ring back tone audio file and ring back tone video file to be played according to the audio and video decoder, reads the corresponding RTP message from the ring back tone audio file and the ring back tone video file, and divides them into audio and video RTP streams and sends them to the caller respectively for playback.
[0008] To optimize the above technical solutions, the specific measures taken also include:
[0009] Furthermore, the decoding server includes a video file, a decoder and an RTP server. The decoder is used to read the video file for decoding and push it to the RTP server, and output it as an RTP stream of audio and video through the RTP server.
[0010] Furthermore, before the decoding server listens to the recorder, the decoding server manually specifies the IP address of the recorder, the audio and video port and the audio and video codec of the recorder for receiving the RTP stream of audio and video.
[0011] Furthermore, the bottom layers of the CRBT audio file and the CRBT video file are stored in the same LV format, where L is a 4-byte unsigned int, storing the length of the RTP message including the RTP header, and V is the recorded RTP message content. The CRBT audio file and the CRBT video file are respectively composed of arrays of several LVs.
[0012] Furthermore, the file name of the ringback tone audio file includes: the original ringback tone name excluding the extension part, the audio decoder and the audio sampling rate, and the file name of the ringback tone video file includes: the original ringback tone name excluding the extension part, the video decoder and the video resolution.
[0013] Furthermore, in the process that the player reads the corresponding RTP messages from the ringback tone audio file and the ringback tone video file, and divides them into audio and video RTP streams and sends them to the caller for playback, it is allowed to define the number of times the SPS / PPS message is sent.
[0014] Furthermore, in the process that the player selects the corresponding ringback tone audio file and ringback tone video file to be played according to the audio and video decoder, reads the corresponding RTP message from the ringback tone audio file and the ringback tone video file, and separates the RTP streams of audio and video into audio and video streams and sends them to the caller respectively for playing, the following steps are specifically included:
[0015] Step 1, input the file name of the ring back tone audio file and the ring back tone video file, the PT of RTP and the number of SPS / PPS messages to be sent, open the ring back tone audio file and the ring back tone video file through the audio and video decoder, record the corresponding SPS / PPS message, and initialize the global variable;
[0016] Step 2, sending multiple SPS / PPS messages according to the configuration;
[0017] Step 3, send a video frame and an audio frame at the beginning, wait for 20ms, then send an audio frame and wait for 20ms; repeat the sending logic until the ringback tone stops playing.
[0018] Furthermore, the step 1 includes: inputting the file names of the color ring audio file and the color ring video file, the PT of RTP and the number of SPS / PPS messages to be sent, and opening the color ring audio file and the color ring video file through the audio and video decoder; thereafter, reading the SPS message of the first color ring video message, and modifying the PT of the header to the PT during negotiation, and caching the modified SPS message, and then reading the PPS message of the second color ring video message, and modifying the PT of the header to the PT during negotiation, and caching the modified PPS message; initializing the video Timestamp to -1, initializing the video Seq to the Seq of the SPS message, and reading the I frame of the third color ring video message as the video cache message.
[0019] Furthermore, step 2 includes: when the number of SPS / PPS messages to be sent is greater than 0 and the video cache message is an I frame, modify the Seq of the cached SPS message to be a video Seq, the Timestamp to be the Timestamp of the I frame, video Seq+1, and then send the SPS message, modify the Seq of the cached PPS message to be a video Seq, the Timestamp to be the Timestamp of the I frame, video Seq+1, and then send the PPS message, and reduce the number of SPS / PPS messages to be sent by -1; otherwise, jump directly to step 3.
[0020] Further, the step 3 includes: modifying the Seq and PT of the video cache message in step 1, video Seq+1, modifying the PT of the header to the PT during negotiation, sending the modified video cache message, and reading the next video cache message as a cache; when the reading of the next video cache message fails or the color ring tone ends, the color ring tone is stopped, otherwise, the Timestamps of the two video cache messages are compared; when the Timestamps of the two video cache messages are the same, it means that the two video cache messages belong to the same frame, and the video cache message used as the cache continues to be modified. eq and PT, and then send them until the Timestamps of the two video cache messages are different, indicating that the current video cache message as the cache belongs to the next frame. At this time, the modified video cache message belonging to the previous frame has been sent; then, read an audio message, modify the PT of the audio message header to the PT during negotiation and send it, and sleep for 20ms; then, if the color ring tone ends, stop the color ring tone, otherwise, continue to read an audio message, modify the PT of the audio message header to the PT during negotiation and send it, and sleep for 20ms; repeat steps 2 to 3 until the color ring tone stops.
[0021] The beneficial effects of the present invention are:
[0022] The present invention separates the audio and video decoding process from the color ring tone playing process and uses it as a separate preprocessing process to record the decoded RTP message into the corresponding color ring tone audio file and color ring tone video file; when playing, only the recorded RTP message needs to be taken out, modified, and sent directly without audio and video decoding, thereby greatly reducing the CPU consumption during the color ring tone playing process. The present invention prevents the color ring tone playing failure by allowing the number of times the key information SPS / PPS message is sent to be defined; and provides technical solutions such as the modification method of the RTP message and the sending rule of the RTP message to ensure the smooth playing of the color ring tone. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 A schematic diagram of the production and playback process of a video ringback tone of a method for playing a video ringback tone proposed by the present invention;
[0024] Figure 2 Standard CRBT server operation diagram;
[0025] Figure 3 This is a flowchart of step 1 of the player operation in the method for playing video ringback tone proposed by the present invention;
[0026] Figure 4 It is a flowchart of step 2 of the player operation in the method for playing video ringback tone proposed by the present invention;
[0027] Figure 5It is a flowchart of step 3 of the player operation in the method for playing video ringback tone proposed by the present invention;
[0028] Figure 6 This is the rule for sending audio and video RTP packets. Implementation
[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0030] As attached Figure 1 As shown, a method for playing video ring back tone according to an embodiment of the present invention is as follows: a ring back tone server is divided into a pre-processing server and a playing server, the pre-processing server includes a decoding server and a recorder, the playing server includes a ring back tone audio file, a ring back tone video file and a player, and the decoding server can be used as shown in the attached Figure 2 The existing color ring back tone server shown is used to decode the standard video file and output it as audio and video RTP (Real-time Transport Protocol, which is a protocol for transmitting audio, video and other real-time data on the network) streams. The decoding server pushes the output audio and video RTP streams to the recorder by listening to the fixed IP and audio and video ports (i.e., audio / video ports) of the recorder; the input audio and video RTP streams are recorded by the recorder respectively, and stored in the corresponding color ring back tone audio files and color ring back tone video files; the player selects the corresponding color ring back tone audio files and color ring back tone video files to be played according to the final negotiated audio and video decoders, reads the corresponding RTP messages from the color ring back tone audio files and color ring back tone video files, and divides them into audio and video RTP streams and sends them to the caller respectively for playback.
[0031] In this way, the audio and video decoding process is separated from the ringback tone playback process and used as a separate preprocessing process to record the decoded RTP message into the corresponding ringback tone audio file and ringback tone video file. When playing, you only need to take out the recorded RTP message, modify it, and send it directly without audio and video decoding, thereby greatly reducing the CPU consumption during the ringback tone playback process.
[0032] In another embodiment, the decoding server includes a video file, a decoder and an RTP server. The decoder is used to read a standard video file for decoding and push it to the RTP server, and output it as an RTP stream of audio and video through the RTP server.
[0033] In another embodiment, before the decoding server listens to the recorder, the decoding server manually specifies the IP address of the recorder, the audio and video ports and the audio and video codecs of the recorder for receiving the RTP streams of audio and video. In this embodiment, the recorder, as the receiving party, has a fixed IP address, and the decoding server, as the sending party, needs to manually specify the configuration information of the recorder.
[0034] In another embodiment, the bottom layers of the CRBT audio file and the CRBT video file are stored in the same L (length) V (value) format, where L is a 4-byte unsigned int storing the length of the RTP message including the RTP header, and V is the recorded RTP message content. The CRBT audio file and the CRBT video file are respectively composed of arrays of several LVs.
[0035] In another embodiment, the file name of the CRBT audio file includes: the original CRBT name excluding the extension, the audio decoder (such as AMR-WB which usually corresponds to 16K sampling rate, PCMU which usually corresponds to 8K sampling rate) and the audio sampling rate (such as 16K, 8K), and the file name of the CRBT video file includes: the original CRBT name excluding the extension, the video decoder (such as H264, VP8) and the video resolution (such as 1080P, 720P, 480P). In this way, it is convenient to find the format supported by the corresponding user's mobile phone during playback, and the necessary information needs to be included in the file name.
[0036] In another embodiment, in the process where the player reads the corresponding RTP messages from the ringback tone audio file and the ringback tone video file, and divides them into audio and video RTP streams and sends them to the caller respectively for playback, it is allowed to define the number of times of sending key information SPS (Sequence Parameter Set) / PPS (Picture Parameter Set) messages to prevent ringback tone playback failure.
[0037] In another embodiment, the player selects the corresponding ring back tone audio file and ring back tone video file to be played according to the finally negotiated audio and video decoder, reads the corresponding RTP message from the ring back tone audio file and the ring back tone video file, and separates the RTP streams of audio and video and sends them to the caller respectively for playing, specifically including the following steps:
[0038] Step 1, input the file name of the ring back tone audio file and the ring back tone video file, the PT of RTP and the number of SPS / PPS messages to be sent, open the ring back tone audio file and the ring back tone video file through the audio and video decoder, record the corresponding SPS / PPS message, and initialize the global variable;
[0039] Step 2, sending multiple SPS / PPS messages according to the configuration;
[0040] Step 3, send a video frame and an audio frame at the beginning, wait for 20ms, then send an audio frame and wait for 20ms; repeat the sending logic until the ringback tone stops playing.
[0041] As attached Figure 3 As shown, the above step 1 includes: inputting the file names of the color ring audio file and the color ring video file, the PT of RTP and the number of SPS / PPS messages to be sent, and opening the color ring audio file and the color ring video file through the audio and video decoder; then, reading the SPS message of the first color ring video message, and modifying the PT of the header to the PT during negotiation, and caching the modified SPS message, and then reading the PPS message of the second color ring video message, and modifying the PT of the header to the PT during negotiation, and caching the modified PPS message; initializing the video Timestamp (timestamp) to -1, initializing the video Seq (sequencenumber) to the Seq of the SPS message, and reading the I frame (Intra-coded picture) of the third color ring video message as the video cache message.
[0042] The reason for modifying the PT is that the PT negotiated by SIP (Session Initiation Protocol, which is a network communication protocol used to establish, modify and terminate real-time sessions) is not necessarily the same, that is, the PT negotiated during recording is A, and the PT negotiated during sending is B; therefore, after reading the RTP message, it is necessary to modify the PT in the RTP header to the PT during negotiation before sending it out.
[0043] As attached Figure 4 As shown, the above step 2 includes: when the number of SPS / PPS messages to be sent is greater than 0, and the video cache message is an I frame, modify the Seq of the cached SPS message to be a video Seq, the Timestamp is the Timestamp of the I frame, video Seq+1, and then send the SPS message, modify the Seq of the cached PPS message to be a video Seq, the Timestamp is the Timestamp of the I frame, video Seq+1, and then send the PPS message, and reduce the number of SPS / PPS messages to be sent by 1; otherwise, jump directly to step 3.
[0044] Usually, the ringback tone player on the caller or mobile phone must receive the SPS / PPS message to determine key information such as resolution to decode the video and display it. Under normal circumstances, the SPS / PPS message is only sent once. Once lost, the mobile phone will always be in a black screen state, resulting in failure to play the video ringback tone. Therefore, in this solution, users are allowed to customize the number of times to send SPS / PPS messages, and insert SPS / PPS messages before subsequent I frames. For the insertion case, if the I frame Seq to be sent is A, Timestamp is B, and the negotiated PT is C, then:
[0045] SPS Seq is A, Timestamp is B, PT is C;
[0046] PPS Seq is A+1, Timestamp is B, PT is C;
[0047] The I frame Seq to be sent is A + 2, Timestamp is B, and PT is C.
[0048] The Seq of all subsequent video frames needs to change accordingly.
[0049] As attached Figure 5 As shown, the above step 3 includes: modifying the Seq and PT of the video cache message in step 1, video Seq+1, modifying the PT of the header to the PT during negotiation, sending the modified video cache message, and reading the next video cache message as a cache; when the next video cache message fails to be read or the color ring tone ends, the color ring tone is stopped, otherwise, the Timestamps of the two video cache messages are compared; when the Timestamps of the two video cache messages are the same, it means that the two video cache messages belong to the same frame, and the video cache message used as the cache continues to be modified Seq. q and PT, and then send them until the Timestamps of the two video cache messages are different, indicating that the current video cache message as the cache belongs to the next frame. At this time, the modified video cache message belonging to the previous frame has been sent; then, read an audio message, modify the PT of the audio message header to the PT during negotiation and send it, and sleep for 20ms; then, if the color ring tone ends, stop the color ring tone, otherwise, continue to read an audio message, modify the PT of the audio message header to the PT during negotiation and send it, and sleep for 20ms; repeat steps 2 to 3 until the color ring tone stops.
[0050] Usually, in one second, 25 video frames need to be sent, that is, each frame contains 40ms of video content, and 50 audio frames, that is, each frame contains 20ms of video content. Figure 6As shown in the figure, the sending rule is set as follows: send one video frame and one audio frame at the beginning; wait 20ms, send one audio frame; wait 20ms, and repeat the above two steps. That is, video frame (each video frame may contain multiple messages), two audio frames (each audio frame usually contains only one message), and video frame is repeated continuously.
[0051] In another embodiment, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the above-mentioned method for playing a video ringback tone.
[0052] In another embodiment, the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for playing a video ringback tone as described above is implemented.
[0053] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0054] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0055] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.
Claims
1. A method for playing a video ringback tone, characterized in that: The color ring back tone server is divided into a pre-processing server and a playback server. The pre-processing server includes a decoding server and a recorder. The playback server includes a color ring back tone audio file, a color ring back tone video file and a player. The decoding server is used to decode the video file and output it as an audio and video RTP stream. The decoding server pushes the output audio and video RTP streams to the recorder by listening to the fixed IP and audio and video ports of the recorder. Record the input audio and video RTP streams respectively through the recorder and store them in the corresponding ring back tone audio file and ring back tone video file; The player selects the corresponding ringback tone audio file and ringback tone video file to be played according to the audio and video decoder, reads the corresponding RTP message from the ringback tone audio file and the ringback tone video file, and divides them into audio and video RTP streams and sends them to the caller for playback.
2. The method for playing video ringback tone according to claim 1, characterized in that: The decoding server includes a video file, a decoder and an RTP server. The decoder is used to read the video file for decoding and push it to the RTP server, and output it as an RTP stream of audio and video through the RTP server.
3. The method for playing video ringback tone according to claim 1, characterized in that: Before the decoding server listens to the recorder, the decoding server manually specifies the recorder's IP address, the audio and video ports and the audio and video codecs used by the recorder to receive the audio and video RTP streams.
4. The method for playing video ringback tone according to claim 1, characterized in that: The bottom layers of the CRBT audio file and CRBT video file are stored in the same LV format, where L is a 4-byte unsigned int that stores the length of the RTP message including the RTP header, and V is the recorded RTP message content. The CRBT audio file and CRBT video file are respectively composed of arrays of several LVs.
5. The method for playing video ringback tone according to claim 1, characterized in that: The file name of the CRBT audio file includes: the original CRBT name excluding the extension, audio decoder and audio sampling rate; the file name of the CRBT video file includes: the original CRBT name excluding the extension, video decoder and video resolution.
6. The method for playing video ringback tone according to claim 1, characterized in that: In the process that the player reads the corresponding RTP messages from the ringback tone audio file and the ringback tone video file, and separates them into audio and video RTP streams and sends them to the caller for playback, the number of times the SPS / PPS messages are sent can be defined.
7. The method for playing video ringback tone according to claim 1, characterized in that: In the process in which the player selects the corresponding ringback tone audio file and ringback tone video file to be played according to the audio and video decoder, reads the corresponding RTP message from the ringback tone audio file and the ringback tone video file, and separates the RTP streams of audio and video into audio and video streams and sends them to the caller for playing, the specific steps include: Step 1, input the file name of the ring back tone audio file and the ring back tone video file, the PT of RTP and the number of SPS / PPS messages to be sent, open the ring back tone audio file and the ring back tone video file through the audio and video decoder, record the corresponding SPS / PPS message, and initialize the global variable; Step 2, sending multiple SPS / PPS messages according to the configuration; Step 3, send a video frame and an audio frame at the beginning, wait for 20ms, then send an audio frame and wait for 20ms; repeat the sending logic until the ringback tone stops playing.
8. The method for playing video ringback tone according to claim 7, characterized in that: The step 1 comprises: inputting the file names of the CRBT audio file and the CRBT video file, the PT of RTP and the number of SPS / PPS messages to be sent, and opening the CRBT audio file and the CRBT video file through an audio and video decoder; then, reading the SPS message of the first CRBT video message, modifying the PT of the header to the PT during negotiation, and caching the modified SPS message, then reading the PPS message of the second CRBT video message, modifying the PT of the header to the PT during negotiation, and caching the modified PPS message; initializing the video Timestamp to -1, initializing the video Seq to the Seq of the SPS message, and reading the I frame of the third CRBT video message as the video cache message.
9. The method for playing video ringback tone according to claim 8, characterized in that: The step 2 includes: when the number of SPS / PPS messages to be sent is greater than 0 and the video cache message is an I frame, modify the Seq of the cached SPS message to be a video Seq, the Timestamp to be the Timestamp of the I frame, the video Seq+1, and then send the SPS message, modify the Seq of the cached PPS message to be a video Seq, the Timestamp to be the Timestamp of the I frame, the video Seq+1, and then send the PPS message, and reduce the number of SPS / PPS messages to be sent by 1; otherwise, jump directly to step 3.
10. The method for playing video ringback tone according to claim 9, characterized in that: The step 3 includes: modifying the Seq and PT of the video cache message in step 1, the video Seq+1, modifying the PT of the head to the PT during negotiation, sending the modified video cache message, and reading the next video cache message as a cache; when the next video cache message fails to be read or the color ring tone ends, the color ring tone is stopped, otherwise, the Timestamps of the two video cache messages are compared; when the Timestamps of the two video cache messages are the same, it means that the two video cache messages belong to the same frame, and the video cache message used as the cache continues to modify the Seq and PT, and then send it again until the Timestamps of the two video cache messages are different, indicating that the current video cache message as the cache belongs to the next frame. At this time, the modified video cache message belonging to the previous frame has been sent; then, read an audio message, modify the PT of the audio message header to the PT during negotiation and send it, and sleep for 20ms; then, if the color ring tone ends, stop the color ring tone, otherwise, continue to read an audio message, modify the PT of the audio message header to the PT during negotiation and send it, and sleep for 20ms; repeat steps 2 to 3 until the color ring tone is stopped.
