Visual identification method and system for browser audio playing

Abstract

The embodiment of the invention discloses a visual identification method and system for browser audio playing. According to the technical scheme provided by the embodiment of the invention, under the condition that the target audio stream is played, the network address of each media fragment to be played of the target audio stream is acquired, and the audio proportion information of the corresponding media fragment is pre-added to the network address; the audio proportion information represents the proportion of sound marks in the audio frames in the corresponding media fragments; analyzing the network address to obtain audio proportion information, and obtaining a corresponding media fragment based on the network address; and playing the media fragments, and synchronously carrying out visual identification on the currently played media fragments based on the corresponding audio proportion information. By adopting the technical means, the audio proportion information is added into the network address of the target audio stream media fragment, and the media fragment played in real time is synchronously and visually identified according to the audio proportion information obtained by network address analysis, so that the visual identification process of audio stream playing can be simplified.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method and system for visually identifying audio playback in a browser. Background Art

[0002] At present, when playing multiple audio streams in a browser, it is often necessary to visually identify the playback of multiple audio streams. For example, in the multi-person voice room mode of a live broadcast scenario, in order to meet regulatory requirements, in addition to using an audit algorithm to automatically audit the audio stream, it is also necessary to conduct an online manual audit of the audio stream. Auditors usually need to monitor the audio streams of multiple rooms at the same time and make compliance judgments on some of the audio streams. Therefore, it is necessary to visually identify the audio playback of each audio stream to clarify which audio stream of the host end the currently playing sound comes from, so as to simplify the process of audio stream positioning and improve the audit efficiency of the audio stream. When visually identifying audio playback, since the browser's streaming media transmission protocol (such as HLS, Http Live Streaming, and Http-based streaming media network transmission protocol) does not support the business scenario of audio playback visual identification, an additional protocol will be used to separately transmit the sound identification of the audio stream to the browser, so that when playing audio, the audio stream playback can be visually identified according to the sound identification.

[0003] However, due to differences in audio transmission link network quality, audio stream cache jitter, etc., using an additional protocol to transmit the sound identifier of the audio stream separately will cause the visual identifier of the audio stream to be out of sync with the sound. At the same time, introducing an additional protocol to transmit the sound identifier will occupy additional network bandwidth and affect the operation of other business services. Summary of the invention

[0004] The embodiments of the present application provide a method and system for visual identification of browser audio playback, which can simplify the visual identification process of audio stream playback, improve browser compatibility, and solve the problem of audio stream playback in the browser and the visual identification being out of sync.

[0005] In a first aspect, an embodiment of the present application provides a method for visually identifying audio playback of a browser, comprising:

[0006] When playing the target audio stream, obtain the network address of each to-be-played media segment of the target audio stream, the network address is pre-added with the sound ratio information of the corresponding media segment, and the sound ratio information represents the ratio of the audio frame in the corresponding media segment that has the sound mark;

[0007] Parse the network address to obtain the voice ratio information, and obtain the corresponding media segment based on the network address;

[0008] Play the media segment and visually identify the currently played media segment synchronously based on the corresponding sound ratio information.

[0009] In a second aspect, an embodiment of the present application provides a visual identification system for browser audio playback, including:

[0010] An acquisition module is configured to acquire the network address of each to-be-played media segment of the target audio stream when playing the target audio stream, wherein the network address is pre-added with the sound ratio information of the corresponding media segment, and the sound ratio information represents the ratio of the audio frame in the corresponding media segment that has the sound mark;

[0011] A parsing module configured to parse a network address to obtain voice ratio information, and obtain corresponding media segments based on the network address;

[0012] The visualization module is configured to play the media segments and synchronously perform visual identification on the currently played media segments based on the corresponding sound ratio information.

[0013] In a third aspect, an embodiment of the present application provides a visual identification device for browser audio playback, including:

[0014] memory and one or more processors;

[0015] The memory is configured to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the visual identification method for browser audio playback as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a computer processor, the method for visual identification of browser audio playback as described in the first aspect is configured to execute.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the method for visual identification of browser audio playback as described in the first aspect.

[0019] The embodiment of the present application obtains the network address of each to-be-played media segment of the target audio stream while playing the target audio stream, and the network address is pre-added with the sound ratio information of the corresponding media segment, and the sound ratio information represents the ratio of the audio frames in the corresponding media segment that have sound marks; parses the network address to obtain the sound ratio information, and obtains the corresponding media segment based on the network address; plays the media segment, and synchronously visually identifies the currently played media segment based on the corresponding sound ratio information. By adopting the above-mentioned technical means, by adding the sound ratio information to the network address of the target audio streaming media segment, the real-time played media segment is synchronously visually identified according to the sound ratio information obtained by parsing the network address. The visual identification process of audio stream playback can be simplified, the visual identification efficiency of audio playback can be improved, and the browser compatibility can be improved, the situation where the bandwidth is occupied due to the transmission of sound identification by additional protocols can be avoided, and the visual identification effect of audio playback can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a method for visually identifying audio playback in a browser provided in an embodiment of the present application;

[0021] Figure 2 This is a flowchart of the audio playback visual identification display in the embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of the audio playback visual identification display in the embodiment of the present application;

[0023] Figure 4 It is another schematic diagram showing the audio playback visual identifier in the embodiment of the present application;

[0024] Figure 5 It is a structural diagram of a visual identification system for browser audio playback provided in an embodiment of the present application;

[0025] Figure 6 It is a structural diagram of a visual identification device for browser audio playback provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0027] The method for visual identification of browser audio playback provided by the present application aims to add the audio ratio information to the network address of the target audio streaming media segment, and visually identify the real-time played media segment synchronously according to the audio ratio information obtained by network address resolution, so as to simplify the visual identification process of audio stream playback and improve the visual identification efficiency of audio playback.

[0028] In the relevant audio playback visualization solution, take HLS audio stream as an example. HLS audio stream is a streaming network transmission protocol based on Http. It is mainly composed of m3u8 media file index and TS media segment file. Its index file has a serial number, and the two adjacent index serial numbers increase continuously. The working principle of HLS is to divide the entire media stream into TS segment files for browser download and playback.

[0029] When playing multiple HLS audio streams in a browser, especially pure audio streams, it is difficult to determine which stream is emitted from the pure sound. This problem becomes particularly prominent in scenarios such as online meetings and multi-person voice rooms. Therefore, it is difficult to implement the corresponding audio playback visualization UI display logic based on the standard HLS audio stream playback. In some business models, the multi-person voice room mode is the main form, and in order to meet regulatory requirements, in addition to audio machine review, online manual review is also required. Auditors usually need to monitor the audio streams of multiple rooms at the same time and make compliance judgments on some of the anchors. However, in a multi-person room, if there are two or more anchors, it is difficult to distinguish which specific sound is emitted by whom, which affects the efficiency of the review.

[0030] In order to solve the problem that the sounds of multiple HLS streams are difficult to distinguish due to the lack of HLS and browser functions, it is necessary to visually identify the audio streams synchronously when they are played. At present, audio streams can be transmitted through private protocols on PCs or mobile applications. In addition, by performing sound energy checks on the host or server side, and attaching the corresponding visual identification information to the private transmission protocol and transmitting it to the audience side for use, this information can also be transmitted to the audience side through additional protocols, thereby achieving visual identification of audio playback.

[0031] However, browsers do not support the playback of private protocols, and usually use standard streaming media transmission protocols such as WebRTC and HLS. These standard protocols do not support business scenarios such as sound visualization. If you use additional protocols to transmit sound signs separately, such as using WebSocket communication, you may encounter the following problems: differences in network quality between different users, and the impact of jitter time when the browser plays the HLS cache, resulting in asynchronous and inaccurate display of information transmitted through WebSocket. In addition, the introduction of additional protocol transmission will increase bandwidth usage and increase the risk of being blocked by operators and causing service unavailability.

[0032] Based on this, a method for visual identification of browser audio playback is provided in an embodiment of the present application to solve the problem of the sound and visual identification being out of sync when the browser audio stream is played.

[0033] Example:

[0034] Figure 1 A flowchart of a method for visual identification of browser audio playback provided in an embodiment of the present application is given. The method for visual identification of browser audio playback provided in this embodiment can be executed by a visual identification device for browser audio playback, which can be implemented by software and / or hardware. The visual identification device for browser audio playback can be composed of two or more physical entities, or can be composed of one physical entity. Generally speaking, the visual identification device for browser audio playback can be a browser-side device such as a computer, a mobile phone, or a tablet.

[0035] The following description is made by taking the visual identification device of the browser audio playback as the main body of the method for visual identification of the browser audio playback as an example. Figure 1 The visual identification method of the browser audio playback specifically includes:

[0036] S110. When playing the target audio stream, obtain the network address of each media segment to be played of the target audio stream, and add the sound ratio information of the corresponding media segment in advance to the network address. The sound ratio information represents the ratio of the audio frame in the corresponding media segment that has the sound mark.

[0037] When visually identifying audio playback, this application obtains the network address URL of each media segment of the corresponding audio stream to obtain the sound ratio information of the pre-added media segment. Taking the HLS audio stream as an example, the currently playing audio stream is defined as the target audio stream. When the target audio stream is played on the browser side, the playlist (Playlist) of the target audio stream is obtained from the server side through the streaming media transmission protocol (such as HLS). The playlist contains the network addresses of each media segment (such as TS files) in the audio stream.

[0038] For the server, each media segment is processed in advance, its sound ratio information is calculated, and the information is added to the network address URL of the media segment. The sound ratio information can be obtained by analyzing the audio frames in the media segment and counting the proportion of audio frames with sound tags to the total audio frames.

[0039] Optionally, the voice ratio information is determined according to the ratio of the number of audio frames containing sound markers to the total number of audio frames in the media segment, and the sound markers are marked based on signal strength detection results of the audio frames.

[0040] For example, taking a multi-person voice room in a live broadcast scene as an example, refer to Figure 2 , for the multi-channel audio streams collected by the host, guests and other anchors, they are processed by the browser server (web background). For the multi-channel audio streams, the sound mark is added to each audio frame of the audio stream through VAD check (Voice activity detection is a technology used for voice processing to check the strength of the voice signal). Generally speaking, when the voice signal strength reaches the set state, the sound mark is added to the audio frame.

[0041] Specifically, when performing VAD inspection on an audio stream, the speech activity of the audio stream is detected to distinguish between speech and noise signals. On the host side, VAD technology can be implemented by integrating the VAD module of WebRTC or using a JavaScript-based VAD library. The VAD algorithm of WebRTC models speech and noise based on the Gaussian mixture model (GMM). GMM is an unsupervised learning method that can effectively distinguish speech and noise by probabilistically modeling the input data. In WebRTC, the VAD algorithm first divides the input spectrum into several sub-bands and calculates the energy of each sub-band. Then, the probability density function of GMM is used to model the energy of these sub-bands to obtain a log-likelihood ratio function. The detection of speech activity is achieved through global and local judgment of the log-likelihood ratio. The workflow of WebRTC VAD includes steps such as initialization and mode setting, feature extraction, probabilistic modeling (GMM), final decision and model update. Through the collaboration of multiple modules, VAD can extract features from the original audio signal, calculate the probability of speech and noise, and make judgments on speech activity. The JavaScript-based VAD library implementation is that by obtaining the user's audio input stream and connecting it to the VAD processing node, the voice activity in the audio stream can be detected in real time. On the server side, VAD technology can be implemented through a dedicated audio processing server or cloud service. The service provides efficient audio processing capabilities and a scalable architecture to meet the needs of large-scale real-time communication applications. The server-side VAD technology can process audio streams from multiple clients and perform voice activity detection, noise suppression, etc. on them, and then send the processed audio data back to the client or for further analysis and storage.

[0042] Optionally, if the original audio stream is recorded by an app, a VAD check can be performed directly on the anchor side. When performing a VAD check, it is also possible to combine factors such as the specific live broadcast room type, echo cancellation, and ambient sound to determine whether there is sound, and mark the corresponding audio frame in the private audio packet protocol. The audio stream is then transmitted to the server. If the original audio stream is collected and generated by a streaming tool, the server can perform a VAD check and mark the corresponding sound.

[0043] Furthermore, for the server, the above audio stream will be pulled through the HLS stream generation service, and TS slices will be generated in real time. In order to improve the real-time performance of audio playback and reduce latency, 2-4 seconds of audio and video frames can be accumulated, and then TS slices, namely media slices, can be generated. The media slice contains multiple media segments divided according to time granularity, and each media segment is configured with corresponding sound ratio information. In the case where the media slice contains multiple media segments divided according to time granularity, the sound ratio information in the file name is sequentially spliced ​​according to the timing of each media segment.

[0044] When generating TS segments, the proportion of audio frames in the segment that are judged to have sound is counted at the second-level granularity, that is, the sound ratio information. The sound ratio information is expressed in thousandths (‰). For example, in a 3-second media segment, there are 42 audio frames in the first second, of which 10 frames have sound marks, then the ratio is 10 / 42=238‰. If the media segment duration is 3 seconds, there are 3 ratios in total.

[0045] Then, add the above per-thousandth information when naming each TS segment file. For example, assuming the HLS stream name is stream, the sequence number of the current TS segment is seq2, the duration is 3 seconds, and the three per-thousandths are 238‰, 45‰, and 0‰ respectively, then the final file name is "stream_seq2_238_45_0.TS", which is then concatenated into the network address URL of the media segment and stored in the M3U8 playlist file of the target audio stream. The URL is mainly used to access the TS media segment files on the HLS server.

[0046] It should be noted that, according to actual needs, the above media segments can be adaptively divided according to different time granularities, and each media segment is configured with a voice ratio information. It is also possible not to split the media segments, and directly configure a voice ratio information. According to the actual time granularity requirements of the visual identification, the media segments can be adaptively split.

[0047] Furthermore, based on the above-mentioned pre-configured network address, when the target audio stream is played later, it is necessary to introduce the HLS.js library in the browser and use it to play the HLS stream. By listening to the events provided by HLS.js (such as the HLS.EvenTS.FRAG_LOADED event), the media segment information being played can be obtained, so that the network address URL of the corresponding media segment can be found in the M3U8 playlist file according to the media segment information, for the collection of the voice ratio information. The browser side does not need to introduce additional protocols to transmit sound identifiers, reducing the occupancy of network bandwidth. And the voice ratio information is directly embedded in the network address of the media segment, which is convenient for the browser side to directly obtain and use.

[0048] S120: parse the network address to obtain voice ratio information, and obtain corresponding media segments based on the network address.

[0049] Furthermore, after obtaining the network address of the media segment, the browser can parse the audio ratio information contained in the network address URL to provide a visual identifier for subsequent audio playback. At the same time, the browser requests and downloads the corresponding media segment from the server according to the network address URL for audio playback.

[0050] It is understandable that the process of resolving the network address to obtain the voice ratio information is simple and efficient, and does not add additional processing burden. It also ensures the one-to-one correspondence between the media segments and the voice ratio information, providing an accurate data basis for subsequent visual identification.

[0051] Optionally, the file name of the media segment contained in the network address is parsed, and the audio ratio information is obtained from the file name. The file name is pre-created based on the audio ratio information and the segment sequence number of the media segment, and is spliced ​​into the network address.

[0052] When parsing the network address to obtain the audio ratio information, since the audio ratio is pre-added to the file name and then spliced ​​into the URL, the character string containing the network address of the media segment is parsed to identify the various components of the network address URL, including the protocol (such as HTTP, HTTPS), host name, port, path, and possible query parameters. Then, the file name of the media segment is extracted from the path part of the network address URL. The file name contains the audio ratio information and the segment sequence number. According to the preset naming rules, the audio ratio information of each media segment arranged in sequence can be parsed from this file name.

[0053] Optionally, the audio ratio information of the above media segments is not limited to being attached to the file name, but can also be carried in the HTTP request header or a location that can be parsed simultaneously when other web front-ends play the stream. This application does not impose fixed restrictions on the specific location where the audio ratio is stored in the network address.

[0054] S130: Play the media segment, and synchronously perform visual identification on the currently played media segment based on the corresponding sound ratio information.

[0055] Furthermore, while playing the media segment, the browser side visually identifies the currently playing media segment according to the sound ratio information. The visual identification can be realized by displaying different colors, icons or texts on the playback interface to distinguish the audio streams of different anchor terminals. In order to achieve synchronization between sound and visual identification, the display of visual identification can be dynamically adjusted based on the playback timestamp and sound ratio information of the media segment. For example, when the sound ratio is high, the brightness or contrast of the visual identification can be increased; when the sound ratio is low, it can be weakened accordingly.

[0056] The currently playing audio streaming media segment data is captured through the Web Audio API instance on the browser side, and passed to the AnalyserNode thread to extract real-time spectrum or time domain data. The extracted data can be used for subsequent visualization. Then, different sound visualization effects are drawn using the data provided by the AnalyserNode. <canvas>Draw a spectrogram or other audio visualization on the element.

[0057] Based on the above-mentioned visual identification method of browser audio playback, better browser compatibility can be provided, and complex audio decoding and VAD checks are not relied on in the browser interface implementation, which reduces the front-end user load, and the visual identification of audio playback can be flexibly controlled by the server. By adding sound tags to the audio frames in the HLS stream and displaying them in the UI on the front end, the sound can be visualized, allowing the audience to intuitively understand the sound situation in the host or audio stream. By performing VAD checks on the audio stream and marking it in the private audio packet protocol, auditors can quickly determine the source of the sound, thereby conducting compliance audits and ensuring the legality and standardization of the audio content.

[0058] Optionally, visually marking the currently played media segment synchronously based on the corresponding sound ratio information includes:

[0059] When the sound ratio information corresponding to the currently played media segment is lower than the set ratio threshold, the visualization icon is hidden. When the sound ratio information corresponding to the currently played media segment reaches the set ratio threshold, the visualization icon is displayed. The visualization icon indicates that there is sound output in the current target audio stream.

[0060] By reading the sound ratio information of the media segment, and deciding whether to display the visualization icon according to the set ratio threshold. Before this, set a sound ratio threshold. This threshold determines when the visualization icon should be displayed or hidden. Then, during the media playback process, regularly check the sound ratio information of the currently playing segment. Compare the sound ratio with the set threshold, and display or hide the visualization icon according to the comparison result.

[0061] Taking a media slice as an example, when the sound ratio of the current media segment of the media slice exceeds the sound ratio threshold, the speaker icon can be displayed for one second; when the sound ratio is lower than the sound ratio threshold, the speaker icon is hidden for one second.

[0062] Based on different sound ratio detection results, the browser can dynamically display a speaker icon or other sound visualization marks. In addition, the visibility and style of UI elements can be dynamically adjusted through HTML / CSS, such as highlighting the icon of an audio source to indicate that the audio stream currently has voice output.

[0063] For example, take the voice review of a multi-person voice room in a live broadcast scenario as an example. Figure 3 As shown, the auditor can visually identify the audio playback through the audio streams of each live broadcast terminal. It can be seen that the current "User 1" and "User 3" are currently displaying the speaker logo, while the anchor terminals of other users are not displayed (i.e. hidden), which means that the current sound comes from "User 1" and "User 3". In this way, the source of the sound can be clearly identified, and the compliance audit of the sound currently output by "User 1" and "User 3" can be performed.

[0064] Optionally, refer to Figure 4 , the auditor can also interact with the user through human-computer interaction, such as clicking the speaker icon of "User 3". In response to this interactive operation, the browser pauses the audio stream output of "User 3". At this time, the speaker icon of "User 3" is marked with an "X", indicating that the sound output of this user is turned off. The auditor can then only receive the sound emitted by "User 1", thereby conducting an accurate and precise sound compliance audit.

[0065] Optionally, the network address also pre-configures subtitle translation information for the media segment;

[0066] After obtaining the network addresses of the media segments to be played of the target audio stream, the following steps are also included:

[0067] The network address is parsed to obtain subtitle translation information, and the subtitle translation information is synchronously displayed while visually marking the currently playing media segment.

[0068] Among them, a URL parsing library (such as Python's urllib.parse) is used to parse the string containing the network address of the media segment. A naming convention or query parameter format is designed to embed the subtitle translation information into the network address. For example, a subtitles field can be added to the query parameter of the URL, and its value is an encoded string representing the translation information of the subtitle or a link to the subtitle file. For the subtitle translation information of the media segment, the real-time semantic content can be obtained by performing semantic analysis on the media segment, and then the subtitle translation information can be obtained.

[0069] When playing the media segment later, the subtitle information is extracted from the URL and decoded to obtain the actual subtitle text or subtitle file path. During the playback of the media segment, the subtitle text is loaded according to the decoded subtitle information and rendered to the appropriate position of the media player. This can be achieved by drawing text on the video frame or using the subtitle display function provided by the media player.

[0070] By pre-configuring the subtitle translation information of the media segment in the network address and synchronously displaying this information and visualization icons when playing the media segment, the auditor's sound review experience can be significantly improved and the auditor's review efficiency can be improved.

[0071] In the above, when playing the target audio stream, the network address of each media segment to be played of the target audio stream is obtained, and the network address is pre-added with the sound ratio information of the corresponding media segment, and the sound ratio information represents the ratio of the audio frame in the corresponding media segment that has the sound mark; the network address is parsed to obtain the sound ratio information, and the corresponding media segment is obtained based on the network address; the media segment is played, and the currently played media segment is synchronously visually identified based on the corresponding sound ratio information. By adopting the above technical means, by adding the sound ratio information to the network address of the target audio streaming media segment, the real-time played media segment is synchronously visually identified according to the sound ratio information obtained by parsing the network address. The visual identification process of audio stream playback can be simplified, the visual identification efficiency of audio playback can be improved, and the browser compatibility can be improved, the situation where the extra protocol transmission of sound identification causes bandwidth occupation can be avoided, and the visual identification effect of audio playback can be improved.

[0072] Based on the above embodiments, Figure 5 A schematic diagram of the structure of a visual identification system for browser audio playback provided by this application. Figure 5 The browser audio playback visualization identification system provided in this embodiment specifically includes: an acquisition module 21, a parsing module 22 and a visualization module 23.

[0073] The acquisition module 21 is configured to acquire the network address of each to-be-played media segment of the target audio stream when playing the target audio stream, wherein the network address is pre-added with the voice ratio information of the corresponding media segment, and the voice ratio information represents the ratio of the audio frame in the corresponding media segment that has the sound mark;

[0074] The parsing module 22 is configured to parse the network address to obtain the voice ratio information, and obtain the corresponding media segment based on the network address;

[0075] The visualization module 23 is configured to play the media segments and synchronously perform visualization marking on the currently played media segments based on the corresponding sound ratio information.

[0076] Specifically, the media segment includes a plurality of media segments divided according to time granularity, and each media segment is configured with corresponding voice ratio information;

[0077] In the case where the media slice contains multiple media segments divided according to time granularity, the voice ratio information in the file name is sequentially spliced ​​according to the time sequence of each media segment.

[0078] The voice ratio information is determined according to the ratio of the number of audio frames containing the sound marker to the total number of audio frames in the media segment, and the sound marker is marked based on the signal strength detection result of the audio frame.

[0079] Specifically, parsing the network address to obtain the voice ratio information includes:

[0080] The file name of the media segment contained in the network address is parsed, and the audio ratio information is obtained from the file name. The file name is pre-created based on the audio ratio information and the segment sequence number of the media segment, and is spliced ​​into the network address.

[0081] Specifically, the currently played media segment is visually marked synchronously based on the corresponding sound ratio information, including:

[0082] When the sound ratio information corresponding to the currently played media segment is lower than the set ratio threshold, the visualization icon is hidden. When the sound ratio information corresponding to the currently played media segment reaches the set ratio threshold, the visualization icon is displayed. The visualization icon indicates that there is sound output in the current target audio stream.

[0083] In the above, when playing the target audio stream, the network address of each media segment to be played of the target audio stream is obtained, and the network address is pre-added with the sound ratio information of the corresponding media segment, and the sound ratio information represents the ratio of the audio frame in the corresponding media segment that has the sound mark; the network address is parsed to obtain the sound ratio information, and the corresponding media segment is obtained based on the network address; the media segment is played, and the currently played media segment is synchronously visually identified based on the corresponding sound ratio information. By adopting the above technical means, by adding the sound ratio information to the network address of the target audio streaming media segment, the real-time played media segment is synchronously visually identified according to the sound ratio information obtained by parsing the network address. The visual identification process of audio stream playback can be simplified, the visual identification efficiency of audio playback can be improved, and the browser compatibility can be improved, the situation where the extra protocol transmission of sound identification causes bandwidth occupation can be avoided, and the visual identification effect of audio playback can be improved.

[0084] The visual identification system for browser audio playback provided in the embodiment of the present application can be configured to execute the visual identification method for browser audio playback provided in the above embodiment, and has corresponding functions and beneficial effects.

[0085] Based on the above practical example, the present application embodiment also provides a visual identification device for browser audio playback, referring to Figure 6 , the visual identification device for browser audio playback includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The memory, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the visual identification method for browser audio playback described in any embodiment of the present application (for example, the acquisition module, parsing module and visualization module in the visual identification system for browser audio playback). The communication module is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, that is, realizes the above-mentioned visual identification method for browser audio playback. The input device can be configured to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device may include a display device such as a display screen. The above-mentioned visual identification device for browser audio playback can be configured to execute the visual identification method for browser audio playback provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0086] On the basis of the above embodiments, the embodiments of the present application further provide a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are configured to execute a method for visually identifying audio playback of a browser when executed by a computer processor, and the storage medium may be any of various types of memory devices or storage devices. Of course, the computer-executable instructions of the non-volatile computer-readable storage medium provided in the embodiments of the present application are not limited to the method for visually identifying audio playback of a browser as described above, and may also execute related operations in the method for visually identifying audio playback of a browser provided in any embodiment of the present application.

[0087] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the visual identification method for browser audio playback described in each embodiment of the present application.< / canvas>

Claims

1. A visual identification method for browser audio playback, characterized in that: include: In the case of playing a target audio stream, obtaining a network address of each to-be-played media segment of the target audio stream, wherein the network address is pre-added with voice ratio information of the corresponding media segment, wherein the voice ratio information represents a ratio of audio frames in the corresponding media segment having a sound marker; Parsing the network address to obtain the voice ratio information, and obtaining the corresponding media segment based on the network address; The media segment is played, and visual identification is synchronously performed on the media segment currently being played based on the corresponding sound ratio information.

2. The method for visually identifying browser audio playback according to claim 1, characterized in that: The media slices include a plurality of media segments divided according to time granularity, and each of the media segments is configured with corresponding voice ratio information; The synchronously visually marking the currently played media segment based on the corresponding sound ratio information includes: When the sound proportion information corresponding to the media segment currently being played is lower than a set proportion threshold, the visualization icon is hidden; when the sound proportion information corresponding to the media segment currently being played reaches the set proportion threshold, the visualization icon is displayed, and the visualization icon indicates that there is sound output in the current target audio stream.

3. The visual identification method for browser audio playback according to claim 2, characterized in that: The step of parsing the network address to obtain the voice ratio information includes: Parse the file name of the media segment contained in the network address, and obtain the voice ratio information from the file name. The file name is pre-created based on the voice ratio information and the segment sequence number of the media segment, and is spliced ​​into the network address.

4. The method for visually identifying audio playback of a browser according to claim 3, characterized in that: In the case that the media slice includes a plurality of media segments divided according to time granularity, each of the voice ratio information in the file name is sequentially spliced ​​according to the time sequence of each of the media segments.

5. The method for visually identifying the audio playback of a browser according to claim 3, characterized in that: The voiced ratio information is determined according to a ratio of the number of audio frames containing a sound mark to the total number of audio frames in the media segment, and the sound mark is marked based on a signal strength detection result of the audio frame.

6. The method for visually identifying audio playback of a browser according to claim 1, characterized in that: The network address also pre-configures subtitle translation information of the media segment; After obtaining the network addresses of the media segments to be played of the target audio stream, the method further includes: The network address is parsed to obtain the subtitle translation information, and the subtitle translation information is synchronously displayed while visually marking the currently played media segment.

7. A visual identification system for browser audio playback, characterized in that: include: an acquisition module configured to acquire, when playing a target audio stream, a network address of each to-be-played media segment of the target audio stream, wherein the network address is pre-added with voice ratio information of the corresponding media segment, wherein the voice ratio information represents a ratio of audio frames in the corresponding media segment that contain a sound marker; A parsing module, configured to parse the network address to obtain the voice ratio information, and obtain the corresponding media segment based on the network address; The visualization module is configured to play the media segment and synchronously perform visual identification on the currently played media segment based on the corresponding sound ratio information.

8. A visual identification device for browser audio playback, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the visual identification method for browser audio playback as described in any one of claims 1-6.

9. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by a computer processor, are configured to execute the visual identification method for browser audio playback as described in any one of claims 1-6.

10. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the method for visually identifying the audio playback of a browser as described in any one of claims 1-6.

Images