Live broadcast information processing method, apparatus and device, and computer readable storage medium

By using voice feature matching in e-commerce live broadcasts to determine the target recommended object and play its content in the live broadcast interface, the problem that viewers cannot easily view the recommended object introduced is solved, and the user experience and live broadcast room conversion rate is improved.

CN118945387APending Publication Date: 2024-11-12TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410936931.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2020-08-17
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In e-commerce live broadcasts, viewers cannot conveniently view the live broadcast content of recommended targets that the anchor has introduced. They need to jump out of the live broadcast room to view. The operation is cumbersome and the user experience is poor.

Method used

Provide a live broadcast information processing method, by presenting the playback function items of the target recommendation object in the live broadcast content interface, and determining the target recommendation object using voice feature matching, realizing the live broadcast content of the target recommendation object in the live broadcast interface, hiding the current live broadcast content , use the live broadcast window to play the content of the target recommended object.

Benefits of technology

This enables viewers to easily view the content of the target recommended targets that have been broadcast without leaving the live broadcast room, simplifying operations, improving user experience, and improving the conversion rate of the live broadcast room.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945387A_ABST
    Figure CN118945387A_ABST
Patent Text Reader

Abstract

The invention provides a live broadcast information processing method and device, equipment and a computer readable storage medium. The method comprises the following steps: presenting a live broadcast content interface, wherein the live broadcast content is used for recommending at least two recommendation objects; in the content interface, a playback function item corresponding to a target recommendation object in the at least two recommendation objects is presented, and the playback function item is used for playing live broadcast content corresponding to the target recommendation object; wherein the target recommendation object is a recommendation object of which the corresponding live broadcast content is before the current live broadcast content; and when a trigger operation for the playback function item is received, playing the live broadcast content corresponding to the target recommendation object. Through application of the method and the device, the user can conveniently view the live broadcast content of the recommended object before the current live broadcast content while watching the current live broadcast content.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Description of the case

[0002] This application is a divisional application of a Chinese patent with an application date of August 17, 2020, application number 202010825173.2, and invention name “Live broadcast information processing method, device, equipment and computer-readable storage medium”. Technical Field

[0003] The present application relates to the field of live video broadcasting, and in particular to a method, device, equipment and computer-readable storage medium for processing live broadcast information. Background Art

[0004] Live streaming is a technology that reaches a large number of users in real time in the form of audio and video. With the rapid development of the e-commerce live streaming industry, more and more users are joining the live streaming army. In an e-commerce live streaming room, the anchor often recommends a variety of recommended objects (such as clothes, food and other commodities). For users watching the live streaming, they can watch the commodities that the anchor is introducing in the live streaming room, but for the target recommended objects that the anchor has introduced, they can only view the related shopping links in the live streaming room, and cannot directly view the live streaming content of the anchor's recommended objects. If the audience wants to view the related live streaming content, they need to jump out of the live streaming room, and then return to the live streaming room after watching it to continue watching the live streaming content being broadcast. The operation is cumbersome and the user experience is poor. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, device and computer-readable storage medium for processing live broadcast information, which can allow users to conveniently view the live broadcast content of recommended objects before the current live broadcast content.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present application provides a method for processing live broadcast information, including:

[0008] Presenting a live broadcast content interface, wherein the live broadcast content is used to recommend at least two recommendation objects;

[0009] In the content interface, a playback function item corresponding to a target recommended object among the at least two recommended objects is presented, wherein the playback function item is used to play the live content corresponding to the target recommended object;

[0010] The target recommended object is a recommended object whose corresponding live broadcast content is before the current live broadcast content;

[0011] When a trigger operation for the playback function item is received, the live content corresponding to the target recommendation object is played in the content interface.

[0012] In the above solution, the playing of the live broadcast content corresponding to the target recommendation object includes:

[0013] The live broadcast content of the live broadcast is hidden, and the live broadcast content corresponding to the target recommendation object is played in the live broadcast window.

[0014] In the above scheme, the method further includes:

[0015] Extracting voice features of the live broadcast content in real time;

[0016] Matching the voice feature with the reference voice feature of each of the recommended objects respectively, and determining the recommended object to which the voice feature belongs based on the matching result;

[0017] Determining a similarity value between the voice feature and a reference voice feature of a recommended object to which the voice feature belongs;

[0018] When it is determined based on the similarity value that the live broadcast end time of the recommended object to which the voice feature belongs has arrived, the recommended object to which the voice feature belongs is used as the target recommended object.

[0019] In the above scheme, the method further includes:

[0020] Obtaining the live broadcast start time of the target recommendation object;

[0021] The content of the live broadcast between the start time of the live broadcast and the end time of the live broadcast is intercepted as the live broadcast content of the target recommendation object.

[0022] In the above solution, the step of determining the recommended object to which the voice feature belongs based on the matching result includes:

[0023] When the matching result indicates that the similarity between the voice feature and the reference voice feature of the first recommended object among the at least two recommended objects reaches a first similarity threshold, and

[0024] When the similarity between the voice feature and the reference voice feature of the other recommended object among the at least two recommended objects does not reach the first similarity threshold, the recommended object to which the voice feature belongs is determined to be the first recommended object.

[0025] In the above solution, the real-time acquisition of the voice features of the live broadcast content includes:

[0026] Performing voice recognition on the live broadcast content of the live broadcast to obtain a voice signal corresponding to the live broadcast content;

[0027] Performing speech boundary detection on the speech signal to obtain corresponding silent audio frames, and filtering the silent audio frames;

[0028] Feature extraction is performed on the filtered voice signal to obtain voice features corresponding to the live broadcast content of the live broadcast.

[0029] The present application provides a device for processing live broadcast information, including:

[0030] A first presentation module, used to present a live content interface, wherein the live content of the live broadcast is used to recommend at least two recommendation objects;

[0031] A second presentation module, configured to present, in the content interface, a playback function item corresponding to a target recommended object among the at least two recommended objects, wherein the playback function item is configured to play the live content corresponding to the target recommended object;

[0032] The target recommended object is a recommended object whose corresponding live broadcast content is before the current live broadcast content;

[0033] The playing module is used to play the live content corresponding to the target recommendation object when receiving a trigger operation for the playback function item.

[0034] In the above solution, when the number of the target recommendation objects is at least two, the second presentation module is further used to present the playback function items corresponding to the target recommendation objects in parallel in the content interface; or

[0035] In the content interface, playback function items corresponding to each target recommendation object are scrolled and presented.

[0036] In the above scheme, when the number of the target recommendation objects is at least two, the second presentation module is further used to respectively obtain the live broadcast time corresponding to the live broadcast content of each of the target recommendation objects, and the live broadcast time is the live broadcast start time or the live broadcast end time;

[0037] In the target area of ​​the content interface, the playback function items corresponding to each target recommended object are presented in a manner such that the live broadcast time is in front and the playback function items corresponding to the corresponding target recommended object are in front.

[0038] In the above solution, when the number of the target recommendation objects is at least two, the second presentation module is further used to obtain the interaction heat corresponding to the live broadcast content of each of the target recommendation objects respectively;

[0039] In the target area of ​​the content interface, the playback function items corresponding to the target recommended objects are presented in a manner such that the playback function items corresponding to the target recommended objects with high interaction popularity are placed in front.

[0040] In the above scheme, when the number of the target recommendation objects is at least two, the playback module is further used to present a selection interface for playing and selecting the at least two target recommendation objects in the content interface;

[0041] In response to a selection operation on a target recommendation object triggered based on the selection interface, live content corresponding to the selected target recommendation object is played.

[0042] In the above solution, the device further includes a third presentation module, the third presentation module being used to present, in the content interface, preview information corresponding to at least one preview recommendation object, the preview information being used to preview the live broadcast start time corresponding to the preview recommendation object;

[0043] The preview recommended object is a recommended object whose corresponding live broadcast content is after the current live broadcast content.

[0044] In the above scheme, the playback module is also used to play the live content corresponding to the target recommendation object through a sub-interface independent of the content interface.

[0045] In the above solution, the playing module is also used to hide the live broadcast content of the live broadcast, and use the live broadcast window to play the live broadcast content corresponding to the target recommendation object.

[0046] In the above solution, the playing module is further used to stop playing the live content corresponding to the target recommended object in response to a trigger operation on the target area during the playing of the live content corresponding to the target recommended object;

[0047] The target area is an area in the current content interface where a playback interface of the live content corresponding to the target recommendation object is located.

[0048] In the above solution, the device further includes a determination module, the determination module is used to

[0049] Extracting voice features of the live broadcast content in real time;

[0050] Matching the voice feature with the reference voice feature of each of the recommended objects respectively, and determining the recommended object to which the voice feature belongs based on the matching result;

[0051] Determining a similarity value between the voice feature and a reference voice feature of a recommended object to which the voice feature belongs;

[0052] When it is determined based on the similarity value that the live broadcast end time of the recommended object to which the voice feature belongs has arrived, the recommended object to which the voice feature belongs is used as the target recommended object.

[0053] In the above solution, the determination module is further used to obtain the live broadcast start time of the target recommendation object;

[0054] The content of the live broadcast between the start time of the live broadcast and the end time of the live broadcast is intercepted as the live broadcast content of the target recommendation object.

[0055] In the above solution, the determining module is further configured to: when the matching result indicates that the similarity between the voice feature and the reference voice feature of the first recommended object among the at least two recommended objects reaches a first similarity threshold, and

[0056] When the similarity between the voice feature and the reference voice feature of the other recommended object among the at least two recommended objects does not reach the first similarity threshold, the recommended object to which the voice feature belongs is determined to be the first recommended object.

[0057] In the above solution, the determining module is further configured to, before presenting the playback function item corresponding to the target recommended object among the at least two recommended objects,

[0058] For each time point of the live broadcast, obtaining the voice features of the live broadcast content of the live broadcast in real time;

[0059] Matching the voice feature with reference voice features of each of the recommended objects;

[0060] When it is determined based on the matching result that the voice feature belongs to the target recommendation object, a similarity value between the voice feature and a reference voice feature of the target recommendation object is determined in real time;

[0061] Taking the time point when the similarity value exceeds the similarity threshold as the starting point, start collecting the live broadcast content of the live broadcast to determine the live broadcast content corresponding to the target recommendation object.

[0062] In the above solution, the determining module is further configured to use the time point when the similarity value exceeds the first similarity threshold as the starting point, and the time point when the similarity value is lower than the second similarity threshold as the end point, and

[0063] Start collecting the live broadcast content of the live broadcast from the starting point and stop collecting at the end point;

[0064] The collected live broadcast content between the starting point and the end point is determined as the live broadcast content corresponding to the target recommendation object.

[0065] In the above solution, the determination module is further used to obtain the score of the live broadcast content corresponding to each time point between the starting point and the end point respectively;

[0066] Filter the live broadcast content corresponding to the scores below the score threshold;

[0067] The obtained filtered live broadcast content is determined as the live broadcast content corresponding to the target recommendation object.

[0068] In the above solution, the determination module is further used to continuously collect the live broadcast content of the live broadcast starting from the time point when the similarity value exceeds the similarity threshold, and stop collecting when the similarity value does not exceed the similarity threshold;

[0069] As the live broadcast proceeds, the live broadcast content of the live broadcast is continuously collected with the time point when the similarity value exceeds the similarity threshold again as a new starting point, and the collection is stopped when the accumulated live broadcast content meets the duration condition or when the similarity value does not exceed the similarity threshold again;

[0070] The collected real-time live broadcast content is determined as the live broadcast content corresponding to the target recommendation object.

[0071] In the above solution, the determination module is further used to perform voice recognition on the live broadcast content of the live broadcast to obtain a voice signal corresponding to the live broadcast content;

[0072] Performing speech boundary detection on the speech signal to obtain corresponding silent audio frames, and filtering the silent audio frames;

[0073] Feature extraction is performed on the filtered voice signal to obtain voice features corresponding to the live broadcast content of the live broadcast.

[0074] An embodiment of the present application provides an electronic device, including:

[0075] A memory for storing executable instructions;

[0076] The processor is used to implement the live broadcast information processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0077] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute instructions to implement the method for processing live broadcast information provided in the embodiment of the present application.

[0078] The embodiments of the present application have the following beneficial effects:

[0079] By triggering the playback function item corresponding to the target recommended object in the live content interface, the live content corresponding to the target recommended object is played. Since the target recommended object is the recommended object whose live content is before the current live content, that is, the recommended object that the anchor has already broadcast, when the audience triggers the playback function item for the target recommended object, what is presented in the live interface is the corresponding live content, not just the corresponding shopping link. This allows the audience to conveniently view the live content of the target recommended object that has been broadcast without leaving the live room. The operation is simple, which improves the user experience, and thus helps to improve the conversion rate of the recommended objects in the live room. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1A-1B A schematic diagram of a live broadcast interface provided in an embodiment of the present application;

[0081] Figure 2 A schematic diagram of an optional architecture of a system for processing live broadcast information provided in an embodiment of the present application;

[0082] Figure 3 An optional structural diagram of an electronic device provided in an embodiment of the present application;

[0083] Figure 4 An optional flowchart of a method for processing live broadcast information provided in an embodiment of the present application;

[0084] Figure 5A-5B A schematic diagram of a display interface provided in an embodiment of the present application;

[0085] Figures 6A-6D A schematic diagram of a display interface provided in an embodiment of the present application;

[0086] Figure 7 An optional flowchart of a method for processing live broadcast information provided in an embodiment of the present application;

[0087] Figure 8 An optional flowchart of a method for processing live broadcast information provided in an embodiment of the present application

[0088] Fig. 9 A schematic diagram of a voice signal provided in an embodiment of the present application;

[0089] Fig.10 A schematic diagram of the structure of a device for processing live broadcast information provided in an embodiment of the present application. DETAILED DESCRIPTION

[0090] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0091] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0092] In the following description, the terms "first\second..." are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second..." can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0093] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0094] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0095] 1) Live streaming with goods: refers to a new service method that uses live streaming technology to display products online, answer questions, and guide sales through some Internet platforms. The specific form can be that the store opens its own live streaming room, or a full-time anchor opens a live streaming room in the store or on the live streaming platform to promote the products.

[0096] 2) Client: an application running in the terminal to provide various services, such as video playback client, live broadcast client, education client, etc.

[0097] 3) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0098] See also Figure 1A-1B , Figure 1A-1B A schematic diagram of the live broadcast interface provided in the embodiment of the present application, such as Figure 1AAs shown in the figure, during the live broadcast, the user can enter the live broadcast room to see the live broadcast content of the recommended object being broadcast by the anchor. When the user wants to view the live broadcast content of the target recommended object introduced by the anchor, he can trigger "see details". In response to the trigger operation, the terminal jumps from the current live broadcast interface to the following figure Figure 1B The playback interface shown in the figure plays the live content of the target promotion content. When the user wants to watch the live content of other recommended objects in the live broadcast room, the function item of "return to live broadcast" can be triggered. In response to the trigger operation, the terminal jumps to the live broadcast interface of the live broadcast room. It can be seen that when viewing the live content of the recommended object that has been introduced, it is necessary to jump out of the live broadcast room. The user cannot watch the live content of the recommended object that has been introduced while watching the current live content.

[0099] In view of this, the embodiments of the present application provide a method, apparatus, device and computer-readable storage medium for processing live broadcast information, so that users can conveniently view the live broadcast content of recommended objects before the current live broadcast content while watching the current live broadcast content.

[0100] See also Figure 2 , Figure 2 An optional architectural diagram of a live information processing system 100 provided in an embodiment of the present application. To support an exemplary application, a terminal 400 (terminal 400-1 and terminal 400-2 are shown as examples) is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two, and a wireless link is used to achieve data transmission.

[0101] In actual applications, the terminal 400 can be various types of user terminals such as a smart phone, a tablet computer, a laptop computer, etc., and can also be a desktop computer, a game console, a television, or a combination of any two or more of these data processing devices; the server 200 can be a separately configured server supporting various services, or it can be configured as a server cluster, or it can be a cloud server, etc.

[0102] In actual implementation, the terminal 400 is provided with a client, such as a video playback client, an instant messaging client, a live broadcast client, etc. When the user opens the client on the terminal 400 for live broadcast to recommend at least two recommendation objects, the terminal 400 is used to present the content interface of the live broadcast, and in the content interface, presents a playback function item corresponding to the target recommendation object among the at least two recommendation objects, wherein the playback function item is used to realize the playback of the live content corresponding to the target recommendation object, and the target recommendation object is a recommendation object whose corresponding live content is before the current live content; when a trigger operation for the playback function item is received, an information acquisition request for obtaining the live content corresponding to the target recommendation object is sent to the server 200; the server 200 is used to obtain and return the live content corresponding to the target recommendation object to the terminal 400 based on the message acquisition request; the terminal 400 is also used to display the live content corresponding to the target recommendation object in the live content interface.

[0103] See also Figure 3 , Figure 3 This is an optional structural diagram of an electronic device 500 provided in an embodiment of the present application. In practical applications, the electronic device 500 may be Figure 2 The terminal 400 or the server 200 in the embodiment of the present invention is an electronic device. Figure 2 Taking the terminal 400 shown as an example, an electronic device for implementing the method for processing live broadcast information in an embodiment of the present application is described. Figure 3 The electronic device 500 shown includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 540 is not described in detail. Figure 3 Various buses are labeled as bus system 540 .

[0104] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0105] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0106] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices that are physically remote from the processor 510.

[0107] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0108] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0109] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0110] A network communication module 552, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0111] a presentation module 553 for enabling presentation of information via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with the user interface 530 (e.g., a user interface for operating peripherals and displaying content and information);

[0112] The input processing module 554 is used to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions.

[0113] In some embodiments, the live broadcast information processing device provided in the embodiments of the present application can be implemented in a software manner. Figure 3 A processing device 555 for live broadcast information stored in a memory 550 is shown, which may be software in the form of a program or plug-in, and includes the following software modules: a first presentation module 5551, a second presentation module 5552, and a playback module 5553. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented.

[0114] The functions of each module will be described below.

[0115] In other embodiments, the live broadcast information processing device provided in the embodiments of the present application can be implemented in hardware. As an example, the live broadcast information processing device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the live broadcast information processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0116] Next, the method for processing the live broadcast information provided in the embodiment of the present application is described. In actual implementation, the method for processing the live broadcast information provided in the embodiment of the present application can be implemented by the server or the terminal alone, or can be implemented by the server and the terminal in collaboration.

[0117] See also Figure 4 , Figure 4 An optional flow chart of a method for processing live broadcast information provided in an embodiment of the present application is combined with Figure 4 The steps shown are explained.

[0118] Step 101: The terminal presents a live content interface, and the live content of the live broadcast is used to recommend at least two recommendation objects.

[0119] Here, when the user opens the terminal and enters the live broadcast room to watch the live broadcast, the live broadcast screen of the recommended object being recommended by the anchor is presented in the live broadcast content interface. Usually, the anchor can introduce multiple recommended objects to the audience through a live broadcast, such as a teacher (i.e., the anchor) explaining multiple test questions (i.e., the recommended objects) to students (i.e., the audience) through live broadcast, or the anchor recommends recommended objects such as clothes and food to the audience through live broadcast and calls on users to place orders to buy, etc.

[0120] Step 102: In the content interface, present a playback function item corresponding to a target recommended object among at least two recommended objects.

[0121] Among them, the playback function item is used to realize the live broadcast content corresponding to the target recommended object; the target recommended object is the recommended object whose corresponding live broadcast content is before the current live broadcast content, or the recommended object whose live broadcast end time corresponding to the corresponding live broadcast content is before the current time, that is, the target recommended object is the recommended object that the anchor has introduced and recommended.

[0122] In some embodiments, when the number of target recommended objects is at least two, the terminal may present the playback function item corresponding to the target recommended object among the at least two recommended objects in the following manner:

[0123] In the content interface, playback function items corresponding to each target recommendation object are presented in parallel; or, in the content interface, playback function items corresponding to each target recommendation object are presented in a scrolling manner.

[0124] Here, when there are multiple target recommendation objects, that is, the host has introduced multiple recommendation objects before the current moment, the playback function items for each target recommendation object are presented side by side or scrolling in the live content interface, so that the audience can intuitively view which recommendation objects have been recommended in the live broadcast.

[0125] See also Figure 5A-5B , Figure 5A-5B A schematic diagram of a display interface provided in an embodiment of the present application, Figure 5A In the live content interface A1, the live content of the recommended object A2 that the anchor is currently broadcasting is presented, and the playback function item A3 and the playback function item A4 are presented in parallel; Figure 5B In the live content interface B1, what is presented is the live content of the recommended object B2 that the host is currently broadcasting, and a plurality of playback function items B3 of the recommended objects that the host has recommended are scrolled and presented.

[0126] In some embodiments, when the number of target recommended objects is at least two, the terminal may present the playback function item corresponding to the target recommended object among the at least two recommended objects in the following manner:

[0127] The live broadcast time corresponding to the live broadcast content of each target recommendation object is obtained respectively, and the playback function items corresponding to each target recommendation object are presented in the target area of ​​the content interface in the manner of the live broadcast time first and the playback function items corresponding to the corresponding target recommendation object first. The live broadcast time is the live broadcast start time or the live broadcast end time;

[0128] Here, when the number of target recommendation objects is at least two, the corresponding playback function items can be presented in sequence in the content interface according to the order of the live broadcast time of each target recommendation object by the anchor. Figure 5A In the content interface, the live broadcast time of the target recommended object corresponding to the playback function item A3 is earlier than the live broadcast time of the target recommended object corresponding to the playback function item A4. Therefore, in the content interface, the playback function item A3 is located in front of the playback function item A4.

[0129] In some embodiments, when the number of target recommended objects is at least two, the terminal may present the playback function item corresponding to the target recommended object among the at least two recommended objects in the following manner:

[0130] The interactive heat corresponding to the live content of each target recommendation object is obtained respectively; in the target area of ​​the content interface, the playback function items corresponding to each target recommendation object are presented in a manner that the playback function items corresponding to the target recommendation object with high interactive heat are in front. The interactive methods include at least one of the following: commenting, rewarding, shopping, and sharing.

[0131] Here, when the number of target recommendation objects is at least two, the corresponding playback function items can be presented in the content interface in order of the interaction heat levels corresponding to the target recommendation objects from high to low, that is, the playback function items corresponding to the target recommendation objects with higher heat levels are presented first, so that the audience who enters the live broadcast room later can view the corresponding target recommendation objects at the first time, which is conducive to further improving the live broadcast conversion rate. For example, Figure 5A In the content interface, the shopping order volume of the target recommendation object corresponding to the playback function item A3 is higher than the shopping order volume of the target recommendation object corresponding to the playback function item A4. Therefore, in the content interface, the playback function item A3 is located in front of the playback function item A4.

[0132] Step 103: When a trigger operation for the playback function item is received, the live content corresponding to the target recommendation object is played.

[0133] In some embodiments, when the number of target recommendation objects is at least two, the terminal may play the live broadcast content corresponding to the target recommendation objects in the following manner:

[0134] In the content interface, a selection interface for selecting at least two target recommendation objects for playback is presented; in response to a selection operation for a target recommendation object triggered based on the selection interface, live content corresponding to the selected target recommendation object is played.

[0135] Here, when the number of target recommendation objects is at least two and the audience triggers the playback function item, a selection interface is presented in the current live content interface. The selection interface contains multiple target recommendation objects, and the audience can select one of the target recommendation objects. The terminal responds to the selection operation and plays the live content of the selected target recommendation object for the audience to view.

[0136] See also Figures 6A-6C , Figures 6A-6C A schematic diagram of a display interface provided in an embodiment of the present application, Fig. 6A In the live content interface A1, the live content of the recommended object A2 currently being broadcast by the anchor is presented, and a playback function item A3 of "product explained" is presented. When the user triggers the playback function item A3, the following is presented: Figure 6B The selection interface B3 shown in FIG. 1 shows a plurality of target recommended objects B4 and target recommended objects B5. When the user triggers the target recommended object B4, the following is shown. Figure 6C The playing interface C3 shown in the figure plays the live content corresponding to the target recommendation object B4 in the playing interface C3.

[0137] In some embodiments, the terminal may play the live broadcast content corresponding to the target recommendation object in the following manner:

[0138] The live broadcast content corresponding to the target recommendation object is played through a sub-interface independent of the content interface.

[0139] Here, the live broadcast content of the recommended object recommended by the current live broadcast is presented in the content interface, and the sub-interface is a floating window floating on the content interface. The live broadcast content of the target recommended object that has been recommended is played on the sub-interface. In some embodiments, the sub-interface may be an interface with a preset transparency. In this way, while watching the live broadcast content presented by the current content interface C1, the live broadcast content of the target recommended object can also be watched through the sub-interface. The audience can obtain more information at the same time, thereby improving the viewing experience.

[0140] In other embodiments, when the live content of the target recommended object is played in the sub-interface, the live content of the recommended object being broadcast live in the content interface may also be subjected to noise reduction processing to output only the sound corresponding to the target recommended object.

[0141] For example, when a user triggers Figure 6B When the target recommendation object B4 is shown, Figure 6CIn the example, the sub-interface C3 and the content interface C1 are independent of each other. The live content corresponding to the target recommended object B4 is played through the sub-interface C3, and the live content of the recommended object C2 currently being broadcast is played through the content interface C1.

[0142] In some embodiments, the terminal may play the live broadcast content corresponding to the target recommendation object in the following manner:

[0143] The live broadcast content is hidden, and the live broadcast window is used to play the live broadcast content corresponding to the target recommendation object.

[0144] In some embodiments, the terminal may cancel the live broadcast content of the target recommendation object by:

[0145] During the process of playing the live content corresponding to the target recommended object, in response to a trigger operation on a target area, stop playing the live content corresponding to the target recommended object; wherein the target area is an area in the current content interface that is different from the playback interface of the live content corresponding to the target recommended object.

[0146] Here, when the viewer triggers any area other than the area where the playback interface of the live content corresponding to the target recommended object is located in the content interface, the terminal responds to the trigger operation and triggers an information management instruction for canceling the presentation of the live content corresponding to the target recommended object. The terminal responds to the information management instruction and cancels the playback of the live content corresponding to the target recommended object. Figure 6C When the audience triggers other areas on the content interface C1 except the sub-interface C3 where the live content corresponding to the target recommendation object is located, the sub-interface C3 disappears.

[0147] In some embodiments, the playback interface of the live content corresponding to the target recommended object may also present a cancel function item for canceling the playback of the live content corresponding to the target recommended object. When the target user triggers the cancel function item, an information management instruction for the live content corresponding to the target recommended object is presented. The terminal responds to the information management instruction and cancels the playback of the live content corresponding to the target recommended object. Figure 6C In the example, a cancel function item C4 is presented in the sub-interface C3 where the live broadcast content corresponding to the target recommendation object is located. When the target user triggers the cancel function item C4, the sub-interface C3 disappears.

[0148] In some embodiments, in the content interface, preview information corresponding to at least one preview recommendation object can also be presented, wherein the preview information is used to preview the live broadcast start time corresponding to the preview recommendation object, and the preview recommendation object is a recommended object whose corresponding live broadcast content is after the current live broadcast content.

[0149] For example, Figure 6BIn the selection interface B3, preview information B6 and B7 are presented to preview the recommended objects that are about to be broadcast live, so that new viewers can intuitively understand the live broadcast progress of the recommended objects in the current live broadcast room, so that users can have a psychological expectation, thereby improving the shopping conversion rate of the live broadcast room.

[0150] When the user triggers Figure 6B When selecting the currently live recommended object A2 in the selection interface or viewing more function item B8 in the selection interface B3, the following is displayed: Fig.6D The details page D1 shown, Fig.6D A schematic diagram of a display interface provided in an embodiment of the present application, in Fig.6D The details page D1 in the video presents the recommended objects that are being broadcast live, the recommended objects that have been broadcast live, and the recommended objects that have not been broadcast live. In order to distinguish the recommended objects that have been broadcast live and those that have not been broadcast live, the details page can be separated to facilitate the audience to intuitively see the live broadcast progress when purchasing or checking the progress of the anchor.

[0151] In some embodiments, before presenting the playback function item corresponding to the target recommended object among the at least two recommended objects, the terminal may also determine the live broadcast content corresponding to the target recommended object in the following manner:

[0152] At each time point of the live broadcast, the voice features of the live broadcast content are obtained in real time; the voice features are matched with the reference voice features of each recommendation object; when it is determined that the voice features belong to the target recommendation object based on the matching result, the similarity value between the voice features and the reference voice features of the target recommendation object is determined in real time; starting from the time point when the similarity value exceeds the similarity threshold, the live broadcast content is collected to determine the live content corresponding to the target recommendation object.

[0153] In some embodiments, the terminal may also obtain the voice features of the live broadcast content in real time in the following manner:

[0154] Perform speech recognition on the live broadcast content to obtain the speech signal corresponding to the live broadcast content; perform speech boundary detection on the speech signal to obtain the corresponding silent audio frame, and filter the silent audio frame; perform feature extraction on the filtered speech signal to obtain the speech feature of the live broadcast content corresponding to the live broadcast.

[0155] Here, voice recognition is performed on the live content of the live broadcast to obtain a voice signal corresponding to the live content, and then the voice signal corresponding to the live content is framed to obtain at least two audio frames corresponding to the live content, and then windowing is performed on the at least two audio frames to obtain a corresponding windowed voice signal, and the windowed voice signal is used as the corresponding filtered voice signal.

[0156] In order to obtain a high-precision speech recognition result, after speech recognition, the speech signal corresponding to the live content should be a stable signal. However, in actual applications, the speech signal in the live content may not be very stable as a whole. Although the speech signal has time-varying characteristics, its characteristics can be considered to be relatively stable in a short period of time (such as within the range of 10-30 milliseconds). Therefore, in order to obtain a stable speech signal, it is necessary to perform frame processing on the speech signal corresponding to the live content, and divide the speech signal into multiple audio frames. In actual implementation, the frame can be divided into continuous segments or overlapping segments. The overlapping segmentation method can avoid smooth transition between frames and maintain the continuity between each audio frame. The overlapping part between two adjacent frames is called frame shift, and the ratio of frame shift to frame length is generally taken as 0 to 1 / 2. For example, since it is considered that the speech signal within the range of 10-30 milliseconds is stable, the speech signal is framed with a frame length of not less than 20 milliseconds and about 1 / 2 of the time as a frame shift frame.

[0157] After the above-mentioned framing, the beginning and end of each frame in the obtained voice signal may be discontinuous. If the more audio frames are divided, the greater the error with the original voice signal will be. In order to reduce the error, make the voice signal after framing continuous, and make each frame of the voice signal show the characteristics of a periodic function, a window function can usually be used to perform windowing processing on the voice signal after framing. In actual implementation, commonly used window functions include rectangular windows, Hamming windows and Hanning windows, and different window functions can be selected according to different situations. Through the above-mentioned windowing processing, the corresponding windowed voice signal is obtained, and the windowed voice signal is used as the filtered voice signal of the corresponding live content, and the filtered voice signal is feature extracted to obtain the voice features of the corresponding live content.

[0158] In some embodiments, it may be determined that the voice feature belongs to the target recommendation object based on the matching result in the following manner:

[0159] The voice features of the live broadcast content obtained in real time are matched with the reference voice features of each recommendation object, and the recommendation object corresponding to the reference voice feature with the highest matching degree is determined as the target recommendation object to which the voice features of the current live broadcast content belong.

[0160] For example, in an e-commerce live broadcast, the host wants to recommend 10 products, and each product has a corresponding recommendation copy. The voice features of the current live broadcast content obtained in real time are matched with the reference voice features in the recommendation copy corresponding to the 10 products to obtain similarity values. The product corresponding to the reference voice feature with the highest similarity value is determined as the target recommendation object to which the voice features of the current live broadcast content belong.

[0161] In some embodiments, the live broadcast content may be collected starting from the time point when the similarity value exceeds the similarity threshold value to determine the live broadcast content corresponding to the target recommendation object in the following manner:

[0162] The time point when the similarity value exceeds the first similarity threshold is taken as the starting point, and the time point when the similarity value is lower than the second similarity threshold is taken as the end point, and the live broadcast content is collected from the starting point until the end point; the live broadcast content between the collected starting point and the end point is determined as the live broadcast content corresponding to the target recommendation object.

[0163] For example, when it is detected that the current similarity value reaches 90%, the background server will transmit the time point of the live broadcast content of the current recommended object to the anchor client. After receiving it, the anchor client will automatically start the screen recording function to record the live broadcast content of the current live broadcast room. When it is detected that the similarity value between the voice features of the live broadcast content of the anchor and the reference voice features of the target recommended object is less than 5%, the screen recording will be automatically stopped.

[0164] In some embodiments, the live broadcast content between the collected starting point and the end point may be determined as the live broadcast content corresponding to the target recommendation object in the following manner:

[0165] For each time point corresponding to the starting point and the end point, the scores of the live broadcast content corresponding to each time point are obtained respectively; the live broadcast content corresponding to the scores below the score threshold is filtered; and the filtered live broadcast content is determined as the live broadcast content corresponding to the target recommendation object.

[0166] Here, the score reflects the degree of recommendation of the live broadcast content to the target recommendation object. After the live broadcast begins, the client will count the interactive data such as comment data, shopping data, reward and gift data of the live broadcast room every second, and upload these interactive data and the timestamp of the generated data to the background server. The background server records these interactive data and generates the score of the live broadcast content based on these interactive data. Alternatively, the client calculates the score of the live broadcast content based on the interactive data.

[0167] In actual implementation, each time point of the live broadcast is obtained by dividing the timeline of the live broadcast according to a certain granularity. Different granularities are set according to the real-time requirements. The smaller the granularity, the higher the real-time. For example, the real-time with a granularity of 1 second is greater than the real-time with a granularity of 1 minute. When the granularity is 2 seconds, the score of the live content of the live broadcast room is determined for each time point of the live broadcast. Specifically, it can be implemented in the following way: the comments, shopping, and reward data (interaction data) for the live content within 2 seconds from the receipt of the collection operation are uploaded to the server, and the server generates a score based on the interaction data and returns it to the client, or the client directly generates a score based on the interaction data. The source of the interaction data can be sent by each audience client to the server and returned by the server to the anchor client. The anchor client directly calculates the score of the live content based on the interaction data, or sends the interaction data to the server so that the server calculates the score of the live content based on the interaction data.

[0168] Here, based on the score of the live broadcast content, the live broadcast content collected between the starting point and the end point is filtered and screened, and the live broadcast content with a higher score is screened out, that is, the obtained live broadcast content is the live broadcast essence corresponding to the target recommendation object, which greatly shortens the duration of the live broadcast content corresponding to the target recommendation object. For example, the duration of the live broadcast content collected from the starting point to the end point is 20 minutes, and based on the score of the live broadcast content, only 30 seconds of live broadcast content is screened out. In this way, viewers who missed the live broadcast can quickly learn about the relevant content of the target recommendation object through the live broadcast essence, which improves the efficiency of obtaining useful information and can improve the viewing experience.

[0169] In some embodiments, the live broadcast content may be collected starting from the time point when the similarity value exceeds the similarity threshold value to determine the live broadcast content corresponding to the target recommendation object in the following manner:

[0170] Taking the time point when the similarity value exceeds the similarity threshold as the starting point, continuously collect the live content of the live broadcast, and stop collecting when the similarity value does not exceed the similarity threshold; along with the progress of the live broadcast, taking the time point when the similarity value exceeds the similarity threshold again as the new starting point, continue to collect the live content of the live broadcast, and stop collecting when the accumulatively collected live content meets the duration condition, or when the similarity value does not exceed the similarity threshold again; determine the collected live content as the live content corresponding to the target recommendation object.

[0171] Here, the time point when collection is stopped is the end point, and the live broadcast content between the starting point and the end point can form a live broadcast segment. Repeating the above steps to determine a new starting point and end point can generate multiple live broadcast segments; the accumulated live broadcast segments are combined, and the combined live broadcast content is determined as the live broadcast content corresponding to the target recommendation object.

[0172] In some embodiments, the terminal may determine the target recommendation object in the following manner:

[0173] Extract voice features of live broadcast content in real time; match the voice features with reference voice features of each recommended object respectively, and determine the recommended object to which the voice features belong based on the matching results; determine the similarity value between the voice features and the reference voice features of the recommended object to which the voice features belong; when it is determined based on the similarity value that the live broadcast end time of the recommended object to which the voice features belong has arrived, use the recommended object to which the voice features belong as the target recommended object.

[0174] In some embodiments, when determining the target recommended object, the voice features of the live broadcast content extracted in real time are matched with the reference voice features of each recommended object. For example, an e-commerce live broadcast wants to recommend 10 recommended objects, and each recommended object has a corresponding recommended copy. The voice features of the current live broadcast content extracted in real time are matched with the reference voice features in the recommended copy corresponding to the 10 recommended objects to obtain a similarity value, and the recommended object corresponding to the reference voice feature with the highest similarity value is determined as the recommended object to which the voice features of the current live broadcast content belong. At the same time, the similarity value between the voice features of the live broadcast content and the reference voice features of the recommended object to which it belongs is obtained in real time, and based on the similarity value, it is determined whether the live broadcast end time of the recommended object to which the voice features belong has arrived. For example, when the similarity value is lower than 5%, it is determined that the live broadcast end time of the recommended object to which the voice features belong has arrived, and then the recommended object to which the voice features belong is determined as the target recommended object.

[0175] In some embodiments, the live broadcast content of the target recommendation object may be obtained in the following manner:

[0176] The live broadcast start time of the target recommendation object is obtained; and the content between the live broadcast start time and the live broadcast end time of the live broadcast content is intercepted as the live broadcast content of the target recommendation object.

[0177] For example, when the similarity value reaches 90% for the first time, the corresponding moment is determined as the start time of the live broadcast of the target recommended object. When the similarity value is lower than 5% for the first time, the corresponding moment is determined as the end time of the live broadcast of the target recommended object, and the content collected between the start time and the end time of the live broadcast is used as the live content of the target recommended object.

[0178] In some embodiments, the recommended object to which the voice feature belongs may be determined based on the matching result in the following manner:

[0179] When the matching result represents that the similarity between the voice feature and the reference voice feature of the first recommended object among at least two recommended objects reaches a first similarity threshold, and the similarity between the voice feature and the reference voice features of other recommended objects among at least two recommended objects does not reach the first similarity threshold, the recommended object to which the voice feature belongs is determined to be the first recommended object.

[0180] In some embodiments, the time point corresponding to the first live video frame that reaches the first similarity threshold is used as the start time of the live broadcast of the first recommended object; and the time point corresponding to the last live video frame that reaches the first similarity threshold is used as the end time of the live broadcast of the first recommended object.

[0181] Through the above method, by triggering the playback function item of the corresponding target recommended object presented in the content interface of the live broadcast, the live broadcast content corresponding to the target recommended object is played. Since the target recommended object is the recommended object whose live broadcast content is before the current live broadcast content, that is, the recommended object that has been introduced by the anchor, when the audience triggers the playback function item for the target recommended object, what is presented in the live broadcast interface is the corresponding live broadcast content, rather than just the corresponding shopping link, so that old users can conveniently view the live broadcast content of the recommended target recommended object again without jumping out of the live broadcast room, or allow new users to view the live broadcast content of the target recommended object that they have missed before; at the same time, they can also intuitively view the time preview of the recommended object to be introduced, so that users can have an expectation psychologically, and the operation is simple, which improves the user experience, and thus helps to improve the conversion rate of the recommended objects in the live broadcast room.

[0182] Next, the method for processing live broadcast information provided by the embodiment of the present application is described. Figure 7 , Figure 7 An optional flowchart of a method for processing live broadcast information provided in an embodiment of the present application is provided. Figure 7 In the example, the first terminal is the terminal corresponding to the anchor client, and the second terminal is the terminal corresponding to the audience client. Figure 7 The steps shown are explained.

[0183] Step 201: presenting live broadcast content in the live broadcast interface of the first terminal.

[0184] Step 202: The first terminal extracts voice features of the live content in real time.

[0185] Step 203: The first terminal matches the voice feature with the reference voice feature of each recommended object.

[0186] For example, in an e-commerce live broadcast, the host wants to recommend 10 products, and each product has a corresponding recommendation copy. The voice features of the current live broadcast content obtained in real time are matched with the reference voice features in the recommendation copy corresponding to the 10 products to obtain similarity values. The product corresponding to the reference voice feature with the highest similarity value is determined as the recommendation object to which the voice features of the current live broadcast content belong.

[0187] Step 204: When the matching result indicates that the voice feature belongs to the first recommended object, a floating layer corresponding to the first recommended object is presented in the live broadcast interface of the first terminal.

[0188] Here, when the matching result represents that the similarity value between the voice feature and the reference voice feature of the first recommended object is the highest, it is determined that the current live broadcast content is the live broadcast content for the first recommended object, that is, the voice signal in the current live broadcast content belongs to the first recommended object.

[0189] Step 205: The first terminal obtains in real time a similarity value between the voice feature and a reference voice feature of the first recommended object.

[0190] Step 206: The first terminal starts collecting live broadcast content from the time point when the similarity value exceeds the first similarity threshold as the starting point, and stops collecting from the time point when the similarity value is lower than the second similarity threshold as the end point.

[0191] For example, when it is detected that the current similarity value reaches 90%, the background server transmits the time point of the live broadcast content of the current recommended object to the anchor client. After receiving it, the anchor client automatically starts the screen recording function to record the live broadcast content of the current live broadcast room. When it is detected that the similarity value between the voice features of the live broadcast content of the anchor and the reference voice features of the first recommended object is less than 5%, the screen recording is automatically stopped.

[0192] Step 207: the first terminal filters the collected live broadcast content between the starting point and the end point, and uses the filtered live broadcast content as the live broadcast content corresponding to the first recommendation object.

[0193] Here, for each time point corresponding to the starting point and the end point, the scores of the live broadcast content corresponding to each time point are obtained respectively; the live broadcast content corresponding to the scores below the score threshold is filtered; and the filtered live broadcast content is determined as the live broadcast content corresponding to the first recommendation object.

[0194] Step 208: The first terminal sends the live broadcast content corresponding to the first recommended object to the server.

[0195] Here, the first terminal stores the recorded live broadcast content corresponding to the first recommended object and the corresponding video identifier in the server.

[0196] Step 209: The server returns the live broadcast end status corresponding to the first recommended object to each terminal.

[0197] Here, the server stores the live broadcast content and the corresponding video identifier of the first recommendation in the server. When the live broadcast corresponding to the first recommendation object ends, the server returns the live broadcast end status corresponding to the first recommendation object to each terminal.

[0198] Step 210: Presenting a playback function item corresponding to the first recommended object and a floating layer corresponding to the second recommended object in the live broadcast interface of the second terminal.

[0199] Here, the anchor has finished introducing the first recommended object and is introducing the second recommended object. Therefore, at this time, the live broadcast interface of the first terminal and the second terminal presents the live broadcast content corresponding to the second recommended object and the playback function item of the first recommended object.

[0200] The process of determining the second recommended object is the same as the process of determining the first recommended object, and will not be repeated here.

[0201] Step 211: the second terminal sends a request for acquiring the live content of the first recommended object to the server in response to the trigger operation on the playback function item.

[0202] Step 212: The server obtains the live broadcast content of the first recommended object based on the acquisition request.

[0203] Step 213: The server returns the live broadcast content of the first recommended object to the second terminal.

[0204] Step 214: Play the live broadcast content of the first recommended object in the live broadcast interface of the second terminal.

[0205] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0206] Normally, the host often introduces a variety of products in an e-commerce live broadcast, but users who enter the live broadcast room late may miss the products that the host has introduced, and cannot conveniently view the products that the host has introduced (i.e., the target recommended objects) and the preview of the products to be introduced in the live broadcast room, resulting in a poor user experience. To this end, the method for processing live broadcast information provided in the embodiment of the present application can be applied to e-commerce live broadcast scenarios. In the content interface of the live broadcast for recommending at least two recommended objects (i.e., products), a playback function item corresponding to the target recommended object among at least two recommended objects is presented, wherein the playback function item is used to realize the playback of the live content corresponding to the target recommended object, and the target recommended object is the recommended object whose corresponding live content is before the current live content; when a trigger operation for the playback function item is received, the live content corresponding to the target recommended object is played.

[0207] Among them, the live broadcast content corresponding to the target recommendation object is a short video of the host introducing the product based on voice recognition of the live broadcast content when the product is live broadcast. When the user enters the live broadcast room, the playback function can be triggered to conveniently and intuitively view the products that the host has introduced and the time preview of the products to be introduced in the live broadcast room, so that new users can intuitively understand the progress of the product explanation in the current live broadcast room, let users have a psychological expectation, and thus improve the shopping conversion rate of the live broadcast room.

[0208] See also Figure 8 , Figure 8 An optional flow chart of a method for processing live broadcast information provided in an embodiment of the present application is combined with Figure 8 Provide explanation.

[0209] Step 301: presenting live broadcast content in the live broadcast interface of the terminal;

[0210] Step 302: The terminal extracts voice features of the live content in real time;

[0211] In actual applications, before performing voice recognition on live content, it is usually necessary to use silence removal operation technology to analyze the voice signal (i.e., the host's voice) in the live content, divide the host's voice into frames, and obtain multiple audio frames. There are overlapping parts between frames (i.e., frame shift). In order to reduce subsequent interference, the silence at the beginning and end is usually removed.

[0212] See also Fig. 9 , Fig. 9 The speech signal diagram provided in the embodiment of the present application, although the speech signal has a time-varying characteristic, but in a short time (such as within the range of 10-30 milliseconds), its characteristic can be considered to be relatively stable, therefore, in Fig. 9 In the process, 25 milliseconds is used as the frame length, and about 15 milliseconds is used as the frame shift frame to perform frame processing on the speech signal to obtain corresponding multiple audio frames.

[0213] After the above-mentioned framing, the beginning and end of each frame in the obtained voice signal may be discontinuous. The more audio frames are divided, the greater the error with the original voice signal will be. In order to reduce the error, make the voice signal after framing continuous, and make each frame of the voice signal show the characteristics of a periodic function, a window function can usually be used to perform windowing processing on the voice signal after framing. In actual implementation, commonly used window functions include rectangular window, Hamming window and Hanning window, and different window functions can be selected according to different situations. Through the above-mentioned windowing processing, the corresponding windowed voice signal is obtained, and the windowed voice signal is used as the voice signal after the silence removal of the corresponding live content, and the obtained voice signal after the silence removal is feature extracted to obtain the voice features of the corresponding live content.

[0214] Step 303: The terminal matches the voice feature with the reference voice feature of the target recommendation object;

[0215] Here, the system detects the dynamics of the product floating layer of this live broadcast in real time. When the product floating layer pops up, it automatically identifies the product text in the floating layer, that is, the terminal matches the voice features with the reference voice features of the product presented in the floating layer (that is, the reference voice features of the target recommendation object mentioned above).

[0216] During the live broadcast, in order to improve the recognition rate and overcome the different pronunciation time of the same sound, the dynamic time warping (DTW) algorithm is used to stretch or shorten the voice signal in the live content until it is consistent with the length of the standard mode. The processed voice signal is then matched with the reference voice template in the product copy of the current recommendation object, and based on the matching results, it is determined whether the voice features in the current live content belong to the corresponding product in the floating layer.

[0217] In actual implementation, when the host's voice signal is detected, the collected voice signal can also be sent to the background server. After the background server decompresses and transcodes the received voice signal, the voice features of the voice signal are similarly matched with the reference voice features of the target recommendation object presented in the floating layer of the current live broadcast room.

[0218] Step 304: When the matching result indicates that the voice feature belongs to the target recommendation object, the terminal records the live broadcast content.

[0219] Here, when it is determined based on the matching results that the voice feature belongs to the product presented in the floating layer, the similarity value between the voice feature and the reference voice feature of the target recommended object presented in the floating layer is determined in real time; starting from the time point when the similarity value exceeds the similarity threshold, the live broadcast content is collected to determine the live broadcast content corresponding to the target recommended object.

[0220] In actual implementation, the time point when the similarity value exceeds the first similarity threshold is taken as the starting point, and the time point when the similarity value is lower than the second similarity threshold is taken as the end point, and the live broadcast content is collected from the starting point until the end point; the live broadcast content collected between the starting point and the end point is determined as the live broadcast content corresponding to the target recommendation object.

[0221] For example, when it is detected that the current similarity value reaches 90%, the background server will transmit the time point of the current live content to the anchor client. After receiving it, the anchor client will automatically start the screen recording function to record the live content of the current live room. When it is detected that the similarity value between the voice features of the live content of the anchor and the reference voice features of the target recommendation object is less than 5%, the screen recording will be automatically stopped.

[0222] Here, the live content collected between the starting point and the end point can also be filtered based on the score of the live content at each time point, so as to obtain the live content with higher scores, that is, the obtained live content is the essence of the live broadcast corresponding to the target recommendation object, which greatly shortens the duration of the live content corresponding to the target recommendation object. For example, the duration of the live content collected from the starting point to the end point is 20 minutes, and the live content with a duration of only 30 seconds is obtained based on the score of the live content.

[0223] Step 305: The terminal sends the recorded live content to the server.

[0224] Here, the live broadcast content and the corresponding video identifier corresponding to the recorded target recommendation object are stored in a content delivery network (CDN, Content Delivery Network) server.

[0225] Step 306: The server pushes the live broadcast end status of the target recommendation object to the terminal.

[0226] Here, when the anchor has finished explaining the target product, the server will push the live broadcast end status of the target recommendation object to the terminal, so that the terminal presents the corresponding reply function item and allows the user to trigger the reply function item to watch the live broadcast content of the explained target product.

[0227] Step 307: The terminal presents the playback function item of the target recommendation object.

[0228] Step 308: The terminal plays the live content of the target recommendation object in response to the triggering operation on the reply function item.

[0229] Here, when the user triggers the playback function, the terminal generates and sends a request to obtain the live content of the target recommended object to the CDN server in response to the trigger operation. The CDN server obtains the corresponding live content based on the video identifier in the acquisition request and returns the obtained live content to the terminal for playback. In this way, the user can quickly view the live content of the product that the anchor has explained in the current live broadcast room.

[0230] In actual applications, at least one preview information corresponding to a previewed product may also be presented in the live broadcast interface, wherein the preview information is used to preview the live broadcast start time corresponding to the product that has not yet been introduced. The unintroduced product is a recommended object whose corresponding live broadcast content is after the current live broadcast content. The live broadcast start time of the product preview is the average time estimated by the host to introduce a single product in this live broadcast room. The product explanation time can be manually set by the host before the broadcast, or it can be generated based on the evaluation time of the product explained in this live broadcast. In order to distinguish between introduced and unintroduced products, a visual distinction can be made between the list of introduced products and the list of unintroduced products, so that the audience can intuitively see the live broadcast progress of this live broadcast when purchasing or checking the progress of the host.

[0231] The following continues to describe the exemplary structure of the live broadcast information processing device 555 provided in the embodiment of the present application implemented as a software module. In some embodiments, Fig.10 As shown, Fig.10 This is a schematic diagram of the structure of the live broadcast information processing device provided in an embodiment of the present application. The software modules stored in the live broadcast information processing device 555 of the memory 550 may include:

[0232] The first presentation module 5551 is used to present a live content interface, where the live content of the live broadcast is used to recommend at least two recommendation objects;

[0233] A second presentation module 5552 is used to present, in the content interface, a playback function item corresponding to a target recommended object among the at least two recommended objects, wherein the playback function item is used to play the live content corresponding to the target recommended object;

[0234] The target recommended object is a recommended object whose corresponding live broadcast content is before the current live broadcast content;

[0235] The playing module 5553 is used to play the live content corresponding to the target recommendation object when a trigger operation for the playback function item is received.

[0236] In some embodiments, when the number of the target recommendation objects is at least two, the second presentation module is further used to present, in the content interface, playback function items corresponding to the target recommendation objects in parallel; or

[0237] In the content interface, playback function items corresponding to each target recommendation object are scrolled and presented.

[0238] In some embodiments, when the number of the target recommendation objects is at least two, the second presentation module is further used to respectively obtain the live broadcast time corresponding to the live broadcast content of each of the target recommendation objects, and the live broadcast time is the live broadcast start time or the live broadcast end time;

[0239] In the target area of ​​the content interface, the playback function items corresponding to each target recommended object are presented in a manner such that the live broadcast time is in front and the playback function items corresponding to the corresponding target recommended object are in front.

[0240] In some embodiments, when the number of the target recommendation objects is at least two, the second presentation module is further used to obtain the interaction heat corresponding to the live broadcast content of each of the target recommendation objects respectively;

[0241] In the target area of ​​the content interface, the playback function items corresponding to the target recommended objects are presented in a manner such that the playback function items corresponding to the target recommended objects with high interaction popularity are placed in front.

[0242] In some embodiments, when the number of the target recommendation objects is at least two, the playback module is further configured to present, in the content interface, a selection interface for selecting the at least two target recommendation objects to be played;

[0243] In response to a selection operation on a target recommendation object triggered based on the selection interface, live content corresponding to the selected target recommendation object is played.

[0244] In some embodiments, the device further includes a third presentation module, the third presentation module being configured to present, in the content interface, preview information corresponding to at least one preview recommendation object, the preview information being configured to preview a live broadcast start time corresponding to the preview recommendation object;

[0245] The preview recommended object is a recommended object whose corresponding live broadcast content is after the current live broadcast content.

[0246] In some embodiments, the playback module is further used to play the live content corresponding to the target recommendation object through a sub-interface independent of the content interface.

[0247] In some embodiments, the playback module is further used to hide the live broadcast content of the live broadcast, and use the live broadcast window to play the live broadcast content corresponding to the target recommendation object.

[0248] In some embodiments, the playback module is further configured to stop playing the live broadcast content corresponding to the target recommendation object in response to a trigger operation on the target area during the playing of the live broadcast content corresponding to the target recommendation object;

[0249] The target area is an area in the current content interface where a playback interface of the live content corresponding to the target recommendation object is located.

[0250] In some embodiments, the device further comprises a determining module, wherein the determining module is configured to

[0251] Extracting voice features of the live broadcast content in real time;

[0252] Matching the voice feature with the reference voice feature of each of the recommended objects respectively, and determining the recommended object to which the voice feature belongs based on the matching result;

[0253] Determining a similarity value between the voice feature and a reference voice feature of a recommended object to which the voice feature belongs;

[0254] When it is determined based on the similarity value that the live broadcast end time of the recommended object to which the voice feature belongs has arrived, the recommended object to which the voice feature belongs is used as the target recommended object.

[0255] In some embodiments, the determination module is further used to obtain the live broadcast start time of the target recommendation object;

[0256] The content of the live broadcast between the start time of the live broadcast and the end time of the live broadcast is intercepted as the live broadcast content of the target recommendation object.

[0257] In some embodiments, the determining module is further configured to, when the matching result indicates that the similarity between the voice feature and the reference voice feature of the first recommended object among the at least two recommended objects reaches a first similarity threshold, and

[0258] When the similarity between the voice feature and the reference voice feature of the other recommended object among the at least two recommended objects does not reach the first similarity threshold, the recommended object to which the voice feature belongs is determined to be the first recommended object.

[0259] In some embodiments, the determining module is further configured to, before presenting the playback function item corresponding to the target recommended object among the at least two recommended objects,

[0260] For each time point of the live broadcast, obtaining the voice features of the live broadcast content of the live broadcast in real time;

[0261] Matching the voice feature with reference voice features of each of the recommended objects;

[0262] When it is determined based on the matching result that the voice feature belongs to the target recommendation object, a similarity value between the voice feature and a reference voice feature of the target recommendation object is determined in real time;

[0263] Taking the time point when the similarity value exceeds the similarity threshold as the starting point, start collecting the live broadcast content of the live broadcast to determine the live broadcast content corresponding to the target recommendation object.

[0264] In some embodiments, the determination module is further configured to use the time point when the similarity value exceeds the first similarity threshold as the starting point, and the time point when the similarity value is lower than the second similarity threshold as the end point, and

[0265] Start collecting the live broadcast content of the live broadcast from the starting point and stop collecting at the end point;

[0266] The collected live broadcast content between the starting point and the end point is determined as the live broadcast content corresponding to the target recommendation object.

[0267] In some embodiments, the determination module is further used to obtain the score of the live broadcast content corresponding to each time point between the starting point and the end point respectively;

[0268] Filter the live broadcast content corresponding to the scores below the score threshold;

[0269] The obtained filtered live broadcast content is determined as the live broadcast content corresponding to the target recommendation object.

[0270] In some embodiments, the determination module is further used to continuously collect the live broadcast content of the live broadcast starting from the time point when the similarity value exceeds the similarity threshold, and stop collecting when the similarity value does not exceed the similarity threshold;

[0271] As the live broadcast proceeds, the live broadcast content of the live broadcast is continuously collected with the time point when the similarity value exceeds the similarity threshold again as a new starting point, and the collection is stopped when the accumulated live broadcast content meets the duration condition or when the similarity value does not exceed the similarity threshold again;

[0272] The collected real-time live broadcast content is determined as the live broadcast content corresponding to the target recommendation object.

[0273] In some embodiments, the determination module is further used to perform voice recognition on the live broadcast content of the live broadcast to obtain a voice signal corresponding to the live broadcast content;

[0274] Performing speech boundary detection on the speech signal to obtain corresponding silent audio frames, and filtering the silent audio frames;

[0275] Feature extraction is performed on the filtered voice signal to obtain voice features corresponding to the live broadcast content of the live broadcast.

[0276] The embodiment of the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the live broadcast information processing method described in the embodiment of the present application.

[0277] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the live broadcast information processing method provided in the embodiment of the present application.

[0278] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0279] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0280] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0281] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0282] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing live broadcast information, characterized in that: The method comprises: Presenting a live content interface, wherein the live content of the live broadcast is used for the anchor to recommend at least one recommended object to the audience; Presenting a playback function item of a target recommended object, wherein the playback function item includes product information of the target recommended object, and the playback function item is used to play the live content of the target recommended object, wherein the target recommended object is a recommended object whose live content is before the current live content; In response to the triggering operation on the playback function item, the live broadcast content of the target recommendation object is played.

2. The method according to claim 1, characterized in that The playback function item of the presentation target recommendation object includes: In the content interface, presenting a playback function item of the target recommended object; or, Presenting a playback function item of the target recommendation object in the content interface by means of a floating layer or a floating window; or, In a first sub-interface independent of the content interface, presenting a playback function item of the target recommendation object; or, In response to a trigger operation on a playback entry in the content interface, a playback guide interface is presented, and in the playback guide interface, playback function items of a target recommended object are presented.

3. The method according to claim 1, characterized in that When the number of the target recommended objects is at least two, the playback function items of the target recommended objects are presented, including: presenting playback function items of each of the target recommendation objects in parallel; or, The playback function items of each of the target recommended objects are presented in a scrolling manner.

4. The method according to claim 1, characterized in that When the number of the target recommended objects is at least two, the playback function items of the target recommended objects are presented, including: Respectively obtain the live broadcast time corresponding to the live broadcast content of each target recommendation object, where the live broadcast time is the live broadcast start time or the live broadcast end time; The playback function items of each target recommended object are presented in a manner that the live broadcast time is in front and the playback function items of the corresponding target recommended object are in front.

5. The method according to claim 1, characterized in that When the number of the target recommended objects is at least two, the playback function items of the target recommended objects are presented, including: Respectively obtain the interaction heat corresponding to the live broadcast content of each target recommendation object; The playback function items of each target recommended object are presented in a manner such that the playback function items of the target recommended objects with high interaction heat are placed in front.

6. The method according to claim 1, characterized in that The playing of the live broadcast content of the target recommendation object includes: Play the live content of the target recommendation object through a second sub-interface independent of the content interface; or The live broadcast content of the live broadcast is hidden, and the live broadcast content of the target recommendation object is played through the live broadcast window.

7. The method according to claim 1, characterized in that The method further comprises: During the process of playing the live content of the target recommended object, the live content of the recommended object being broadcast live in the content interface is played silently.

8. The method according to claim 1, characterized in that When the number of the target recommendation objects is at least two, playing the live content of the target recommendation objects includes: Presenting a selection interface for selecting the at least two target recommendation objects to be played; In response to a play selection operation triggered based on the selection interface, the live broadcast content of the selected target recommendation object is played.

9. The method according to claim 1, characterized in that The method further comprises: Presenting preview information corresponding to the preview recommendation object, wherein the preview information is used to preview the live broadcast start time corresponding to the preview recommendation object; The preview recommended object is a recommended object whose corresponding live broadcast content is after the current live broadcast content.

10. The method according to claim 9, characterized in that The presenting of the forecast information corresponding to the forecast recommendation object includes: In the content interface, presenting the preview information corresponding to the preview recommendation object; or, In a third sub-interface independent of the content interface, presenting the preview information corresponding to the preview recommendation object; or, In response to a triggering operation on a preview entry in the content interface, a preview guide interface is presented, and preview information corresponding to the preview recommendation object is presented in the preview guide interface.

11. The method according to claim 1, characterized in that The playback function item of the presentation target recommendation object includes: Presenting a detailed viewing entrance for the live broadcast; In response to a trigger operation on the detail viewing entrance, presenting a detail page of the live broadcast; In the details page, the recommended objects being live-recommended, the playback function items of the target recommended objects, and the preview information of the preview recommended objects are presented in a segmented manner.

12. The method according to claim 1, characterized in that The method further comprises: During the process of playing the live content of the target recommended object, in response to a trigger operation on a target area, or in response to a trigger operation on a cancel function item, stop playing the live content corresponding to the target recommended object; The target area is an area in the current content interface where the playback interface of the live content that is different from the target recommendation object is located.

13. The method according to claim 1, characterized in that Before presenting the playback function item corresponding to the target recommendation object, the method further includes: For each time point of the live broadcast, obtaining the voice features of the live broadcast content of the live broadcast in real time; Matching the voice feature with reference voice features of each of the recommended objects; When it is determined based on the matching result that the voice feature belongs to the target recommendation object, a similarity value between the voice feature and a reference voice feature of the target recommendation object is determined in real time; Taking the time point when the similarity value exceeds the similarity threshold as the starting point, start collecting the live broadcast content of the live broadcast to determine the live broadcast content of the target recommendation object.

14. The method according to claim 13, characterized in that The step of starting to collect the live broadcast content of the live broadcast from the time point when the similarity value exceeds the similarity threshold value to determine the live broadcast content of the target recommendation object includes: The time point when the similarity value exceeds the first similarity threshold is taken as the starting point, and the time point when the similarity value is lower than the second similarity threshold is taken as the end point, and Start collecting the live broadcast content of the live broadcast from the starting point and stop collecting at the end point; The collected live broadcast content between the starting point and the end point is determined as the live broadcast content of the target recommendation object.

15. The method according to claim 14, characterized in that The step of determining the collected live broadcast content between the starting point and the end point as the live broadcast content of the target recommendation object includes: For each time point corresponding to the starting point and the end point, respectively obtain the score of the live broadcast content corresponding to each time point; Filter the live broadcast content corresponding to the scores below the score threshold; The obtained filtered live broadcast content is determined as the live broadcast content of the target recommendation object.

16. The method according to claim 13, characterized in that The step of starting to collect the live broadcast content of the live broadcast from the time point when the similarity value exceeds the similarity threshold value to determine the live broadcast content of the target recommendation object includes: Taking the time point when the similarity value exceeds the similarity threshold as the starting point, continuously collecting the live broadcast content of the live broadcast, and stopping the collection when the similarity value does not exceed the similarity threshold; As the live broadcast proceeds, the live broadcast content of the live broadcast is continuously collected with the time point when the similarity value exceeds the similarity threshold again as a new starting point, and the collection is stopped when the accumulated live broadcast content meets the duration condition or when the similarity value does not exceed the similarity threshold again; The collected live broadcast content is determined as the live broadcast content of the target recommendation object.

17. The method according to claim 13, characterized in that The real-time acquisition of the voice features of the live broadcast content includes: Performing voice recognition on the live broadcast content of the live broadcast to obtain a voice signal of the live broadcast content; Performing speech boundary detection on the speech signal to obtain silent audio frames, and filtering the silent audio frames; Feature extraction is performed on the filtered voice signal to obtain voice features of the live broadcast content.

18. The method of claim 1, wherein: Before presenting the playback function item of the target recommendation object, the method further includes: Extracting voice features of the live broadcast content in real time; Matching the voice feature with the reference voice feature of each of the recommended objects respectively, and determining the recommended object to which the voice feature belongs based on the matching result; Determining a similarity value between the voice feature and a reference voice feature of a recommended object to which the voice feature belongs; When it is determined based on the similarity value that the live broadcast end time of the recommended object to which the voice feature belongs has arrived, the recommended object to which the voice feature belongs is used as the target recommended object.

19. The method according to claim 18, characterized in that The method further comprises: Obtain the live broadcast start time of the target recommendation object; The content of the live broadcast between the start time of the live broadcast and the end time of the live broadcast is intercepted as the live broadcast content of the target recommendation object.

20. The method of claim 18, wherein: The determining the recommended object to which the voice feature belongs based on the matching result includes: When the matching result indicates that the similarity between the voice feature and the reference voice feature of the first recommended object among the at least two recommended objects reaches a first similarity threshold, and When the similarity between the voice feature and the reference voice feature of the other recommended object among the at least two recommended objects does not reach the first similarity threshold, the recommended object to which the voice feature belongs is determined to be the first recommended object.

21. A device for processing live broadcast information, characterized in that: The device comprises: A first presentation module, used to present a live content interface, wherein the live content of the live broadcast is used for the anchor to recommend at least one recommendation object to the audience; A second presentation module is used to present a playback function item of a target recommended object, wherein the playback function item includes product information of the target recommended object, and the playback function item is used to play the live content of the target recommended object, and the target recommended object is a recommended object whose live content is before the current live content; The playing module is used to play the live content of the target recommendation object in response to the triggering operation on the playback function item.

22. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the live broadcast information processing method described in any one of claims 1 to 20 when executing the executable instructions stored in the memory.

23. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to implement the live broadcast information processing method described in any one of claims 1 to 20 when executed by a processor.

24. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method for processing live broadcast information described in any one of claims 1 to 20 is implemented.