Information playback control method, device, equipment and computer readable storage medium

Abstract

The present application provides an information playback control method, device, equipment and computer-readable storage medium; the method includes: when a trigger operation is detected on a rate control mark of a playback interface, in response to the trigger operation for the rate control mark, displaying rate options; wherein the rate options include intelligent adjustment options, and the intelligent adjustment options are used to trigger dynamic rate adjustment of multimedia information; when a trigger operation is detected on the intelligent adjustment options, in response to the trigger operation for the intelligent adjustment options, based on the content information of the multimedia information, determining the target rate ratio of the multimedia information; wherein the target rate ratio is dynamic; using the target rate ratio, adjusting the playback rate of the multimedia information to obtain the adjusted multimedia information; playing the adjusted multimedia information. Through the present application, the effect of multimedia information playback control can be improved through artificial intelligence.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to an information playback control method, device, equipment and computer-readable storage medium. Background Art

[0002] There are a large number of multimedia resources on the Internet, such as audio, video, etc. These resources can be played on the user's terminal, so that the user can play audio or video anytime and anywhere. When playing multimedia information, the user usually has the need to adjust the playback speed of the multimedia information, for example, speeding up the playback speed of the video so that the entire video can be played in a shorter time, or slowing down the audio playback speed so as to carefully learn a song. However, the control method of the playback speed of multimedia information in the related art is poorly adaptable to different types of multimedia information, so that the control effect of the multimedia information playback rate is low, and ultimately the effect of multimedia information playback control is poor. Summary of the invention

[0003] The embodiments of the present application provide an information playback control method, device, equipment and computer-readable storage medium, which can improve the effect of multimedia information playback control.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an information playback control method, including:

[0006] When a trigger operation is detected on a rate control mark on a playback interface, in response to the trigger operation on the rate control mark, rate options are displayed; wherein the rate options include an intelligent adjustment option, and the intelligent adjustment option is used to trigger dynamic rate adjustment of multimedia information;

[0007] When a trigger operation is detected on the smart adjustment option, in response to the trigger operation for the smart adjustment option, a target rate ratio of the multimedia information is determined based on the content information of the multimedia information; wherein the target rate ratio is dynamic;

[0008] Using the target rate ratio, adjusting the playback rate of the multimedia information to obtain adjusted multimedia information;

[0009] The adjusted multimedia information is played.

[0010] The present application provides an information playback control device, including:

[0011] An information display module, configured to display rate options in response to a trigger operation on a rate control mark on a playback interface when a trigger operation is detected on the rate control mark; wherein the rate options include an intelligent adjustment option, and the intelligent adjustment option is used to trigger dynamic rate adjustment of multimedia information;

[0012] a rate ratio determination module, configured to determine a target rate ratio of the multimedia information based on content information of the multimedia information in response to a trigger operation for the intelligent adjustment option when a trigger operation is detected on the intelligent adjustment option; wherein the target rate ratio is dynamic;

[0013] A rate adjustment module, used to adjust the playback rate of the multimedia information by using the target rate ratio to obtain adjusted multimedia information;

[0014] The information display module is also used to play the adjusted multimedia information.

[0015] In some embodiments of the present application, the content information of the multimedia information includes at least one sub-content information; the rate ratio determination module is further used to divide the multimedia information into at least one information segment based on the at least one sub-content information; based on the sub-content information corresponding to each information segment after the multimedia information is divided, calculate the rate ratio of each information segment to obtain the rate ratio corresponding to each information segment; and determine the target rate ratio of the multimedia information based on the rate ratio corresponding to each information segment.

[0016] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: text information; the rate ratio determination module is also used to calculate the display rate of the text information to obtain a text display rate; using the text display rate, a preset text rate and a preset maximum rate ratio, an adjustment ratio is calculated for each information segment to obtain the rate ratio corresponding to each information segment.

[0017] In some embodiments of the present application, the rate ratio determination module is further used to compare the text display rate with the preset text rate to obtain a rate ratio calculation factor; and perform a difference calculation on the preset maximum rate ratio and the rate ratio calculation factor to obtain the rate ratio corresponding to each information segment.

[0018] In some embodiments of the present application, the rate ratio determination module is further used to extract a matching rate ratio that matches the sub-content information corresponding to each information segment from a plurality of preset rate ratios; and use the matching rate ratio as the rate ratio corresponding to each information segment.

[0019] In some embodiments of the present application, the multimedia information includes at least one information segment, and the target rate ratio includes the rate ratio corresponding to each information segment; the rate adjustment module is further used to adjust the playback rate of each information segment according to the rate ratio corresponding to each information segment, so as to obtain an adjusted information segment corresponding to each information segment; and the adjusted information segment corresponding to each information segment is used to splice out the adjusted multimedia information.

[0020] In some embodiments of the present application, the rate ratio determination module is further used to smooth the rate ratio corresponding to each information segment to obtain a smoothed rate ratio corresponding to each information segment.

[0021] In some embodiments of the present application, the rate ratio determination module is also used for the rate ratio determination module to use the smoothed rate ratio corresponding to each information segment to splice out the target rate ratio of the multimedia information.

[0022] In some embodiments of the present application, the rate ratio of each information segment includes the sub-rate ratios corresponding to each time period in each information segment; the rate ratio determination module is further used to perform difference calculation between the rate ratio corresponding to each information segment and the rate ratio corresponding to the previous information segment of each information segment to obtain a rate ratio difference; the rate ratio difference is compared with the number of time periods to obtain a rate ratio change amplitude of each time period in the time periods; the number of time periods represents the number of time periods in each information segment; the sub-rate ratio of the previous time period of each time period is added to the rate ratio change amplitude of each time period to obtain the sub-rate ratio of each time period; when the calculation of the sub-rate ratio is completed for each time period, the sub-rate ratios of the various time periods in each information segment are obtained; the sub-rate ratios of the various time periods are used to splice out a smoothed rate ratio corresponding to each information segment.

[0023] In some embodiments of the present application, the rate ratio determination module is also used to determine the ratio of the rate ratio difference and the number of time periods as the rate ratio change amplitude for each time period when the ratio of the rate ratio difference and the number of time periods is less than or equal to a preset change amplitude; and when the ratio of the rate ratio difference and the number of time periods is greater than the preset change amplitude, determine the preset change amplitude as the rate ratio change amplitude for each time period.

[0024] In some embodiments of the present application, the rate ratio determination module is further used to add the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period to obtain a sum result; for each time period, a preset number of associated time periods are extracted, and the sub-rate ratios corresponding to each of the preset number of associated time periods are used as the preset number of associated sub-rate ratios corresponding to the sub-rate ratio of each time period; from the preset number of associated sub-rate ratios, the maximum associated sub-rate ratio is extracted; when the sum result is less than or equal to the maximum associated sub-rate ratio, the sum result is used as the sub-rate ratio of each time period.

[0025] The present application provides an information playback control device, including:

[0026] A memory, used for storing executable information playback control instructions;

[0027] The processor is used to implement the information playback control method provided in the embodiment of the present application when executing the executable information playback control instruction stored in the memory.

[0028] An embodiment of the present application provides a computer-readable storage medium storing executable information playback control instructions for causing a processor to execute and implement the information playback control method provided in the embodiment of the present application.

[0029] The embodiments of the present application have the following beneficial effects: the information playback control device responds to a trigger operation for a rate control identifier, displays a rate selection port including an intelligent adjustment option, and then responds to a trigger operation for the intelligent adjustment option, analyzes the content information of the multimedia information, determines a dynamic target rate ratio, and then uses the target rate ratio to adjust the playback rate of the multimedia information, and displays the adjusted multimedia information. In this way, the playback rate of the multimedia information can be dynamically controlled according to the content of the multimedia information, thereby making the control of the multimedia information playback rate more intelligent and improving the playback control effect of the multimedia information. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the video playback rate adjustment method;

[0031] Figure 2 It is a schematic diagram for adjusting the speed of the video;

[0032] Figure 3 It is a schematic diagram of controlling the video playback rate by identifying key segments;

[0033] Figure 4 It is a schematic diagram of an optional architecture of the information playback control system 100 provided in an embodiment of the present application;

[0034] Figure 5 is a structural diagram of an information playback control device 400 provided in an embodiment of the present application;

[0035] Figure 6 This is an optional flow diagram of the information playback control method provided in the embodiment of the present application. Figure 1 ;

[0036] Figure 7 is a schematic diagram of a display rate option window provided in an embodiment of the present application;

[0037] Figure 8 is a schematic diagram of the adjusted multimedia information provided in an embodiment of the present application;

[0038] FIG. 9( a ) is a schematic diagram of presenting adjusted multimedia information provided by an embodiment of the present application Figure 1 ;

[0039] FIG. 9( b ) is a schematic diagram of presenting adjusted multimedia information provided by an embodiment of the present application Figure 2 ;

[0040] Fig.10 This is an optional flow diagram of the information playback control method provided in the embodiment of the present application. Figure 2 ;

[0041] Fig.11 This is an optional flow diagram of the information playback control method provided in the embodiment of the present application. Figure 3 ;

[0042] Fig.12 This is a diagram showing the effect of the smoothing process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0044] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0045] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0047] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0048] 1) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the ability to perceive, reason and make decisions.

[0049] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Artificial intelligence technology generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0050] 2) Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further image processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition. Among them, OCR refers to the process of determining the shape of printed characters by detecting darkness and light, and then translating the shape into computer text using character recognition methods.

[0051] 3) Video understanding can be understood as extracting semantic concepts in human thinking from video content, so as to make decisions on video processing based on the obtained semantic concepts.

[0052] 4) Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural speech. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, language understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0053] 5) Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies.

[0054] There are a large number of multimedia information resources on the Internet, such as videos and audios, which can be played on the user's terminal so that the user can watch the multimedia information anytime and anywhere. When watching multimedia information, the user usually needs to adjust the playing speed of the multimedia information, for example, speed up the playing speed of the video so that the whole video can be played in a shorter time, or slow down the playing speed of the audio so as to learn a song carefully.

[0055] Below, taking video as an example, the multimedia information playback control method in the related art is described. In the related art, three methods of adjusting the video playback speed are provided. The first method is based on a fixed rate selected by the user as the video playback rate. For example, Figure 1 It is the video playback rate adjustment method, see Figure 1 , when the terminal detects that the user clicks the speed 1-11 button in the video playback interface 1-1, 1.0x is selected from the various rate options provided in the pop-up speed selection window 1-12, namely 0.5x, 1.0x and 2.0x, so that the terminal controls the playback of the video according to the original playback speed. However, this method always plays the video at a constant rate. When the amount of information in the video content is small, this method will not cause discomfort to the user, but when the amount of information in the video is large, or the video speech speed is fast, this method is very likely to cause discomfort to the user.

[0056] The second method is to analyze the speed of the video content from the dynamic situation of the picture, and then adjust the video to a more uniform speed effect. For example, Figure 2 is a schematic diagram for adjusting the speed of the video, such as Figure 2 As shown, the terminal determines the sense of speed based on the actions of the characters in the video, and then always keeps the sense of speed of the characters at 2 times, or 1 times, 3 times, or even 4 times (the specific speed is determined by the degree of compression of the video time, for example, when the time requirement is half of the original, it is 2 times). However, when this method is applied to video playback control, it actually allocates the playback rate for each moment at a constant rate, so that the adaptive adjustment of the video playback speed cannot be achieved at all. In addition, this method has a better control effect on the playback rate of content with strong sports and dynamic effects, but for video content with more voice information such as movies and TV, the control effect of the playback rate is limited.

[0057] The third method is to identify the key segments of the video through video understanding and NPL processing, and then set different rates for each segment. This method usually requires text information to guide the identification of key segments. For example, Figure 3 This is a schematic diagram of controlling the video playback rate by identifying key segments. Figure 3, the text information 3-1 includes "spread butter on the bread slices 3-11", "put the buttered bread slices into the pan 3-12", etc. In actual use, the text information 3-1 and the video frame sequence 3-2 are input into the network model of the background server 3-3, and the key segments are judged according to the network model, so as to inform the client 3-4 which are the key segments. The terminal accelerates the non-key segments, plays the key segments at the initial rate, and even slows down the more important segments, so as to control the playback rate of the video. However, this method is effective when the guiding text information describes the entity content. When the text information describes the semantics, plot, starring role, etc. of the video, or when there is no guiding text information, the effect of controlling the video playback rate will be poor.

[0058] From the above analysis, it can be seen that the control method of the playback rate in the related art has a low degree of adaptability to different types of videos, which makes the control effect of the video playback rate poor, and ultimately makes the effect of video playback control poor.

[0059] The embodiments of the present application provide an information playback control method, device, equipment and computer-readable storage medium, which can improve the effect of multimedia information playback control. The following describes an exemplary application of the information playback control device provided in the embodiments of the present application. The information playback control device provided in the embodiments of the present application can be implemented as a terminal or a server. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiments of the present application. Below, an exemplary application of the information playback control device will be described.

[0060] See also Figure 4 , Figure 4 It is an optional architecture diagram of the information playback control system 100 provided in an embodiment of the present application. To support an information playback control application, the information playback control device 400 is connected to the server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two.

[0061] The server 200 is used to provide multimedia information data to the information playing control device 400, and the information playing control device 400 is used to receive the multimedia information data, play the multimedia information, and control the playing of the multimedia information based on the user's operation.

[0062] When the information playback control device 400 detects a trigger operation on the rate control mark of the playback interface, the rate options are displayed in response to the trigger operation on the rate control mark. Among them, the rate options include intelligent adjustment options, and the intelligent adjustment options are used to trigger dynamic rate adjustment of multimedia information. When the information playback control device 400 detects a trigger operation on the intelligent adjustment option, the target rate ratio of the multimedia information is determined based on the content information of the multimedia information in response to the trigger operation on the intelligent adjustment option, wherein the target rate ratio is dynamic. Then, the information playback control device 400 uses the target rate ratio to adjust the playback rate of the multimedia information to obtain the adjusted multimedia information. Finally, the information playback control device 400 will play the multimedia information. For example, when the multimedia information is a video, the information playback control device presents the adjusted video in the video playback area of ​​the video playback interface.

[0063] See also Figure 5 , Figure 5 is a structural diagram of an information playback control device 400 provided in an embodiment of the present application, Figure 5 The information playback control device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the information playback control device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 5 Various buses are labeled as bus system 440 .

[0064] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0065] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0066] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0067] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0068] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0069] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0070] A network communication module 452, for reaching other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (Wi-Fi), and Universal Serial Bus (USB);

[0071] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0072] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0073] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 5 An information playback control device 455 stored in a memory 450 is shown, which may be software in the form of a program or plug-in, including the following software modules: an information display module 4551, a rate ratio determination module 4552, and a rate adjustment module 4553. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0074] The functions of each module will be described below.

[0075] In other embodiments, the information playback control device provided in the embodiments of the present application can be implemented in hardware. As an example, the information playback control device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the information playback control method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0076] Exemplarily, an embodiment of the present application provides an information playback control device, including:

[0077] A memory, used for storing executable information playback control instructions;

[0078] The processor is used to implement the information playback control method provided in the embodiment of the present application when executing the executable information playback control instruction stored in the memory.

[0079] Below, the information playback control method provided by the embodiment of the present application will be described in combination with the exemplary application and implementation of the information playback control device provided by the embodiment of the present application.

[0080] See also Figure 6 , Figure 6 This is an optional flow diagram of the information playback control method provided in the embodiment of the present application. Figure 1 , will combine Figure 6 The steps shown are explained.

[0081] S101: When a trigger operation is detected on a rate control mark on a playback interface, rate options are displayed in response to the trigger operation on the rate control mark.

[0082] The embodiment of the present application is implemented in a scenario where multimedia information, such as video and audio, is played and controlled based on user operations. The information playback control device must first detect in real time whether the user has a trigger operation on the playback interface. After detecting the trigger operation, it is also necessary to determine in which area or on which mark the user has performed the trigger operation, so as to determine what function the user wants to achieve based on the location where the trigger operation occurs. When the information playback control device detects the user's trigger operation on the rate control mark of the playback interface, it will be considered that the user has a need to adjust the playback speed of the multimedia information, thereby creating a window for displaying rate options, and displaying different rate options in this window, so that the user can choose according to their own needs.

[0083] It should be noted that the rate options in the embodiments of the present application include an intelligent adjustment option, which is used to trigger dynamic rate adjustment of multimedia information. In other words, when the user does not want to adjust the playback rate of multimedia information according to a constant ratio, this option can be used to achieve adaptive multimedia information playback rate control. Of course, in other embodiments, the rate options may also include some fixed rate options, such as 2x speed, 1.5x speed, etc., which are not limited in the present application.

[0084] It is understandable that the play interface is displayed when the user triggers the multimedia information to be played. The multimedia information may be multimedia data that has been cached in the information play control device, or multimedia data that is received in real time by the information play control device, which is not limited in this application.

[0085] It should be noted that the rate control logo can be set in the first preset area of ​​the playback interface, wherein the size and position of the first preset area can be set according to actual conditions. For example, it can be set at the bottom of the playback interface with a size of 100×100 so as not to affect the user's viewing of multimedia information, or it can be set in the upper right corner of the playback interface with a size of 50×50.

[0086] Of course, the rate control mark may also appear after the user operates the playback interface. For example, the original playback interface only has multimedia information to be played, and the rate control mark appears at the bottom of the playback interface after the user clicks on the playback interface.

[0087] Similarly, the rate option is set in the second preset area of ​​the playback interface, and the size and position of the second preset area can also be set according to actual conditions, for example, it is set around the rate control mark and the size is set to 200×200, or it is set in the center of the playback interface and the size is set to 100×100. In some embodiments, the transparency of the rate option can also be set as required, so that the rate option can not block the video or audio being played.

[0088] It is understandable that the triggering operation for the rate control mark may be a click, a double click, or a long press by the user, etc. Of course, in some embodiments of the present application, the triggering operation for the rate control mark may also be the user's voice, for example, the user saying "help me adjust the video playback speed" or the user's regular high-five sound.

[0089] For example, the present application embodiment provides a schematic diagram of display rate options, see Figure 7 At the bottom of the playback interface 7-1, a rate control mark 7-11 is displayed. When the user clicks the rate control mark 7-11, a rate option 7-12 pops up around the rate control mark 7-11. The rate option 7-12 includes an intelligent adjustment option: intelligent control 7-13, and other options, such as 2x speed, 1.5x speed, and other options.

[0090] S102: When a trigger operation is detected on the smart adjustment option, in response to the trigger operation on the smart adjustment option, a target rate ratio of the multimedia information is determined based on content information of the multimedia information.

[0091] When the information playback control device detects the user's trigger operation on the intelligent adjustment option, it will be clear that the user has specified the dynamic rate adjustment function for the multimedia information being played. As a result, the information playback control device will analyze the content information contained in the multimedia information to determine which parts of the multimedia information contain specific content or content that the user is interested in. Based on the specific content or the content that the user is interested in, it is determined whether the multimedia information should be accelerated or decelerated, and the acceleration ratio or the deceleration ratio is used as the target rate ratio of the multimedia information.

[0092] It should be noted that it is possible that not all content information of the multimedia information contains specific content, or contains content that is important and interesting to the user. Therefore, the information playback control device segments the multimedia information according to whether the specific content is included, or whether the important content or the content of interest is included, so as to determine the appropriate rate ratio for adjusting the playback rate of the multimedia information for different segments. In other words, the target rate ratio determined by the information playback control device of the embodiment of the present application for the multimedia information is dynamic, that is, it will change, and it is not a constant value. It can be composed of rate ratios corresponding to different time periods, so as to realize dynamic adjustment of the rate based on the content of the multimedia information.

[0093] In the embodiment of the present application, the content information of the multimedia information includes but is not limited to image information, audio information, text information, and semantic information. Among them, the image information can be the characters, animals, objects, etc. that appear, the audio information can be music, sound effects, etc., the text information can be the lines displayed on the screen, the lyrics of the audio, etc., and the semantic information can be the plot, the interaction of the characters, etc. All of the above information can be used to control the playback rate of the multimedia information, for example, to determine whether the information contains specific content, and if so, to speed up the video, or to determine whether the information contains information that the user is interested in, and if there are lyrics that the user is interested in, to slow down the audio, etc.

[0094] It is understandable that specific content can be specified by the user or by the publisher of the multimedia information; information that is important and interesting to the user can be analyzed in combination with the user portrait.

[0095] S103: Using the target rate ratio, adjust the playback rate of the multimedia information to obtain adjusted multimedia information.

[0096] After obtaining the target rate ratio, the information playback control device will use the target rate ratio to determine whether different parts of the multimedia information should be accelerated or decelerated, and specifically determine the acceleration ratio and the deceleration ratio, and then adjust the playback rate of each part according to the determined acceleration or deceleration ratio. For example, when it is determined that a part of the video without lines needs to be accelerated according to a rate ratio of 1.5 times, the information playback control device will accelerate this part of the content, thereby obtaining an adjusted video so that this part of the content can be played faster.

[0097] Exemplarily, the present application embodiment provides a schematic diagram of the adjusted multimedia information, see Figure 8The multimedia information is a video. The information playback control device determines the target rate ratio based on the content information of the video. The target rate ratio will fluctuate between 1x and 4x at different times. Therefore, the adjusted video playback rate will dynamically change from 1x to 4x the original playback rate.

[0098] S104: Play the adjusted multimedia information.

[0099] When the multimedia information is a video, the information playback control device presents the adjusted video in the playback area; when the multimedia information is an audio, the information playback device can play the adjusted audio in the background or in the playback interface, at which time, the playback interface can display the lyrics, sound effects, etc. of the audio. Since the adjusted multimedia information is adjusted according to a dynamic target rate ratio determined by specific content or user interests, the adjusted multimedia information can highlight specific content and content that is important and interesting to the user, and thus the multimedia information playback rate can be dynamically controlled to make the multimedia information playback control more intelligent.

[0100] It can be understood that the visual playback area is set in the third preset area of ​​the playback interface, wherein the size and position of the third preset area can be set according to actual conditions. For example, the third preset area is set in the center of the playback interface, and the size is 90% of the playback interface, or the third preset area is set in the left half of the playback interface, and the size is set to 50% of the playback interface.

[0101] Exemplarily, the present application embodiment provides a schematic diagram of presenting the adjusted multimedia information. Figure 1 , see FIG9(a), the multimedia information is a video, and a video playback area 9-11 is provided in the playback interface 9-1, in which the adjusted video is displayed, wherein the playback rate of the adjusted video varies between 1 times and 4 times the original playback rate. As shown in FIG9(b), the embodiment of the present application provides a schematic diagram of presenting the adjusted multimedia information. Figure 2 When the multimedia information is audio and the audio is a song, a lyrics display area 9-12 is provided in the playback interface 9-1, and the information playback control device can play the adjusted audio through a device such as headphones, and at the same time, the corresponding lyrics are displayed in the lyrics display area 9-12 at the adjusted audio playback rate. It can be understood that the audio can also be any audio program, such as storytelling, conversation, etc., and the playback rate can be dynamically adjusted according to the rhythm, language density, etc. of the story or conversation content.

[0102] In an embodiment of the present application, the information playback control device responds to a trigger operation for a rate control identifier, displays rate options including intelligent selection options, and then responds to a trigger operation for an intelligent adjustment option to analyze the content information of the multimedia information, determine a dynamic target rate ratio, and then uses the target rate ratio to adjust the playback rate of the multimedia information and display the adjusted multimedia information. In this way, the playback rate of the multimedia information can be dynamically controlled according to the content of the multimedia information, thereby making the control of the playback rate of the multimedia information more intelligent and improving the playback control effect of the multimedia information.

[0103] See also Fig.10 , Fig.10 This is an optional flow diagram of the information playback control method provided in the embodiment of the present application. Figure 2 In some embodiments of the present application, the content information of the multimedia information includes at least one sub-content information. In this case, based on the content information of the multimedia information, the target rate ratio of the multimedia information is determined, that is, the specific implementation process of S102 may include: S1021-S1023, as follows:

[0104] S1021. Divide the multimedia information into at least one information segment according to at least one sub-content information.

[0105] The sub-content information in the content information of the multimedia information may refer to information that can distinguish one segment of the multimedia information from another segment. For example, a star, a line, a plot, etc. Therefore, the information playback control device can segment the multimedia information according to different sub-information contents, so as to obtain the same number of information segments as the number of sub-information contents. It should be noted that at least one sub-content information and at least one information segment are in one-to-one correspondence, that is, there is a corresponding relationship between the sub-content information and the information segment.

[0106] For example, the information playback control device can divide the video into video segments according to different lines, and can also divide the video into video segments according to different actors. For another example, the information playback control device can divide the audio into audio segments according to the lyrics, or divide the audio into audio segments according to the comments and mark popularity of netizens.

[0107] S1022: Calculate the rate ratio of each information segment based on the sub-content information corresponding to each information segment after the multimedia information is segmented, and obtain the rate ratio corresponding to each information segment.

[0108] After obtaining at least one information segment, the information playback control device will analyze the sub-content information corresponding to each information segment to determine whether each information segment contains specific content and whether it is of interest to the user, thereby determining whether each information segment needs to be accelerated or decelerated for playback, and then use the ratio of acceleration or deceleration as the rate ratio corresponding to each information segment.

[0109] It should be noted that whether to accelerate or decelerate the information segment can be determined according to the magnitude relationship between the rate ratio and the preset value. For example, when the preset value is 1, if the rate ratio is less than 1, it means that the information segment is decelerated, and if the rate ratio is greater than 1, it means that the information segment is accelerated.

[0110] It is understandable that the specific content in the embodiment of the present application can be content that the user specifies to be watched carefully, or content that needs to be skipped, and then the information playback control device can determine the rate ratio according to the type specified by the user. For example, when the user specifies to watch a certain star carefully, the information segment containing the star needs to be played normally or even at a reduced speed, and the rate ratio is determined according to the degree of careful viewing of the star specified by the user.

[0111] Similarly, when the information segment contains content that the user is interested in according to the user profile analysis, the rate ratio can be determined according to the analyzed degree of interest of the user. For example, if the user is very interested, the rate ratio is determined to be 0.5, and if the user is not interested, the rate ratio is set to 3.

[0112] S1023. Determine a target rate ratio of the multimedia information according to the rate ratio corresponding to each information segment.

[0113] After the information playback control device calculates the rate ratio corresponding to each information segment, it arranges the rate ratio of each information segment in the order of the information segments, and the sequence formed is the target rate ratio of the multimedia information.

[0114] For example, when there are three information segments in the multimedia information, and their rate ratios are 0.5, 1, and 1.5 respectively, then the target rate ratio can be expressed as [0.5, 1, 1.5].

[0115] In the embodiment of the present application, the information playback control device can divide the multimedia information into individual information segments according to different sub-content information in the content information of the multimedia information, and then calculate the rate ratio corresponding to each information segment according to the sub-content information corresponding to each information segment, so as to obtain the target rate ratio of the multimedia information by using the rate ratio corresponding to each information segment. In this way, the target rate ratio of the multimedia information can be made variable.

[0116] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: text information. In this case, based on the sub-content information corresponding to each information segment after the multimedia information is segmented, the rate ratio is calculated for each information segment to obtain the rate ratio corresponding to each information segment, that is, the specific implementation process of S1022 may include: S201-S202, as follows:

[0117] S201. Calculate the display rate of text information to obtain a text display rate.

[0118] When each information segment includes text information, the information playback control device can first count the number of characters in the text information of each information segment, and then subtract the start time and end time of the text information display to obtain the time difference, and then use the number of characters divided by the time difference to obtain the text display rate of each information segment.

[0119] It should be noted that when the sub-content information includes text information, it is actually each continuous text information that is used to divide the multimedia information into different information segments. Therefore, the text display rate at any time in an information segment should be the same as the text display rate calculated based on the start and end time of the information segment. It is understandable that when there is no text information in an information segment, the text display rate of the information segment is 0.

[0120] Exemplarily, the present application embodiment provides a formula for calculating the text display rate, see formula (1), as follows:

[0121]

[0122] Among them, T s is the time when the text information starts to be displayed, T e is the time when the text message ends displaying, t s and t e It is T s 、T e Any time between NumWords(T s ,T e ) is the number of characters in the text information, WordRate(T s ,T e ) is the text display rate, WordRate(t s ,t e ) is the text display rate of any time period in each information segment. After the information playback control device learns the values ​​of the above parameters, it substitutes the values ​​of these parameters into formula (1) to calculate the text display rate.

[0123] For example, Table 1 is an example of text display rates of some subtitles:

[0124]

[0125] As can be seen from Table 1, the pause durations in the subtitles "What happens in the future?", "You have to tell your mother," and "You can't act on your own" all exceed 0.5. This is because these subtitles are defined as different sentences, and the text display rate can be determined based on the number of words in these sentences.

[0126] It is understandable that text information can refer to subtitles in a video, comments from netizens, or lyrics of audio, etc., and the embodiments of the present application are not limited here. Furthermore, different text information is defined according to the pause time when the text information is displayed. For example, when the pause time between one paragraph and another paragraph in the subtitle exceeds the time threshold, the two text information will be regarded as two different text information. Subtitles, comments from netizens, etc.

[0127] S202: Calculate the adjustment ratio of each information segment using the text display rate, the preset text rate and the preset maximum rate ratio to obtain the rate ratio corresponding to each information segment.

[0128] After obtaining the text display rate, the information playback control device will obtain the preset text rate and the preset maximum rate ratio, and then calculate the rate adjustment ratio for each information segment based on the relationship between the three dimensions of text display rate, preset text rate and preset maximum rate ratio, so as to obtain the rate ratio corresponding to each information segment.

[0129] In some embodiments of the present application, the information playback control device may merge the text display rate and the preset text rate, and then subtract the merged rate ratio from the preset maximum rate ratio to obtain the rate ratio corresponding to each information segment.

[0130] In an embodiment of the present application, the information playback control device can first calculate the text display rate of the text information included in the sub-content information corresponding to each information segment, and then determine the rate ratio of each information segment based on the text display rate, so that the rate ratio can be determined for each information segment based on the text information in each information segment.

[0131] In some embodiments of the present application, the text display rate, the preset text rate, and the preset maximum rate ratio are used to calculate the adjustment ratio for each information segment to obtain the rate ratio corresponding to each information segment, that is, the specific implementation process of S202 may include: S2021-S2022, as follows:

[0132] S2021. Compare the text display rate with a preset text rate to obtain a rate ratio calculation factor.

[0133] S2022. Perform difference calculation on the preset maximum rate ratio and the rate ratio calculation factor to obtain the rate ratio corresponding to each information segment.

[0134] The information playback control device first uses the text display rate as the numerator and the preset text rate as the denominator to perform a ratio operation, and records the obtained ratio as the rate ratio calculation factor. After that, the information playback control device uses the preset maximum rate ratio to subtract the rate ratio calculation factor, and the difference obtained is the rate ratio corresponding to each information segment.

[0135] It is understandable that the preset text rate represents a baseline speech rate that does not need to be adjusted, that is, the original rate of the multimedia information. The preset text rate can be set based on experience or actual needs. For example, the preset text rate is set between 7-10 (i.e., 7-10 words are displayed per second), or can be set to 5. This application does not limit this.

[0136] Similarly, the preset maximum rate ratio is also set according to actual conditions, for example, it can be set to 2 or 3. It should be noted that the rate ratio calculated by the information playback control device in the embodiment of the present application is always smaller than the preset maximum rate ratio.

[0137] Exemplarily, the method for calculating the rate ratio provided in the embodiment of the present application is as shown in formula (2):

[0138]

[0139] Among them, WordRate(t s ,t e ) is the text display rate, N is the preset text rate, 2 is the preset maximum rate ratio, SpeedRate(t s ,t e ) is the rate ratio of each information segment. From formula (2), we can see that 1≤SpeedRate(t s ,t e )≤2.

[0140] In an embodiment of the present application, the information playback control device can first compare the text display rate with the preset text display rate to obtain a rate ratio calculation factor, and then combine the rate ratio calculation factor, the preset text rate and the preset maximum rate ratio to calculate the rate ratio of each information segment, so as to obtain the target rate ratio later.

[0141] In some embodiments of the present application, based on the sub-content information corresponding to each information segment in at least one information segment, a rate ratio is calculated for each information segment to obtain a rate ratio corresponding to each information segment, that is, a specific implementation process of S1022 may include: S203-S204, as follows:

[0142] S203: Extract, from a plurality of preset rate ratios, a matching rate ratio that matches the sub-content information corresponding to each information segment in at least one information segment.

[0143] The information playback control device matches each sub-content information with a plurality of preset rate ratios, so that when the preset rate ratio set for each sub-content information is matched, the preset rate ratio is used as the matching rate ratio.

[0144] It should be noted that, in the embodiment of the present application, the sub-content information corresponding to each information segment includes at least any one of the following: object information, audio information, and video semantic information.

[0145] S204: Use the matching rate ratio as the rate ratio corresponding to each information segment.

[0146] The information playback control device uses the extracted matching rate ratio as the rate ratio corresponding to each information segment, so that the information playback control device can determine the corresponding rate ratio for each information segment.

[0147] In the embodiment of the present application, the information playback control device can determine a rate ratio for each information segment according to the sub-content information contained in each information segment, so as to determine a dynamic target rate ratio for the multimedia information according to the set rate ratio.

[0148] Further, in some embodiments of the present application, the sub-content information corresponding to each information segment includes: object information, based on the sub-content information corresponding to each information segment in at least one information segment, a rate ratio is calculated for each information segment to obtain a target rate ratio corresponding to each information segment, that is, a specific implementation process of S1022 may include: S205-S206, as follows:

[0149] S205: Extract a preset rate ratio corresponding to the object information from a plurality of preset rate ratios.

[0150] S206: Use the preset rate ratio corresponding to the object information as the rate ratio corresponding to each information segment.

[0151] In the embodiment of the present application, multiple preset rate ratios may be pre-set, and each preset rate ratio may correspond to a type of sub-content information. When the sub-content information corresponding to each information segment is object information, the information playback control device may extract the preset rate ratio corresponding to the object information from the multiple preset rate ratios based on the object information. In this case, the extracted preset rate ratio is the rate ratio corresponding to each information segment.

[0152] It is understandable that the object information may be information about a person, an animal or even an object in the information segment. For example, when the object information is a well-known actor, the preset rate ratio corresponding to the well-known actor may be determined as the rate ratio of the information segment.

[0153] In an embodiment of the present application, the information playback control device selects a preset rate ratio corresponding to the object information from multiple preset rate ratios based on the object information included in the sub-content information, as the rate ratio corresponding to each information segment, so that the information playback control device can determine the rate ratio for each information segment from the image dimension.

[0154] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: audio information, and based on the sub-content information corresponding to each information segment in at least one information segment, a rate ratio is calculated for each information segment to obtain a rate ratio corresponding to each information segment, that is, a specific implementation process of S1022 may include: S207-S209, as follows:

[0155] S207: Identify the audio beat from the audio information.

[0156] The information playing control device identifies the melody beat of the audio information to obtain the audio beat of the audio information, that is, the audio beat of each information segment.

[0157] It is understandable that the information playback control device can use the voice recognition function to identify the name corresponding to the audio information, and then search for the audio beat on the Internet based on the name. The information playback control device can also compare the melody in the audio information with the melody in the Internet music library, so as to directly search for the matching melody and use the beat of the melody as the audio beat of each information segment.

[0158] S208: Extract a preset rate ratio corresponding to the audio beat from a plurality of preset rate ratios.

[0159] S209: Using the preset rate ratio corresponding to the audio beat as the rate ratio corresponding to each information segment.

[0160] After obtaining the audio beat of each information segment, the information playback control device extracts the preset rate ratio corresponding to the audio beat from multiple preset rate ratios, and uses the extracted rate ratio as the rate ratio corresponding to each information segment.

[0161] Exemplarily, when the information playback control device recognizes that the name of the audio information is "Twinkle Twinkle Little Star", the melody beat of "Twinkle Twinkle Little Star" can be used as the audio beat of the information segment, and then the preset rate ratio corresponding to "Twinkle Twinkle Little Star" can be selected from multiple preset rate ratios. For example, the selected acceleration ratio is 2, and this ratio is used as the rate ratio of each information segment.

[0162] In an embodiment of the present application, when the sub-content information of each information segment includes audio information, the information playback control device can first identify the audio beat of the audio information, and then extract the preset rate ratio corresponding to the audio beat, and use the extracted rate ratio as the rate ratio of the information segment, thereby determining the rate ratio for each information segment based on the audio dimension.

[0163] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: video semantic information. At this time, based on the sub-content information corresponding to each information segment in at least one information segment, the rate ratio is calculated for each information segment to obtain the rate ratio corresponding to each information segment, that is, the specific implementation process of S1022 may include: S2010-S2011, as follows:

[0164] S2010: Extracting a preset rate ratio corresponding to video semantic information from a plurality of preset rate ratios.

[0165] S2011. Using the preset rate ratio corresponding to the video semantic information as the rate ratio corresponding to each information segment.

[0166] The information playback control device extracts a preset rate ratio corresponding to the video semantic information from a plurality of preset rate ratios according to the video semantic information, and uses the extracted preset rate ratio as the rate ratio corresponding to each information segment.

[0167] Exemplarily, when the video semantic information indicates that the plot of the information segment is "identifying the true face of the bad guy", the rate ratio of the plot can be extracted from multiple preset rate ratios, for example, 1 can be extracted, and the rate ratio can be used as the rate ratio of the information segment.

[0168] It should be noted that in the embodiment of the present application, video semantic information can be identified by a video understanding algorithm. Video semantic information can refer to the plot of the video, or the interaction between the characters in the video, the scene of the video, etc., which is not limited in the present application.

[0169] In an embodiment of the present application, the information playback control device can determine a rate ratio for each information segment based on video semantic information, thereby being able to determine a rate ratio for the information segment based on the dimension of the video sentence.

[0170] In some embodiments of the present application, the multimedia information includes at least one information segment, and the target rate ratio includes a rate ratio corresponding to each information segment; the target rate ratio is used to adjust the playback rate of the multimedia information to obtain the adjusted multimedia information, that is, the specific implementation process of S103 may include: S1031-S1032, as follows:

[0171] S1031. According to the rate ratio corresponding to each information segment, the playback rate of each information segment is adjusted to obtain an adjusted information segment corresponding to each information segment.

[0172] When the information playback control device controls the playback of multimedia information, it extracts the rate ratio corresponding to each information segment from the target rate ratio, and then uses each information segment to adjust the playback rate according to the corresponding rate ratio, so that the playback rate of each information segment is accelerated or slowed down, and the adjusted information segment corresponding to each information segment is obtained.

[0173] S1032: Use the adjusted information segments corresponding to each information segment to splice out the adjusted multimedia information.

[0174] Next, the information playing control device splices all the adjusted information segments together again according to the order of the information segments, so that the spliced ​​multimedia information is the adjusted multimedia information.

[0175] In an embodiment of the present application, the information playback control device can adjust the playback rate of each information segment according to the rate ratio corresponding to each information segment, and finally use the adjusted information segments to form adjusted multimedia information. In this way, dynamic adjustment of the playback rate of the multimedia information is achieved.

[0176] In some embodiments of the present application, after calculating the rate ratio of each information segment based on the sub-content information corresponding to each information segment after the multimedia information is segmented to obtain the rate ratio corresponding to each information segment, that is, after S1022, the method may further include: S1024, as follows:

[0177] S1024. Smoothing the rate ratio corresponding to each information segment to obtain a smoothed rate ratio corresponding to each information segment.

[0178] Since the difference between the calculated rate ratios of each information segment may be large, it may cause a sudden change in the playback rate, causing discomfort to the user. Therefore, in the embodiment of the present application, after calculating the rate ratio corresponding to each information segment, the information playback control device will continue to smooth the rate ratio corresponding to each information segment, thereby obtaining the smoothed rate ratio corresponding to each information segment.

[0179] For further information, see Fig.11 , Fig.11 This is an optional flow diagram of the information playback control method provided in the embodiment of the present application. Figure 3 In some embodiments of the present application, after smoothing the rate ratio corresponding to each information segment to obtain the smoothed rate ratio corresponding to each information segment, that is, after S1024, the method may further include: S1023a, as follows:

[0180] S1023a: Use the smoothed rate ratio corresponding to each information segment to splice out a target rate ratio of the multimedia information.

[0181] In this way, the target rate ratio of the multimedia information can be changed more smoothly, so that the playing of the multimedia information will not have a sudden change in speed, which will affect the control effect of the multimedia information.

[0182] For example, the present application embodiment provides a smoothing effect diagram, see Fig.12 , the horizontal axis is time (ranging from 0-178s), the vertical axis is rate ratio (ranging from 1-2), the dotted line is the target rate ratio obtained without smoothing, it can be seen that due to the sudden change in the rate between different information segments, many spikes are caused, which will affect the user experience during the playback of multimedia information. The solid line is the target rate ratio obtained after smoothing, it can be seen that compared with the dotted line, that is, the target rate ratio obtained without smoothing, the solid line has fewer spikes, the playback effect will be smoother, and thus improve the playback effect.

[0183] In an embodiment of the present application, the information playback control device can also smooth the rate ratio corresponding to each information segment to obtain a smoothed rate ratio, and then use the smoothed rate ratio to form a target rate ratio of the multimedia information, so that the target rate ratio will not change suddenly, thereby improving the playback effect.

[0184] In some embodiments of the present application, each information segment is actually composed of smaller time periods, that is, each information segment includes multiple time periods, so that the rate ratio of each information segment is composed of the sub-rate ratios corresponding to each time period in each information segment. In this case, the rate ratio corresponding to each information segment is smoothed to obtain the smoothed rate ratio corresponding to each information segment, that is, the specific implementation process of S1024, which may include: S301-S305, as follows:

[0185] S301. Calculate the difference between the rate ratio corresponding to each information segment and the rate ratio corresponding to the previous information segment of each information segment to obtain the rate ratio difference.

[0186] Before smoothing each information segment, the information playback control device needs to first determine the difference between the rate ratio of each information segment and the corresponding previous information segment, so as to determine how to smooth. Therefore, the information playback control device will first subtract the rate ratio corresponding to each information segment from the rate ratio corresponding to the previous information segment, so as to obtain the rate ratio difference between each information segment and the corresponding previous information segment.

[0187] S302: Compare the rate ratio difference with the number of time periods to obtain a rate ratio change range of each time period.

[0188] The number of time periods represents the number of time periods in each information segment.

[0189] Since each information segment is composed of smaller time periods, for example, a 1-minute information segment can be regarded as composed of 6 small information segments corresponding to 10-second time periods, and the smoothing process can actually be based on allocating the rate ratio difference to each time period corresponding to each information segment, so that in each information segment, the rate ratio is gradually adjusted instead of changing suddenly, thereby avoiding user discomfort. Therefore, in the embodiment of the present application, the rate ratio difference can be compared with the total number of time periods in the information segment to obtain the rate ratio change range that each time period in the information segment should bear.

[0190] It is understandable that the time period may be 10s, 1s, or other values ​​set according to actual conditions, and the embodiments of the present application are not limited thereto.

[0191] For example, when the rate ratio difference between a 5s information segment and its previous information segment is 2, the rate ratio changes by 0.4 per second in this information segment, i.e., the rate ratio increases from the previous information segment to this information segment in increments of 0.4.

[0192] S303: The sub-rate ratio of each time period is obtained by adding the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period.

[0193] S304: When the calculation of the sub-rate ratio for each time period is completed, the sub-rate ratio of each time period in each information segment is obtained.

[0194] The information playback control device adds the sub-rate ratio of the previous time period of each time period and the change amplitude of the rate ratio to be allocated for the rate ratio of each time period, and can calculate the sub-rate ratio of each time period. When the information playback control device completes the calculation of the sub-rate ratio for all time periods, the sub-rate ratio of each time period can be obtained, and the allocation of the rate ratio difference is completed.

[0195] It should be noted that when a time period is the first time period in an information segment, the sub-rate ratio of the previous time period is actually equal to the rate ratio of the previous information segment.

[0196] S305: Use the sub-rate ratios of each time period to splice out a smoothed rate ratio corresponding to each information segment.

[0197] Finally, the information playing control device aggregates the sub-rate ratios of each time period in the order of the time period, and the obtained aggregate is the smoothed rate ratio of each information segment.

[0198] In an embodiment of the present application, the information playback control device can calculate the rate ratio of each information segment and the rate ratio of the corresponding previous information segment, and then distribute the rate ratio difference among the various time periods of each information segment, so that the rate ratio can be adjusted gradually instead of changing suddenly, thereby avoiding sudden changes in the playback rate causing user discomfort and improving the playback control effect.

[0199] In some embodiments of the present application, the rate ratio difference is compared with the number of time periods to obtain the rate ratio change amplitude of each time period in each time period, that is, the specific implementation process of S302 may include: S3021 or S3022, as follows:

[0200] S3021. When the ratio of the rate ratio difference to the number of time periods is less than or equal to a preset variation range, the ratio of the rate ratio difference to the number of time periods is determined as the rate ratio variation range of each time period.

[0201] In an embodiment of the present application, certain conditions can also be set for the rate ratio change amplitude. For example, in order to avoid excessively rapid changes in the sub-rate ratio of a time period within an information segment, a condition can be set for the sub-rate ratio that the growth rate must not be greater than a certain threshold. Thus, after the information playback control device calculates the ratio of the rate ratio difference and the number of time periods, it first compares the ratio with a preset change amplitude. When the ratio is less than or equal to the preset change amplitude, the ratio is used as the rate ratio change amplitude.

[0202] It is understandable that the preset variation range may be set to 0.3, or 0.5, or may be set to other values ​​according to actual needs, and this application does not limit this.

[0203] S3022: When the ratio of the rate ratio difference to the number of time periods is greater than a preset change range, the preset change range is determined as the rate ratio change range of each time period.

[0204] When the ratio of the calculated rate ratio difference to the number of time periods is greater than the preset change range, the information playback control device will directly use the preset change range as the rate ratio change range for each time period, thereby avoiding excessive rate ratio change range for each time period.

[0205] In an embodiment of the present application, the information playback control device can compare the ratio of the rate ratio difference and the number of time periods with a preset change range, thereby determining the rate ratio change range for each time period to avoid excessive rate ratio change range for each time period.

[0206] In some embodiments of the present application, the sub-rate ratio of each time period is obtained by adding the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period, that is, the specific implementation process of S303 may include: S3031-S3034, as follows:

[0207] S3031. Add the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period to obtain a sum result.

[0208] S3032. For each time period, extract a preset number of associated time periods, and use the sub-rate ratios corresponding to each of the preset number of associated time periods as the preset number of associated sub-rate ratios corresponding to the sub-rate ratio of each time period.

[0209] The information playback control device extracts a preset number of associated time periods for each time period according to the associated time period selection strategy, and extracts the sub-rate ratios corresponding to each of these associated time periods as the associated sub-rate ratios of each time period. In this way, the information playback control device can obtain a preset number of associated sub-rate ratios.

[0210] It should be noted that the associated time period selection strategy may refer to selecting the adjacent time periods before and after each time period, for example, the first two time periods and the last two time periods of each time period, or may refer to selecting only a number of time periods after each time period, for example, the last five time periods of each time period. Of course, the associated time period selection strategy may also be set to other strategies according to actual needs, and this application does not limit this.

[0211] It is understandable that the preset number can be set according to actual conditions, for example, set to 5, set to 10, etc., and this application does not limit it here.

[0212] S3033. Extract the largest correlation sub-rate ratio from a preset number of correlation sub-rate ratios.

[0213] S3034. When the sum result is less than or equal to the maximum associated sub-rate ratio, the sum result is used as the sub-rate ratio of each time period.

[0214] Afterwards, the information playback control device compares the preset number of associated sub-rate ratios to determine the maximum associated sub-rate ratio, extracts the maximum associated sub-rate ratio, and compares it with the previously calculated sum value result. When the sum value result is less than or equal to the maximum associated sub-rate ratio, the information playback control device uses the sum value result as the sub-rate ratio corresponding to each time period, thereby avoiding the sub-rate ratio of each time period being too large, causing discomfort to the user.

[0215] In some embodiments of the present application, when the sum result is greater than the maximum associated sub-rate ratio, the information playback control device will use the maximum associated sub-rate ratio as the sub-rate ratio of each time period to avoid the sub-rate ratio of each time period being too large.

[0216] In an embodiment of the present application, the information playback control device can first extract the maximum associated sub-rate ratio for the sub-rate ratio corresponding to each time period, and then compare the sum of the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period with the maximum associated sub-rate ratio, so as to determine whether the sum result can be used as the sub-rate ratio of each time period, so as to avoid the sub-rate ratio of each time period being too large.

[0217] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0218] The embodiment of the present application is implemented in the scenario of intelligent acceleration of the video played by the video playback application. The terminal (information playback control device) can use the video content information (content information of the multimedia information), mainly the time period when the specific content appears, for example, the line information (text information) refers to the time range when each line appears, the character information (object information) refers to the time period when each character appears, and so on.

[0219] The embodiment of the present application takes lines as an example to illustrate the process of intelligent acceleration. The source of line information can be soft subtitles, OCR recognition, speech-to-text conversion, etc.

[0220] First, the terminal needs to calculate the speech rate (text display rate) of the lines in each video (information segment). Here, the terminal can use formula (1) to calculate the speech rate. When there is no line, the speech rate is 0.

[0221] Next, the terminal gives a relatively large speed-up ratio (rate ratio) for the case where there is no dialogue or the dialogue is slow. The speed-up ratio can be calculated according to formula (2). Generally, the speed-up ratio is limited to 1-2 times.

[0222] Finally, since the calculated acceleration ratios of each video segment vary greatly, a sudden change in the acceleration ratio will cause a sudden change in the playback speed, which will cause discomfort to the user. Therefore, the terminal needs to use a smoothing strategy to make the change in playback speed more natural. The terminal can calculate the smoothed rate (the smoothed rate ratio corresponding to each video segment) based on the original rate (the rate ratio of each video segment) and apply the following strategy:

[0223] 1. The smoothing rate is not higher than the highest rate (maximum correlation sub-rate ratio) in the last five seconds (preset number of correlation time periods);

[0224] 2. The increase in the smoothing rate per second (the ratio of the rate difference to the number of time periods) shall not exceed 0.3 (the preset change range).

[0225] Through the smoothing strategy, the terminal can avoid drastic acceleration and deceleration to ensure smooth playback.

[0226] The accelerated speed of the video can be calculated in real time when the terminal reads the dialogue data, or it can be processed offline in advance, that is, the server determines the speed for each video segment and sends it to the terminal, which can then read and play it.

[0227] At this point, intelligent acceleration based on the lines has been completed.

[0228] For some other scenes, intelligent acceleration can be performed according to the following strategies. For example, a fixed acceleration ratio can be set for scenes where stars appear (the preset rate ratio corresponding to the object information is extracted as the rate ratio corresponding to each video segment); scenes with music can be intelligently accelerated according to the beat (the preset rate ratio corresponding to the audio beat of the audio information is used as the rate ratio corresponding to each video segment); based on semantics (video semantic information), a specific rate can be assigned to each scene (the preset rate ratio corresponding to the video semantic information is used as the rate ratio corresponding to each video segment).

[0229] Through the above method, when the video playback application plays a video, the acceleration ratio can be calculated based on the video content, so that the video acceleration is changed from conventional fixed acceleration to dynamically adjusted acceleration, thereby improving the effect of video playback control and improving user experience.

[0230] It is understandable that in the embodiments of the present application, related data such as user portraits are involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0231] The following is a description of an exemplary structure of the information playback control device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 5 As shown, the software modules stored in the information playback control device 455 of the memory 450 may include:

[0232] The information display module 4551 is used for displaying rate options in response to the trigger operation on the rate control mark when a trigger operation is detected on the rate control mark of the playback interface; wherein the rate options include intelligent adjustment options, and the intelligent adjustment options are used to trigger dynamic rate adjustment of multimedia information;

[0233] a rate ratio determination module 4552, configured to determine a target rate ratio of the multimedia information based on content information of the multimedia information in response to a trigger operation for the intelligent adjustment option when a trigger operation is detected on the intelligent adjustment option; wherein the target rate ratio is dynamic;

[0234] The rate adjustment module 4553 is used to adjust the playback rate of the multimedia information by using the target rate ratio to obtain the adjusted multimedia information;

[0235] The information display module 4551 is also used to play the adjusted multimedia information.

[0236] In some embodiments of the present application, the content information of the multimedia information includes at least one sub-content information; the rate ratio determination module 4552 is further used to divide the multimedia information into at least one information segment based on the at least one sub-content information; based on the sub-content information corresponding to each information segment after the multimedia information is divided, calculate the rate ratio of each information segment to obtain the rate ratio corresponding to each information segment; and determine the target rate ratio of the multimedia information based on the rate ratio corresponding to each information segment.

[0237] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: text information; the rate ratio determination module 4552 is also used to calculate the display rate of the text information to obtain the text display rate; using the text display rate, the preset text rate and the preset maximum rate ratio, the adjustment ratio of each information segment is calculated to obtain the rate ratio corresponding to each information segment.

[0238] In some embodiments of the present application, the rate ratio determination module 4552 is further used to compare the text display rate with the preset text rate to obtain a rate ratio calculation factor; and to perform a difference calculation on the preset maximum rate ratio and the rate ratio calculation factor to obtain the rate ratio corresponding to each information segment.

[0239] In some embodiments of the present application, the rate ratio determination module 4552 is further used to extract a matching rate ratio that matches the sub-content information corresponding to each information segment from a plurality of preset rate ratios; and use the matching rate ratio as the rate ratio corresponding to each information segment.

[0240] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: object information; the rate ratio determination module 4552 is further used to extract the preset rate ratio corresponding to the object information from multiple preset rate ratios; and use the preset rate ratio corresponding to the object information as the rate ratio corresponding to each information segment.

[0241] In some embodiments of the present application, the rate ratio determination module 4552 is further used to extract a matching rate ratio that matches the sub-content information corresponding to each information segment in the at least one information segment from a plurality of preset rate ratios; and use the matching rate ratio as the rate ratio corresponding to each information segment.

[0242] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: audio information; the rate ratio determination module 4552 is also used to identify audio beats from the audio information; extract the preset rate ratio corresponding to the audio beat from multiple preset rate ratios; and use the preset rate ratio corresponding to the audio beat as the rate ratio corresponding to each information segment.

[0243] In some embodiments of the present application, the sub-content information corresponding to each information segment includes: video semantic information; the rate ratio determination module 4552 is further used to extract the preset rate ratio corresponding to the video semantic information from multiple preset rate ratios; and use the preset rate ratio corresponding to the video semantic information as the rate ratio corresponding to each information segment.

[0244] In some embodiments of the present application, the multimedia information includes at least one information segment, and the target rate ratio includes the rate ratio corresponding to each information segment; the rate adjustment module 4553 is also used to adjust the playback rate of each information segment according to the rate ratio corresponding to each information segment, so as to obtain the adjusted information segment corresponding to each information segment; and use the adjusted information segment corresponding to each information segment to splice out the adjusted multimedia information.

[0245] In some embodiments of the present application, the rate ratio determination module 4552 is further used to smooth the rate ratio corresponding to each information segment to obtain a smoothed rate ratio corresponding to each information segment.

[0246] In some embodiments of the present application, the rate ratio determination module 4552 is further used to use the smoothed rate ratio corresponding to each information segment to splice out the target rate ratio of the multimedia information.

[0247] In some embodiments of the present application, the rate ratio of each information segment includes the sub-rate ratio corresponding to each time period in each information segment; the rate ratio determination module 4552 is further used to perform a difference calculation between the rate ratio corresponding to each information segment and the rate ratio corresponding to the previous information segment of each information segment to obtain a rate ratio difference; the rate ratio difference is compared with the number of time periods to obtain a rate ratio change amplitude of each time period in the time periods; the number of time periods represents the number of time periods in each information segment; the sub-rate ratio of the previous time period of each time period is added to the rate ratio change amplitude of each time period to obtain the sub-rate ratio of each time period; when the calculation of the sub-rate ratio is completed for each time period, the sub-rate ratio of each time period in each information segment is obtained; the sub-rate ratio of each time period is spliced ​​out using the sub-rate ratios of each time period.

[0248] In some embodiments of the present application, the rate ratio determination module 4552 is also used to determine the ratio of the rate ratio difference and the number of time periods as the rate ratio change amplitude for each time period when the ratio of the rate ratio difference and the number of time periods is less than or equal to a preset change amplitude; and when the ratio of the rate ratio difference and the number of time periods is greater than the preset change amplitude, determine the preset change amplitude as the rate ratio change amplitude for each time period.

[0249] In some embodiments of the present application, the rate ratio determination module 4552 is further used to add the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period to obtain a sum result; for each time period, a preset number of associated time periods are extracted, and the sub-rate ratios corresponding to each of the preset number of associated time periods are used as the preset number of associated sub-rate ratios corresponding to the sub-rate ratio of each time period; from the preset number of associated sub-rate ratios, the maximum associated sub-rate ratio is extracted; when the sum result is less than or equal to the maximum associated sub-rate ratio, the sum result is used as the sub-rate ratio of each time period.

[0250] The embodiment of the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the information playback control method described in the embodiment of the present application.

[0251] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein executable information playback control instructions are stored. When the executable instructions are executed by a processor, the processor will execute the information playback control method provided by the present application embodiment, for example, Figure 6 The method shown.

[0252] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0253] In some embodiments, the executable information playback control instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or as modules, components, subroutines or other units suitable for use in a computing environment.

[0254] As an example, executable information playback control instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0255] As an example, the executable information playback control instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0256] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for controlling information playback, characterized in that: include: When a trigger operation is detected on a rate control mark on a playback interface, in response to the trigger operation on the rate control mark, rate options are displayed; wherein the rate options include an intelligent adjustment option, and the intelligent adjustment option is used to trigger dynamic rate adjustment of multimedia information; When a trigger operation is detected on the smart adjustment option, in response to the trigger operation for the smart adjustment option, segmenting the multimedia information into at least one information segment based on at least one sub-content information included in the content information of the multimedia information; When the sub-content information corresponding to each information segment after the multimedia information is segmented includes text information, the display rate of the text information is calculated to obtain a text display rate; using the text display rate, the preset text rate and the preset maximum rate ratio, the adjustment ratio of each information segment is calculated to obtain a rate ratio corresponding to each information segment; Determining a target rate ratio of the multimedia information according to the rate ratio corresponding to each information segment; wherein the target rate ratio is dynamic; Using the target rate ratio, adjusting the playback rate of the multimedia information to obtain adjusted multimedia information; The adjusted multimedia information is played.

2. The method according to claim 1, characterized in that The calculating the adjustment ratio of each information segment by using the text display rate, the preset text rate and the preset maximum rate ratio to obtain the rate ratio corresponding to each information segment includes: Comparing the text display rate with the preset text rate to obtain a rate ratio calculation factor; The preset maximum rate ratio and the rate ratio calculation factor are difference calculated to obtain the rate ratio corresponding to each information segment.

3. The method according to claim 1, characterized in that The sub-content information corresponding to each information segment also includes at least any one of the following: object information, audio information and video semantic information.

4. The method according to any one of claims 1 to 3, characterized in that: The step of adjusting the playback rate of the multimedia information by using the target rate ratio to obtain the adjusted multimedia information includes: According to the rate ratio corresponding to each information segment, the playback rate of each information segment is adjusted to obtain an adjusted information segment corresponding to each information segment; The adjusted multimedia information is spliced ​​using the adjusted information segments corresponding to each information segment.

5. The method according to any one of claims 1 to 3, characterized in that: After calculating the adjustment ratio of each information segment by using the text display rate, the preset text rate and the preset maximum rate ratio to obtain the rate ratio corresponding to each information segment, the method further includes: The rate ratio corresponding to each information segment is smoothed to obtain a smoothed rate ratio corresponding to each information segment.

6. The method according to claim 5, characterized in that After smoothing the rate ratio corresponding to each information segment to obtain the smoothed rate ratio corresponding to each information segment, the method further includes: The target rate ratio of the multimedia information is spliced ​​using the smoothed rate ratio corresponding to each information segment.

7. The method according to claim 5, characterized in that The rate ratio of each information segment includes sub-rate ratios corresponding to respective time periods in each information segment; and smoothing the rate ratio corresponding to each information segment to obtain the smoothed rate ratio corresponding to each information segment includes: Calculating the difference between the rate ratio corresponding to each information segment and the rate ratio corresponding to the previous information segment of each information segment to obtain a rate ratio difference; The rate ratio difference is compared with the number of time periods to obtain the rate ratio change amplitude of each time period in each time period; the number of time periods represents the number of time periods in each information segment; The sub-rate ratio of each time period is obtained by adding the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period; When the calculation of the sub-rate ratio is completed for each time period, the sub-rate ratio of each time period in each information segment is obtained; The sub-rate ratios of the various time periods are used to splice out a smoothed rate ratio corresponding to each information segment.

8. The method according to claim 7, characterized in that The step of comparing the rate ratio difference with the number of time periods to obtain a rate ratio change amplitude of each time period in each time period includes: When the ratio of the rate ratio difference to the number of time periods is less than or equal to a preset change range, the ratio of the rate ratio difference to the number of time periods is determined as the rate ratio change range of each time period; When the ratio of the rate ratio difference to the number of time periods is greater than the preset change amplitude, the preset change amplitude is determined as the rate ratio change amplitude of each time period.

9. The method according to claim 7, characterized in that: The step of obtaining the sub-rate ratio of each time period by adding the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period includes: Adding the sub-rate ratio of the previous time period of each time period and the rate ratio change amplitude of each time period to obtain a sum result; For each time period, extract a preset number of associated time periods, and use the sub-rate ratios corresponding to each of the preset number of associated time periods as the preset number of associated sub-rate ratios corresponding to the sub-rate ratio of each time period; Extracting the maximum correlation sub-rate ratio from the preset number of correlation sub-rate ratios; When the sum result is less than or equal to the maximum associated sub-rate ratio, the sum result is used as the sub-rate ratio of each time period.

10. An information broadcast control device, characterized in that: include: An information display module, configured to display rate options in response to a trigger operation on a rate control mark on a playback interface when a trigger operation is detected on the rate control mark; wherein the rate options include an intelligent adjustment option, and the intelligent adjustment option is used to trigger dynamic rate adjustment of multimedia information; A rate ratio determination module, configured to, when a trigger operation is detected on the intelligent adjustment option, respond to the trigger operation for the intelligent adjustment option, segment the multimedia information into at least one information segment based on at least one sub-content information included in the content information of the multimedia information; when the sub-content information corresponding to each information segment after the multimedia information is segmented includes text information, calculate the display rate of the text information to obtain a text display rate; calculate the adjustment ratio of each information segment using the text display rate, a preset text rate and a preset maximum rate ratio to obtain a rate ratio corresponding to each information segment; determine a target rate ratio of the multimedia information according to the rate ratio corresponding to each information segment; wherein the target rate ratio is dynamic; A rate adjustment module, used to adjust the playback rate of the multimedia information by using the target rate ratio to obtain adjusted multimedia information; The information display module is also used to play the adjusted multimedia information.

11. An information playback control device, characterized in that: include: A memory, used for storing executable information playback control instructions; A processor, configured to implement the method according to any one of claims 1 to 9 when executing the executable information playback control instruction stored in the memory.

12. A computer-readable storage medium, characterized in that: Executable information playback control instructions are stored and are used to implement the method described in any one of claims 1 to 9 when executed by a processor.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images