Chat video transcoding method, system and equipment based on browser and medium

Abstract

The invention is applicable to the technical field of browsers, and provides a browser-based chat video transcoding method, which comprises the following steps of: acquiring a source video link of a video file, and constructing video source data based on the source video link; transcoding the video file based on the source video link to obtain transcoded video data; writing the transcoded video data into a callback queue, and caching the transcoded video data to a chat page by calling the callback queue; a pre-trained neural network model is loaded, and the neural network model is used for analyzing the transcoding video data, so that a video file is played; according to the performance indexes and the network environment parameters of the browser, a video playing strategy library is constructed, the video playing strategy library comprises a plurality of video playing strategies, and the video playing quality is dynamically adjusted by dynamically selecting the video playing strategies. According to the method and the device, the problem that different browsers support different video formats inconsistently can be effectively solved, and the chat tool video transcoding efficiency of the browsers and the user experience are improved.

Description

Technical Field

[0001] The present application belongs to the field of browser technology, and in particular relates to a browser-based chat video transcoding method, system, device and medium. Background Art

[0002] With the rapid development of Internet technology, browser-based real-time chat tools have become an indispensable part of social media interaction, customer service response and efficient team collaboration. These real-time chat tools have greatly promoted the circulation and sharing of information by providing instant messaging functions, and users can easily send multimedia content including text, pictures, audio and video on the platform.

[0003] However, while enjoying this convenience, the transmission and display of video files often face compatibility issues with different browsers. Specifically, there are significant differences in the support of video encoding formats by different browsers. For example, MOV format video, as a more professional video packaging format, is very popular in some professional applications due to its high-quality encoding characteristics. However, in the browser environment of ordinary users, the video cannot be played normally due to lack of native support or inconsistent decoders, and even a black screen phenomenon occurs, which seriously affects the user experience.

[0004] In order to alleviate this problem, the existing technology usually adopts the server-side video transcoding method, that is, after the video file is uploaded to the server, the server uniformly converts the MOV format video into MP4 and other formats widely supported by the browser, and then distributes it to the recipient. Although this method solves the browser compatibility problem to a certain extent, its inherent disadvantages cannot be ignored: first, the server needs to undertake additional transcoding tasks, which significantly increases the load pressure of the server; second, the transcoding process takes a certain amount of time, resulting in obvious delays in video transmission, which cannot meet the strict requirements of real-time chat tools for immediacy; finally, a large number of transcoding operations not only consume precious computing resources, but also may aggravate hardware loss due to frequent reading and writing of disks, resulting in waste of resources. In summary, the server-side video transcoding method of the existing technology has problems such as high server load, large delay, and waste of resources, and cannot meet the requirements of real-time chat tools for efficiency and immediacy. Summary of the invention

[0005] The embodiments of the present application provide a browser-based chat video transcoding method, system, device and medium, which can solve one of the above-mentioned prior art problems.

[0006] In a first aspect, an embodiment of the present application provides a browser-based chat video transcoding method, comprising:

[0007] Obtain a source video link of a video file, and construct video source data based on the source video link;

[0008] Based on the source video link, transcode the video file to obtain transcoded video data;

[0009] Writing the transcoded video data into a callback queue, and caching the transcoded video data to the chat page by calling the callback queue;

[0010] Loading a pre-trained neural network model, wherein the neural network model is used to analyze the transcoded video data to enable the video file to be played;

[0011] A video playback strategy library is constructed according to the performance indicators of the browser and the network environment parameters, wherein the video playback strategy library includes a plurality of video playback strategies, and the video playback quality is dynamically adjusted by dynamically selecting the video playback strategy.

[0012] Further, before obtaining the source video link of the video file and constructing the video source data based on the source video link, the method includes:

[0013] Determining, according to the file name of the video file, whether the video format of the video file is a preset format;

[0014] If the video format of the video file is a preset format, caching the video file to the chat page;

[0015] If the video format of the video file is not a preset format, video source data is constructed and written into a storage queue, wherein the video source data includes a source video link, a callback queue name, and a video source tag.

[0016] Further, the step of transcoding the video file based on the source video link to obtain transcoded video data includes:

[0017] Obtaining a video data stream through a source video link, and dividing the video data stream into a plurality of first data blocks according to a preset segmentation rule, wherein each of the first data blocks includes a video segment of a predetermined length;

[0018] Each of the first data blocks is transcoded by dynamically adjusting encoding parameters to obtain a second data block, wherein the second data block includes transcoded video data.

[0019] Furthermore, the step of transcoding each of the first data blocks by dynamically adjusting the encoding parameters to obtain the second data block includes:

[0020] For each of the first data blocks, extract all key frames of the first data block, calculate similarities between adjacent segment frames, and obtain inter-frame similarity;

[0021] Determining the complexity of the content of the first data block according to the key frame and the inter-frame similarity;

[0022] If the key frames are dense and the similarity between frames is low, it means that the content of the first data block is complex;

[0023] If the key frames are sparse and the similarity between frames is high, it means that the content of the first database is simple;

[0024] Adopting a decision tree algorithm, taking the complexity of the content of the first data block as a judgment condition, and dynamically selecting appropriate encoding parameters;

[0025] By comparing the video quality and transcoding time between the first data block and the second data block, a reinforcement learning algorithm is used to dynamically optimize the encoding parameter selection strategy.

[0026] Furthermore, the step of writing the transcoded video data into a callback queue and caching the transcoded video data to the chat page by calling the callback queue includes:

[0027] storing the plurality of second data blocks in a callback queue in sequence according to a transcoding order, and establishing callback information for each second data block, wherein the callback information includes a start timestamp, a duration, and video source data;

[0028] When the callback queue is called, the cached video duration of the video file is updated in real time through the callback information;

[0029] Based on the duration of the cached video, the cache progress of the video file is obtained.

[0030] Further, after caching the transcoded video data to the chat page by calling the callback queue, the method further includes:

[0031] Obtaining a playback request for the video file, and traversing the callback queue according to the playback request to obtain all second data blocks and cache progress in the video file;

[0032] If the video file has been transcoded, the transcoded video data of the second data block is played in sequence according to the playback order;

[0033] If the video file contains a first data block that has not been transcoded, obtaining a second data block for the transcoded first data block, and playing a corresponding video clip according to the transcoded video data of the second data block;

[0034] For the untranscoded first data block, video source data of the untranscoded video segment is obtained according to the callback information of the second data block, and the video source data is input into the video transcoder for real-time transcoding.

[0035] Further, the loading of a pre-trained neural network model, wherein the neural network model is used to analyze the transcoded video data to enable the video file to be played, includes:

[0036] Compiling the neural network model to convert the neural network model into a compiled model;

[0037] Creating a WebWorker thread, loading the compiled model into the WebWorker for asynchronous execution;

[0038] Acquire input requirement data of the compilation model, and process the video segment corresponding to the transcoded video data according to the input requirement data to obtain a standard input segment;

[0039] The standard input segment is input into the compilation model, and the corresponding video segment is played.

[0040] In a second aspect, an embodiment of the present application provides a browser-based chat video transcoding system, including:

[0041] The first processing module is used to obtain a source video link of a video file and construct video source data based on the source video link;

[0042] A second processing module: used for transcoding the video file based on the source video link to obtain transcoded video data;

[0043] The third processing module is used for writing the transcoded video data into a callback queue, and caching the transcoded video data to the chat page by calling the callback queue;

[0044] A fourth processing module: used for loading a pre-trained neural network model, wherein the neural network model is used for analyzing the transcoded video data to enable the video file to be played;

[0045] The fifth processing module is used to construct a video playback strategy library according to the performance indicators of the browser and the network environment parameters, wherein the video playback strategy library includes multiple video playback strategies, and the video playback quality is dynamically adjusted by dynamically selecting the video playback strategy.

[0046] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned browser-based chat video transcoding method when executing the computer program.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned browser-based chat video transcoding method is implemented.

[0048] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0049] The present application's browser-based chat video transcoding method can effectively solve the problem of inconsistent support for different video formats by different browsers by transcoding video files on the browser side, and avoid the video black screen phenomenon. At the same time, in order to avoid the high load and high latency problems caused by server-side transcoding, the present application also sets a neural network model and a video playback strategy library to improve the efficiency and user experience of the browser's chat tool video transcoding. Therefore, the present application has good scalability and compatibility, and can adapt to the needs of different browsers and versions. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 It is a flowchart of a browser-based chat video transcoding method provided by an embodiment of the present invention;

[0052] Figure 2 It is a structural schematic diagram of a browser-based chat video transcoding system provided by an embodiment of the present invention;

[0053] Figure 3 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] See also Figure 1 As shown, the present invention is a chat video transcoding method based on a browser, comprising the following steps:

[0061] S100, obtaining a source video link of a video file, and constructing video source data based on the source video link;

[0062] The chat video transcoding method of the present application is applied to the real-time chat tool of the browser. When the user sends a video file during the chat, the network protocol and parsing technology are used to obtain the video file and the source video link of the video file. By constructing the video source data, the video file has a unique data source, ensuring the security of the data, providing stable and reliable input for subsequent video transcoding, analysis, processing and other operations, and ensuring the continuity and accuracy of the entire video processing flow.

[0063] In some embodiments, before the above step S100, the following steps are included:

[0064] Determining, according to the file name of the video file, whether the video format of the video file is a preset format;

[0065] If the video format of the video file is a preset format, caching the video file to the chat page;

[0066] If the video format of the video file is not a preset format, video source data is constructed and written into a storage queue, wherein the video source data includes a source video link, a callback queue name, and a video source tag.

[0067] In the present application, before performing transcoding operation on the received video file, it is necessary to determine whether the video format of the video file is a preset format, and then determine whether transcoding operation is required. If the video file is already in the preset format, then the subsequent cache operation can be directly performed without performing unnecessary transcoding process, thereby saving computing resources and time. For video files that need to be transcoded, this step can be used to screen them out in advance, so that the browser can concentrate resources to perform efficient transcoding processing on these video files without having to perform transcoding attempts on all received video files, further improving processing efficiency, and at the same time, by providing timely transcoding services for video files that do not conform to the preset format, the user chat process is made smoother and more efficient, thereby improving the overall experience of the user. In addition, performing file format judgment before transcoding can also avoid transcoding failures or system errors caused by format errors or lack of support. This helps to reduce the system burden and failure rate caused by processing erroneous video files.

[0068] Since video files are usually stored and transmitted in the form of binary data in computers, in the present application, a binary data stream of the video file is obtained based on the received video file; the file name of the video file is extracted by parsing the binary data stream. Specifically, a pattern matching algorithm is used to match the file name string from the binary data stream of the video file, determine the starting position and the ending position of the file name string, and extract the complete file name.

[0069] In this embodiment, the file name string is obtained, the file name string is stored in a string variable, and the position index of the last period is obtained by traversing the string variable from right to left. According to the position index, the substring after the period is intercepted and stored in a format string variable, and the format string variable is compared with a preset format to determine whether the video format of the video file is the preset format, and the determination result is returned.

[0070] In this embodiment, the preset format is usually set based on the compatibility requirements of the browser to ensure that all video files that need to be processed meet the playback requirements of the browser and reduce problems such as black screens caused by format incompatibility. Therefore, the number of preset formats set is usually multiple, and multiple preset formats are stored in a preset format list. By traversing the above preset format list, it is determined whether the video format of the video file meets the playback requirements of the browser, and then it is determined whether the video file needs to be transcoded.

[0071] During a chat, a user may send more than one video file. Therefore, in this embodiment, by timely writing the unique video source data constructed by each video file into the storage queue, the system can manage the transcoding tasks of multiple video files in an orderly manner. The orderly queuing of the video source data ensures that the transcoding tasks can be performed in the order of receipt or priority, avoiding resource conflicts and efficiency reduction caused by processing multiple videos at the same time, ensuring that each video is processed in a timely and orderly manner, reducing the user's waiting time, improving processing efficiency, and enhancing the overall user experience. In addition, through queue management, the browser can allocate resources more flexibly, avoiding browser crashes or performance degradation caused by sudden large numbers of video transcoding requests, and enhancing the stability and reliability of the browser.

[0072] S200, based on the source video link, transcode the video file to obtain transcoded video data;

[0073] In some embodiments, the step S200 includes:

[0074] Obtaining a video data stream through a source video link, and dividing the video data stream into a plurality of first data blocks according to a preset segmentation rule, wherein each of the first data blocks includes a video segment of a predetermined length;

[0075] Each of the first data blocks is transcoded by dynamically adjusting encoding parameters to obtain a second data block, wherein the second data block includes transcoded video data.

[0076] In this embodiment, the video data stream of the video file is obtained through the Web Streams API, the format and encoding method of the video data stream are determined, and the appropriate segmentation rules and data block size are determined. The Web Streams API is an interface for processing streaming data in a Web browser. It provides a mechanism for transmitting data streams from one place to another without loading the entire data set into memory. Therefore, the Web Streams API includes the data stream of the video file.

[0077] In this embodiment, the predetermined duration of the first data block is determined according to the total duration of the video and the preset number of data blocks. The size of the first data block needs to evaluate the impact of data blocks of different sizes on the storage space and transcoding efficiency. Specifically, a machine learning algorithm such as a decision tree or a support vector machine is used to obtain the optimal first data block size, which can minimize the number of transcoding times while satisfying progressive caching. For example, when the size of the first data block is 2MB, the storage space is 200GB, and the transcoding time is 10 minutes. When the size of the first data block is 4MB, the storage space is 100GB, and the transcoding time is 8 minutes. The optimal data block size can be obtained as 4MB through the decision tree algorithm.

[0078] In this embodiment, for each first data block, the transcoded second data block should be consistent with the video segment length of the first data block to ensure the continuity of video playback.

[0079] In some embodiments, the step of transcoding each of the first data blocks by dynamically adjusting the encoding parameters to obtain the second data block includes:

[0080] For each of the first data blocks, extract all key frames of the first data block, calculate similarities between adjacent segment frames, and obtain inter-frame similarity;

[0081] Determining the complexity of the content of the first data block according to the key frame and the inter-frame similarity;

[0082] If the key frames are dense and the similarity between frames is low, it means that the content of the first data block is complex;

[0083] If the key frames are sparse and the similarity between frames is high, it means that the content of the first database is simple;

[0084] Adopting a decision tree algorithm, taking the complexity of the content of the first data block as a judgment condition, and dynamically selecting appropriate encoding parameters;

[0085] By comparing the video quality and transcoding time between the first data block and the second data block, a reinforcement learning algorithm is used to dynamically optimize the encoding parameter selection strategy.

[0086] In this embodiment, each first data block is segmented to obtain multiple segment frames, wherein the segmentation granularity can be dynamically adjusted according to the total duration and content characteristics of the first data block, such as a first data block with a shorter predetermined duration can be segmented into 5-10 segment frames, and a first data block with a longer duration can be segmented into 20-30 segment frames. In addition, for the segmented segment frames, the Harris corner point detection algorithm is used to extract key frames therein, specifically, all corner points in each segment frame are obtained, and by counting the number of corner points in each segment frame, the segment frames whose number of corner points exceeds the third quartile are selected as key frames according to the distribution characteristics of the number of corner points.

[0087] In this embodiment, a structural similarity method is used to calculate the similarity between two adjacent segment frames. Specifically, the calculation formula for the inter-frame similarity is as follows:

[0088]

[0089] Among them, SSIM is the inter-frame similarity, and the value of the inter-frame similarity ranges from -1 to 1. The larger the value, the more similar the two adjacent fragment frames are. x and y represent two adjacent fragment frames, respectively, and μ x and μ y represent the means of x and y respectively, and denote the variance of x and y, σ xy represents the covariance between x and y, and C1 and C2 are constants used to prevent the denominator from being zero.

[0090] In this embodiment, the content complexity of the corresponding first data block is determined by the number of key frames and the similarity between frames, and then the encoding parameters when the first data block is transcoded are determined to ensure the quality of the video file after transcoding. Specifically, if the number of key frames is large, it means that the video segment of the first data block contains more important information or significant changes, thereby increasing the complexity of the content. In addition, if the similarity between frames is generally low, it means that the difference between adjacent segments is large, and the content in the video segment of the first data block changes frequently, which increases the complexity of the content. On the contrary, if the similarity between frames is generally high and the number of key frames is low, it means that the video content is relatively stable and the complexity is low.

[0091] In this embodiment, a CART decision tree is constructed, and the number of key frames and the similarity between frames are used as input features to train and generate a decision tree that can output coding parameters, so that the coding parameters are determined according to the content complexity of the first data block during the transcoding process, wherein the coding parameters include quantization parameter (QP), frame rate (FPS), resolution, bit rate, etc. In addition, during the transcoding process, the Q-Learning reinforcement learning algorithm is used to dynamically optimize the coding parameter selection strategy, so that the coding parameter selection strategy achieves a better balance between transcoding quality and transcoding speed. Specifically, when constructing a coding parameter optimization strategy based on Q-Learning reinforcement learning, it is necessary to clearly define the state and action space, wherein the state space is used to reflect various characteristics of the current coding environment, including but not limited to the resolution and complexity of the current frame, and the average PSNR value and average time of the past coding frames. At the same time, it is also necessary to record the previous coding parameter settings and their effects, such as quantization parameter QP, frame rate setting, and coding mode selection. The action space is the adjustment operations that can be performed when optimizing the encoding parameters, such as changing the quantization parameter QP (affecting the compression ratio and quality), changing the encoding mode (such as switching from I frame to P / B frame), adjusting the encoding rate, etc. After that, the reward function is designed. For example, according to the optimization goal, when the PSNR value is significantly improved by 1dB and the encoding time is reduced by 10%, a significant reward (+10) should be given to encourage such positive changes. On the contrary, if the PSNR value drops by 1dB and the time consumption increases by 10%, a severe penalty (-10) should be implemented to avoid such unfavorable situations. For the intermediate situation between the two, the reward function can be flexibly designed to reflect the trade-off between PSNR improvement and time reduction through linear interpolation or more complex functions, thereby guiding the learning algorithm to find the best balance between quality and speed.

[0092] S300, writing the transcoded video data into a callback queue, and caching the transcoded video data to the chat page by calling the callback queue;

[0093] In some embodiments, the step S300 includes:

[0094] storing the plurality of second data blocks in a callback queue in sequence according to a transcoding order, and establishing callback information for each second data block, wherein the callback information includes a start timestamp, a duration, and video source data;

[0095] When the callback queue is called, the cached video duration of the video file is updated in real time through the callback information;

[0096] Based on the duration of the cached video, the cache progress of the video file is obtained.

[0097] In order to improve the transcoding efficiency of video files, in this embodiment, each first data block is transcoded independently, and then the multiple second data blocks obtained by transcoding are stored in the callback queue in sequence. By calling the callback queue, the second data blocks are cached to the chat page. At the same time, when the callback queue is called, the callback information is returned to obtain the caching progress of the video file.

[0098] In some embodiments, after caching the transcoded video data to the chat page by calling the callback queue, the following steps are included:

[0099] Obtaining a playback request for the video file, and traversing the callback queue according to the playback request to obtain all second data blocks and cache progress in the video file;

[0100] If the video file has been transcoded, the transcoded video data of the second data block is played in sequence according to the playback order;

[0101] If the video file contains a first data block that has not been transcoded, obtaining a second data block for the transcoded first data block, and playing a corresponding video clip according to the transcoded video data of the second data block;

[0102] For the untranscoded first data block, video source data of the untranscoded video segment is obtained according to the callback information of the second data block, and the video source data is input into the video transcoder for real-time transcoding.

[0103] In this example, according to the obtained playback request, the video file requested to be played is obtained from the storage queue, and the callback queue is traversed to obtain all the first data blocks of the corresponding video file that have not been transcoded and cached and the transcoded video data that have been transcoded, and the video files are played according to the transcoded video data in chronological order. Therefore, when a playback request for a video file is obtained, the transcoding cache status of the video file can be obtained in time, and the transcoded video data that has been transcoded can be responded to immediately for video playback, thereby reducing user waiting time and providing users with a chat viewing experience.

[0104] It can be understood that in this embodiment, the first data block is transcoded in the time sequence of the video file to obtain the second data block. Therefore, when there is an untranscoded first data block in the video file, the system obtains the cached video fragments in the video file based on the transcoded video data and callback information obtained by transcoding and plays the cached video fragments in the time sequence.

[0105] In this embodiment, the video transcoder is a pre-trained model that can transcode the untranscoded first data block into the second data block according to the browser's performance indicators and network environment parameters, and at the same time call transcoding to obtain the transcoded video data of the second data block, and cache it in time to improve the smoothness of video playback. In a preferred embodiment, the video transcoder uses the browser's built-in WebAssembly technology to transcode the first video block, without relying on third-party tools, and the transcoding process is completed on the browser side, which is real-time.

[0106] S400, loading a pre-trained neural network model, where the neural network model is used to analyze the transcoded video data to enable the video file to be played;

[0107] In some embodiments, the step S400 includes:

[0108] Compiling the neural network model to convert the neural network model into a compiled model;

[0109] Creating a WebWorker thread, loading the compiled model into the WebWorker for asynchronous execution;

[0110] Acquire input requirement data of the compilation model, and process the video segment corresponding to the transcoded video data according to the input requirement data to obtain a standard input segment;

[0111] The standard input segment is input into the compilation model, and the corresponding video segment is played.

[0112] In this embodiment, the pre-trained neural network model is a network model that is obtained by training with a large amount of training data and is capable of performing video analysis and video playback. By loading the pre-trained neural network model, the transcoded video data after transcoding can be quickly and efficiently analyzed, thereby reducing waiting time and improving the user's viewing experience. In addition, the pre-trained neural network model has strong generalization ability and robustness, can work stably in different scenarios and environments, reduces dependence on manual intervention and customized development, and thus reduces the operating costs of video file processing and chat tools.

[0113] In this embodiment, WebAssembly technology is used for compilation to convert the neural network model into a binary format that can be directly executed by the browser, so that the video can be played in the browser environment. In addition, a Web Worker thread is created and the compiled model is loaded into the Web Worker to realize asynchronous execution of the compiled model, avoid blocking the main thread, and improve the user experience.

[0114] In this embodiment, the video clip corresponding to the transcoded video data needs to be processed to adapt to the input requirements of the neural network model, so that the corresponding video clip can be played after the transcoded video data is input into the neural network model. Specifically, the input requirement data of the neural network model is obtained, wherein the input requirement data includes the dimension and data type of the input tensor, and the size of the video file is obtained through the video source data of the video file. If the size of the video file is larger than the size requirement of the input layer of the neural network model, the video clip corresponding to the transcoded video data is cropped to remove redundant pixels and obtain a standard input clip to match the model input size; if the size of the video frame is smaller than the size requirement of the model input layer, the corresponding video clip is padded to add pixels around it to meet the size requirement of the model input. Taking the ResNet-50 neural network model as an example, ResNet-50 requires an input tensor of size 224x224x3 and a data type of float32. If the size of the video file obtained through the video source data is 1920x1080, the video segment of the transcoded video data needs to be scaled to 224x224, where the scaling ratio is 117, and the central area of ​​224x224 is cropped out; if the video frame size is smaller than the model input size, a padding operation is performed, such as adding black pixels around the video frame to make it reach a size of 224x224.

[0115] S500: construct a video playback strategy library according to the browser's performance indicators and network environment parameters, wherein the video playback strategy library includes multiple video playback strategies, and dynamically adjusts the video playback quality by dynamically selecting the video playback strategy.

[0116] In this embodiment, the performance indicators and network environment parameters of the current browser are obtained, wherein the performance indicators are obtained through the performance monitoring interface, the performance indicators include CPU occupancy rate and memory occupancy rate, and the network environment parameters are obtained in real time through the network detection tool, and the network environment parameters include network bandwidth, delay, and packet loss rate. According to the preset network threshold, the network environment is divided into three levels: excellent, general, and poor, and according to the preset performance threshold, the performance indicators are divided into three levels: high, medium, and low; the network environment type and the performance indicators are combined to establish a video playback strategy library to determine the video playback quality. For example, by combining the above three levels of network environment and three levels of performance indicators, 9 combinations can be obtained, each of which is a video playback strategy, wherein the video playback strategy is provided with parameters such as video encoding format, bit rate, and resolution to measure the video playback quality.

[0117] In another embodiment, the real-time network speed change data during the video playback process is obtained, and the network speed change trend in the future period of time is predicted through a time series analysis algorithm. If the predicted future network speed change trend decreases significantly, the current video playback strategy is dynamically adjusted to reduce the playback quality and avoid video freezes. If the predicted future network speed change trend remains stable or increases, the current video playback strategy is maintained, or the video playback quality is appropriately improved to improve the video clarity. By continuously monitoring network environment parameters, device performance parameters, and video playback status parameters, the video playback strategy is adjusted in real time to ensure that a smooth video playback experience can be obtained under different network conditions and device capabilities.

[0118] See also Figure 2 As shown, the present invention also provides a browser-based chat video transcoding system, the system comprising:

[0119] The first processing module 201 is used to obtain a source video link of a video file and construct video source data based on the source video link;

[0120] The second processing module 202 is used to transcode the video file based on the source video link to obtain transcoded video data;

[0121] The third processing module 203 is used to write the transcoded video data into a callback queue, and cache the transcoded video data to the chat page by calling the callback queue;

[0122] The fourth processing module 204 is used to load a pre-trained neural network model, wherein the neural network model is used to analyze the transcoded video data to enable the video file to be played;

[0123] The fifth processing module 205 is used to construct a video playback strategy library according to the performance indicators of the browser and the network environment parameters, wherein the video playback strategy library includes multiple video playback strategies, and the video playback quality is dynamically adjusted by dynamically selecting the video playback strategy.

[0124] It is understandable that if Figure 1 The contents of the browser-based chat video transcoding method embodiment shown in the figure are all applicable to the browser-based chat video transcoding system embodiment. The functions specifically implemented by the browser-based chat video transcoding system embodiment are similar to those in the embodiment shown in the figure. Figure 1 The embodiment of the chat video transcoding method based on the browser shown is the same as that of the embodiment of the chat video transcoding method based on the browser shown, and the beneficial effects achieved are the same as those of the embodiment of the chat video transcoding method based on the browser shown. Figure 1 The beneficial effects achieved by the browser-based chat video transcoding method embodiment shown are also the same.

[0125] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0126] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0127] See also Figure 3 As shown, an embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored on the memory 302. When the computer program 303 is executed on the processor 301, a browser-based chat video transcoding method as described in any one of the above methods is implemented.

[0128] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0129] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0130] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit of the computer device 3 and an external storage device. The memory 302 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0131] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the browser-based chat video transcoding method as described in any one of the above methods is implemented.

[0132] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0133] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A chat video transcoding method based on a browser, characterized in that: include: Obtain a source video link of a video file, and construct video source data based on the source video link; Based on the source video link, transcode the video file to obtain transcoded video data; Writing the transcoded video data into a callback queue, and caching the transcoded video data to the chat page by calling the callback queue; Loading a pre-trained neural network model, wherein the neural network model is used to analyze the transcoded video data to enable the video file to be played; A video playback strategy library is constructed according to the performance indicators of the browser and the network environment parameters, wherein the video playback strategy library includes a plurality of video playback strategies, and the video playback quality is dynamically adjusted by dynamically selecting the video playback strategy.

2. The method according to claim 1, characterized in that Before obtaining the source video link of the video file and constructing the video source data based on the source video link, the method includes: Determining, according to the file name of the video file, whether the video format of the video file is a preset format; If the video format of the video file is a preset format, caching the video file to the chat page; If the video format of the video file is not a preset format, video source data is constructed and written into a storage queue, wherein the video source data includes a source video link, a callback queue name, and a video source tag.

3. The method according to claim 1, characterized in that The step of transcoding the video file based on the source video link to obtain transcoded video data comprises: Obtaining a video data stream through a source video link, and dividing the video data stream into a plurality of first data blocks according to a preset segmentation rule, wherein each of the first data blocks includes a video segment of a predetermined length; Each of the first data blocks is transcoded by dynamically adjusting encoding parameters to obtain a second data block, wherein the second data block includes transcoded video data.

4. The method according to claim 3, characterized in that The step of transcoding each of the first data blocks by dynamically adjusting the encoding parameters to obtain the second data block includes: For each of the first data blocks, extract all key frames of the first data block, calculate similarities between adjacent segment frames, and obtain inter-frame similarity; Determining the complexity of the content of the first data block according to the key frame and the inter-frame similarity; If the key frames are dense and the similarity between frames is low, it means that the content of the first data block is complex; If the key frames are sparse and the similarity between frames is high, it means that the content of the first database is simple; Adopting a decision tree algorithm, taking the complexity of the content of the first data block as a judgment condition, and dynamically selecting appropriate encoding parameters; By comparing the video quality and transcoding time between the first data block and the second data block, a reinforcement learning algorithm is used to dynamically optimize the encoding parameter selection strategy.

5. The method according to claim 3, characterized in that The step of writing the transcoded video data into a callback queue and caching the transcoded video data to the chat page by calling the callback queue includes: storing the plurality of second data blocks in a callback queue in sequence according to a transcoding order, and establishing callback information for each second data block, wherein the callback information includes a start timestamp, a duration, and video source data; When the callback queue is called, the cached video duration of the video file is updated in real time through the callback information; Based on the duration of the cached video, the cache progress of the video file is obtained.

6. The method according to claim 5, characterized in that After caching the transcoded video data to the chat page by calling the callback queue, the method further comprises: Obtaining a playback request for the video file, and traversing the callback queue according to the playback request to obtain all second data blocks and cache progress in the video file; If the video file has been transcoded, the transcoded video data of the second data block is played in sequence according to the playback order; If the video file contains a first data block that has not been transcoded, obtaining a second data block for the transcoded first data block, and playing a corresponding video clip according to the transcoded video data of the second data block; For the untranscoded first data block, video source data of the untranscoded video segment is obtained according to the callback information of the second data block, and the video source data is input into the video transcoder for real-time transcoding.

7. The method according to claim 1, characterized in that The loading of a pre-trained neural network model, wherein the neural network model is used to analyze the transcoded video data so as to play the video file, comprises: Compiling the neural network model to convert the neural network model into a compiled model; Create a WebWorker thread, load the compiled model into the WebWorker, and execute it asynchronously; Acquire input requirement data of the compilation model, and process the video segment corresponding to the transcoded video data according to the input requirement data to obtain a standard input segment; The standard input segment is input into the compilation model, and the corresponding video segment is played.

8. A browser-based chat video transcoding system, characterized in that: include: The first processing module is used to obtain a source video link of a video file and construct video source data based on the source video link; A second processing module: used for transcoding the video file based on the source video link to obtain transcoded video data; The third processing module is used for writing the transcoded video data into a callback queue, and caching the transcoded video data to the chat page by calling the callback queue; A fourth processing module: used for loading a pre-trained neural network model, wherein the neural network model is used for analyzing the transcoded video data to enable the video file to be played; The fifth processing module is used to construct a video playback strategy library according to the performance indicators of the browser and the network environment parameters, wherein the video playback strategy library includes multiple video playback strategies, and the video playback quality is dynamically adjusted by dynamically selecting the video playback strategy.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images