Preprocessing method for browser intelligent plug-in multimedia resource file

Abstract

The invention discloses a method for preprocessing a multimedia resource file of a browser intelligent plug-in. The method comprises the following steps of: 1) acquiring the multimedia resource file uploaded to a browser by a user by utilizing the browser intelligent plug-in; 2) obtaining related information of an edge end or terminal intelligent model and a central cloud intelligent model for processing the multimedia resource file; (3) detecting whether the multimedia resource file is multi-file uploading or not, if yes, traversing a file object array generated by a browser when the file is uploaded, and obtaining all uploaded file contents; 4) analyzing the content of the multimedia resource file, and selecting the corresponding edge end or terminal intelligent model to provide service if the capability and the performance of the edge end or terminal intelligent model are calculated to meet the requirements of the current file; otherwise, selecting the central cloud intelligent model to provide service; and 5) if the current file exceeds the maximum data volume which can be processed by the selected model, compressing the current file to the size which can be processed by the intelligent model. According to the invention, the user experience is improved.

Description

Technical Field

[0001] The present invention belongs to the field of electronic information technology and relates to a preprocessing method for multimedia resource files of a browser intelligent plugin. Background Art

[0002] In the existing technology, the browser intelligent plugin uses the preloading technology of browser web pages to improve the loading ability of multimedia file data, and then directly hands over the uploaded multimedia file data to the central cloud intelligent model for processing, and finally returns the result processed by the central cloud intelligent model to the browser.

[0003] Through research and analysis, the present invention finds that there are mainly three defects and their reasons in the performance of the browser intelligent plugin:

[0004] 1. For the upload of multimedia resource files, the browser generally starts to process the file after the user uploads the file and clicks the function operation button. At this time, all file upload requests are handed over to the central cloud intelligent model for judgment and processing, which will cause performance degradation.

[0005] 2. Handing over all multimedia file data to the central cloud intelligent model for processing will cause great pressure on the central cloud intelligent model, resulting in performance degradation.

[0006] 3. The central cloud intelligent model usually has limited support for the input type and format, and users need to perform file processing to meet the requirements of the model, resulting in poor user experience. Summary of the Invention

[0007] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide a preprocessing method for multimedia resource files of a browser intelligent plugin, which can realize the preprocessing operation of the input file data of the intelligent model, mainly including multimedia resource files of types such as audio, video, and images.

[0008] The present invention can asynchronously perform preprocessing operations such as multi-file deduplication, data downsampling and compression, detection and conversion of audio, video and image file types, and text type conversion on the uploaded resource file data, and then the browser intelligent plugin parses and decides whether to hand it over to the central cloud intelligent model for processing according to the capabilities and performance of the edge or terminal intelligent model, reducing the traffic pressure on the central cloud intelligent model, improving the interaction performance of the browser intelligent plugin, and enhancing the user experience.

[0009] The technical solution of the present invention is as follows:

[0010] A preprocessing method for multimedia resource files of a browser intelligent plugin, the steps of which include:

[0011] 1) Use the browser intelligent plugin to obtain the multimedia resource files uploaded by the user to the browser;

[0012] 2) Use the browser intelligent plugin to obtain the relevant information of the edge - side or terminal intelligent model and the central cloud intelligent model for processing the multimedia resource files, including the file types, file formats, and maximum data volumes that can be processed;

[0013] 3) The browser intelligent plugin detects whether the currently uploaded multimedia resource file is a multi - file upload. If so, the browser intelligent plugin first traverses the file object array generated by the browser when uploading the files to obtain the content of all uploaded files, and then generates a unique identifier for each uploaded file through the MD5 algorithm, and deduplicates the duplicate - uploaded multimedia resource files according to the unique identifier of the files;

[0014] 4) The browser intelligent plugin analyzes and calculates the capabilities and performance of the edge - side or terminal intelligent model by parsing the content of the multimedia resource file, and determines whether to select the edge - side or terminal intelligent model to provide services; if the capabilities and performance of the edge - side or terminal intelligent model meet the requirements for processing the current multimedia resource file, select the corresponding edge - side or terminal intelligent model to provide services; otherwise, select the central cloud intelligent model to provide services;

[0015] 5) The browser intelligent plugin traverses the file object array generated by the browser when uploading the files to obtain the file size of each currently uploaded file. If the currently uploaded multimedia resource file exceeds the maximum data volume that the selected intelligent model can process, compress the currently uploaded multimedia resource file to the size that the intelligent model can process without changing the upload file format.

[0016] Furthermore, when the selected intelligent model only accepts input of text - type data, the browser intelligent plugin parses and converts the currently uploaded multimedia resource file into text data.

[0017] Furthermore, if the selected intelligent model can only accept audio - type files in MP3 format, when the uploaded files are audio files in WAV or AAC format, the browser intelligent plugin converts the WAV and AAC format audio files into MP3 format; if the selected intelligent model can only accept video - type files in MP4 format, when the uploaded files are video files in AVI or MKV format, the browser intelligent plugin converts the AVI and MKV format video files into MP4 format; if the selected intelligent model can only accept image - type files in JPEG format, when the uploaded files are image files in PNG or GIF format, the browser intelligent plugin converts the PNG and GIF format image files into JPEG format.

[0018] Further, the multimedia resource files include types such as audio files, video files, image files, and text files.

[0019] A server, comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps in the above method.

[0020] A computer-readable storage medium, having stored thereon a computer program, characterized in that when the computer program is executed by a processor, the steps of the above method are implemented.

[0021] The advantages of the present invention are as follows:

[0022] Beneficial effects brought by the patent: The data processing and interaction performance of the browser intelligent plugin are improved, and the user experience of the browser intelligent plugin users is enhanced.

[0023] In addition to the above solutions, a separate intelligent app can also be made to specifically perform data preprocessing operations for uploading multimedia resource files, but the form of the browser intelligent plugin saves more space and is more convenient for users to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The present invention will be further described in detail below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0026] As Figure 1 shown, the preprocessing method in this patent includes the following steps.

[0027] Step 1: Asynchronously obtain the multimedia resource files uploaded by the user to the browser

[0028] The multimedia resource files targeted by this patent refer to resource files of types such as audio, video, and images. This patent combines the computing power of the edge or terminal intelligent model by obtaining multimedia resource file data to facilitate subsequent preprocessing operations. Optionally, audio files mainly include music, sound effects, and voice recordings, and common audio formats include MP3, WAV, AAC, FLAC, etc.; video files mainly include movies, video clips, and animations, and common video formats include MP4, AVI, MKV, MOV, etc.; image files can contain static images or graphics, and common image formats include JPEG, PNG, GIF, BMP, etc.

[0029] The user clicks the "File Upload" button in the browser intelligent plugin, then selects the multimedia resource file to be uploaded on the local machine. After clicking "OK to Upload", the multimedia resource file data will be uploaded to the browser. At the same time, the intelligent plugin asynchronously obtains the multimedia resource data uploaded by the user through the browser cache.

[0030] Step 2: Asynchronously obtain information related to the connected intelligent model

[0031] The browser intelligent plugin preprocessing mechanism provided by this patent is very flexible. It can be connected to various edge or terminal intelligent models and central cloud intelligent models as part of the data processing mechanism. The intelligent plugin needs to perform subsequent preprocessing operations on the uploaded file data according to the capabilities of the edge or terminal intelligent models and central cloud intelligent models. The intelligent plugin will obtain information such as the type of input file, input file format, and maximum input data volume of the edge or terminal intelligent models and central cloud intelligent models. The purpose of obtaining the input file type is to detect whether the type of the currently uploaded file is a file type that can be processed by the edge or terminal intelligent models and central cloud intelligent models. The essence of the file type is distinguished by the file format. One file type contains multiple file formats. Here, the file type is taken out separately as a discriminant condition to improve the execution efficiency and reduce the number of comparisons because the number of file formats is much higher than the number of file types. The purpose of obtaining the input file format is to detect whether the format of the currently uploaded file type is the same type of format that can be processed by the edge or terminal intelligent models and central cloud intelligent models. The purpose of obtaining the maximum input data volume is to detect whether the size of the currently uploaded file exceeds the input data volume limit of the edge or terminal intelligent models and central cloud intelligent models.

[0032] Step 3: When uploading multiple files, perform file deduplication based on the MD5 technology

[0033] The MD5 technology is a hash algorithm that generates a unique 32-bit string by calculating the file content. The MD5 value can be used to verify the integrity of the file and can also be used as the unique identifier of the file. The MD5 algorithm uses fast calculation methods such as bitwise operations and exclusive OR operations, so the calculation speed is very fast. The MD5 algorithm has uniqueness, that is, different files get different MD5 values, and the same files get the same MD5 values, so the accuracy of deduplication can be guaranteed. The MD5 algorithm is an irreversible algorithm, that is, it is impossible to infer the original content of the file from the MD5 value, so the MD5 value is not easily cracked.

[0034] When a user uploads multiple files, the intelligent plugin first traverses the array of file objects generated by the browser during the file upload, obtains the content of all uploaded files, generates a unique identifier for the files through the MD5 algorithm, and then clears the multimedia resource files that the user uploads repeatedly.

[0035] Step 4: Analyze the content of the multimedia resource file and select the intelligent model to provide services

[0036] The browser intelligent plugin, the edge or terminal intelligent model, and the central cloud intelligent model are three independent entities. The edge or terminal intelligent model and the central cloud intelligent model provide powerful intelligent computing and data processing capabilities. The browser intelligent plugin can access the edge or terminal intelligent models and the central cloud intelligent models with different domain biases to call their services to complete relevant business processing.

[0037] Usually, we hand over all data to the central cloud intelligent model for analysis and processing, and then feedback the processing results to the browser user interface. This causes excessive pressure on the central cloud intelligent model server, thus affecting performance. In fact, for some tasks, the capabilities and performance of the edge or terminal intelligent models are already sufficient and do not need to be handed over to the central cloud intelligent model for analysis and processing.

[0038] Evaluating the capabilities and performance of intelligent models requires considering multiple aspects. Capability evaluation includes task-specific metrics, multi-task capabilities, transfer learning effects, etc. Performance evaluation includes generalization ability, robustness, timeliness, user experience, etc.

[0039] The browser intelligent plugin analyzes and calculates the capabilities and performance of the edge or terminal intelligent models by parsing the content of the multimedia resource file, and intelligently determines whether to select the edge or terminal intelligent model for processing. If the capabilities and performance of the edge or terminal intelligent models meet the requirements for processing the current multimedia resource file, select the edge or terminal intelligent model to provide services; otherwise, since the performance and capabilities of the central cloud intelligent model are more powerful, select the central cloud intelligent model to provide services.

[0040] Step 5: Detection of the multimedia resource file type and conversion and processing of data types

[0041] Usually, intelligent models cannot guarantee the diversification of input types. In most cases, some intelligent models can only receive a single file type input, and text is one of the most basic types. Therefore, when the intelligent model cannot process the uploaded file type, the intelligent plugin can convert multiple formats of other types of multimedia resource files into text types.

[0042] If the accessed intelligent model only accepts the input of text-type data, the intelligent plug-in will parse and convert the data of the multimedia resource file into text data. When the uploaded file is of audio type, the intelligent plug-in will parse the audio file into text data and input it into the intelligent model; if the uploaded file is of video type, the intelligent plug-in will parse the video, split each frame of the video into corresponding images, then automatically recognize the text in the images, and combine all the text in the images to convert it into text and input it into the intelligent model; if the uploaded file is of image type, the intelligent plug-in will automatically recognize the text in the image and convert it into text and input it into the intelligent model.

[0043] Step 6: Multimedia resource file format detection and data format conversion processing

[0044] The multimedia resource files targeted by this patent include but are not limited to audio, video, and image files. Common audio formats include MP3, WAV, AAC, FLAC, etc., common video formats include MP4, AVI, MKV, MOV, etc., and common image formats include JPEG, PNG, GIF, BMP, etc. The intelligent model has a basic type format input for different multimedia resource files. If the intelligent model providing the service can only accept audio type files in MP3 format, when the uploaded audio file is in other incompatible formats such as WAV and AAC, the intelligent plug-in will automatically convert the formats of other audio types into MP3 format; if the intelligent model can only accept video type files in MP4 format, when the uploaded video file is in other incompatible formats such as AVI and MKV, the intelligent plug-in will automatically convert the formats of other video types into MP4 format; if the intelligent model can only accept image type files in JPEG format, when the uploaded image file is in other incompatible formats such as PNG and GIF, the intelligent plug-in will automatically convert the formats of other image types into JPEG format. If the file format is correct after the multimedia resource file format detection, the next step of processing will be carried out normally.

[0045] Step 7: Multimedia resource file size detection and data compression processing

[0046] The intelligent plug-in obtains the value of the "size" attribute, that is, the file size, by traversing the file object array generated by the uploaded file. At the same time, the intelligent plug-in will evaluate the data calculation ability of the intelligent model, that is, the maximum amount of data that the intelligent model can process. If the data of the multimedia resource file uploaded by the user exceeds the maximum amount of data that the intelligent model can process, without changing the file format, the intelligent plug-in will automatically reduce the data sampling rate, compress the data, reduce the amount of data, and at the same time retain the main features of the data until the intelligent model can process it; if the size of the multimedia resource file data uploaded by the user does not exceed the limit, it will be submitted to the intelligent model for processing.

[0047] Although specific embodiments of the present invention are disclosed for illustrative purposes, the purpose is to help understand the content of the present invention and implement it accordingly. Those skilled in the art can understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the best embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined by the claims.

Claims

1. A preprocessing method for multimedia resource files of a browser intelligent plugin, the steps of which include: 1) Use the browser intelligent plugin to obtain the multimedia resource files uploaded by the user to the browser; 2) Use the browser intelligent plugin to obtain the relevant information of the edge-side or terminal intelligent model and the central cloud intelligent model for processing the multimedia resource files, including the file types, file formats, and maximum data volumes that can be processed; 3) The browser intelligent plugin detects whether the currently uploaded multimedia resource file is a multi-file upload. If so, the browser intelligent plugin first traverses the file object array generated by the browser when uploading the files, obtains the contents of all uploaded files, and then removes duplicates from the repeatedly uploaded multimedia resource files; 4) The browser intelligent plugin analyzes and calculates the capabilities and performance of the edge-side or terminal intelligent model by parsing the content of the multimedia resource file, and determines whether to select the edge-side or terminal intelligent model to provide services; if the capabilities and performance of the edge-side or terminal intelligent model meet the requirements for processing the current multimedia resource file, select the corresponding edge-side or terminal intelligent model to provide services; otherwise, select the central cloud intelligent model to provide services; 5) The browser intelligent plugin traverses the file object array generated by the browser when uploading the files, obtains the file size of each currently uploaded file. If the currently uploaded multimedia resource file exceeds the maximum data volume that the selected intelligent model can process, compress the currently uploaded multimedia resource file to the size that the intelligent model can process without changing the upload file format.

2. The method according to claim 1, wherein When the selected intelligent model only accepts the input of text-type data, the browser intelligent plugin parses and converts the currently uploaded multimedia resource file into text data.

3. The method according to claim 2, characterized in that, When the selected intelligent model can only accept audio-type files in MP3 format, when the uploaded file is an audio file in WAV or AAC format, the browser intelligent plugin converts the WAV and AAC format audio files into MP3 format; when the selected intelligent model can only accept video-type files in MP4 format, when the uploaded file is a video file in AVI or MKV format, the browser intelligent plugin converts the AVI and MKV format video files into MP4 format; when the selected intelligent model can only accept image-type files in JPEG format, when the uploaded file is an image file in PNG or GIF format, the browser intelligent plugin converts the PNG and GIF format image files into JPEG format.

4. The method according to claim 1, characterized in that, The multimedia resource files include, but are not limited to, audio files, video files, image files, and text file types.

5. A server, characterized in that, It includes a memory and a processor. The memory stores a computer program, and the computer program is configured to be executed by the processor. The computer program includes instructions for executing the steps in any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 4.

Images