Multi-level data optimization method and system based on big data

Abstract

The invention relates to the technical field of data processing, in particular to a multi-level data optimization method and system based on big data, and the method comprises the steps: obtaining pre-selected data and browsing data of a user; screening out a plurality of pieces of first video data from the browsing data; performing line fusion processing and screening on the first video data to obtain to-be-played video data; when the type of the browsed video is a negative type, judging whether the duration of browsing the negative type of video is greater than a second preset duration or not, and judging whether the number of the negative type of video browsing the negative type of video is greater than a preset number or not; if yes, playing the to-be-played video data; and if one is yes, screening out video data meeting a preset rule from the video information base by adopting a video screening model, and playing the video data. The method can solve the problem that the watching mood of the user is affected due to the fact that the user is continuously caught in the negative emotion because the existing video intelligent recommendation algorithm does not combine with the interest of the user and continuously recommends the negative type of video for the user.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-level data optimization method and system based on big data. Background Art

[0002] Most existing short video platforms have a video intelligent recommendation system. After a user watches a video on the short video platform, the intelligent recommendation system of the short video platform will automatically recommend a series of other videos related to the video the user watched, which can quickly help the user search for other videos, saving the user's manual search time, facilitating the user's operation and continuously attracting the user's attention.

[0003] Most of the existing intelligent video recommendation algorithms search for related videos and determine the similarity between videos, and then recommend videos to users based on the similarity. When determining the similarity between videos, the method usually adopted is based on the video content. However, without considering the user's interests, the recommendation of related videos does not change much, so that users keep watching the same type of videos, which will cause the user's emotions to continue to be in the reaction emotional state of the type of videos; if the user keeps watching negative videos, the user will continue to fall into negative emotions, which seriously affects the user's viewing mood.

[0004] Therefore, the present invention provides a multi-level data optimization method and system based on big data to solve the above problems. Summary of the invention

[0005] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a multi-level data optimization method and system based on big data to solve the problem that the above-mentioned video intelligent recommendation algorithm does not take into account the interests of users, but continues to recommend the same type of videos to users. When users continue to watch negative videos, users continue to fall into negative emotions, which seriously affects the user's viewing mood.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention provides a multi-level data optimization method based on big data, comprising: obtaining pre-selected data and browsing data of a user, wherein the pre-selected data is a plurality of interest types selected by the user, and the browsing data includes video data browsed by the user, duration of the video data, video type, duration of browsing the same type of videos, and number of negative type videos; filtering out first browsing data whose browsing duration is greater than a first preset duration from the browsing data, and obtaining a plurality of first video data from the first browsing data; obtaining first feature information of the first video data from a preset feature information library, and fusing each first feature information to obtain fused feature information; based on the fused feature information, Filter out multiple second feature information including the fused feature information from a preset feature information library; obtain second video data based on the second feature information; use a random sampling method to select a set number of video data to be played from each second video data; when the browsed video type is a negative type, determine whether the duration of browsing the negative type video is greater than a second preset duration, and determine whether the number of negative type videos browsed is greater than a preset number; if all are no, play the video data to be played in the order of update time; if one of them is yes, use a video screening model to filter out video data that meets the preset rules from the video information library and play them.

[0008] Preferably, the process of obtaining fused feature information includes: determining whether each first feature information contains the same feature data, and if so, using the same feature data as the first fused feature data; obtaining a first preset number of third feature information close to the current viewing time in each first feature information; obtaining feature data with a repetition degree in a set repetition interval in the overall data of each third feature information as the second fused feature data; integrating the first fused feature data and the second fused feature data to obtain initial fused feature information; determining whether the data amount of the initial fused feature information is greater than the set data amount, and if so, using the feature data in the second fused feature data that is less than or equal to the set data amount as the fused feature information; if not, using the initial fused feature information as the fused feature information.

[0009] Preferably, the process of obtaining the fused feature information includes: determining the repetition of each feature data in the overall data of each first feature information, removing the feature data whose repetition is greater than the maximum value in the first preset repetition to obtain the feature data to be arranged; adding the feature data to be arranged to the fused feature information in order from high to low in terms of the repetition of the feature data to be arranged; when the data amount of the fused feature information reaches the set data amount, the final fused feature information is obtained.

[0010] Preferably, the processing process of the video screening model includes: obtaining the interest type of the positive type in the pre-selected data; and obtaining the video information library corresponding to the interest type; screening the video data in the video information library at least in turn according to the video length, video upload time, and video activity to obtain multiple initial screening video data; randomly extracting a second preset number of video data from each initial screening video data, and adding them to the to-be-played list in the order of upload time.

[0011] Preferably, the processing process of the video screening model includes: obtaining a third preset number of active video data that are currently ranked at the top in terms of activity; obtaining positive type video data from the active video data to obtain second initial video data; screening out a second preset number of video data from the second initial video data in order of activity from high to low, and adding them to a to-be-played list in order of activity from high to low.

[0012] Preferably, a multi-level data optimization method based on big data also includes: after obtaining the initial video data uploaded by the user, obtaining all video frame image data and audio data of the initial video data, using a feature extraction model constructed based on the attention mechanism to perform feature extraction on each video frame data to obtain image feature information corresponding to each video frame image data, and using an audio feature extraction model to perform feature extraction on audio data to obtain audio feature information; using a clustering analysis model to perform cluster analysis on each image feature information and audio feature information to obtain the video type of the initial video data; and integrating each image feature information and audio feature information to obtain video feature information, and adding the video feature information to a preset feature information library.

[0013] Preferably, the storage and analysis process of the feature information in the preset feature information library includes: according to the video type of the initial video data, storing the video feature information of the initial video data in a to-be-inserted list in the sub-feature information library corresponding to the video type; according to the similarity between the video feature information of the initial video data and the existing video feature information in the sub-feature information library, inserting the video feature information of the initial video data into the next position of the feature information with the maximum similarity.

[0014] Preferably, the method of extracting features from audio data using an audio feature extraction model to obtain audio feature information includes: preprocessing the audio data; converting the preprocessed audio data into text data using a speech recognition model constructed using a recurrent neural network; and performing keyword analysis on the text data to extract key information from the text data to form audio feature information.

[0015] Preferably, a multi-level data optimization method based on big data also includes: when there is no preset feature information containing fused feature information in the preset feature information library, the ratio of repeated feature data in the preset feature information and the fused feature information to the total data of the fused feature information is used as the similarity between the preset feature information and the fused feature information, and multiple second feature information are screened out in order of similarity from high to low.

[0016] In a second aspect, the present invention provides a multi-level data optimization system based on big data, comprising:

[0017] The data acquisition module acquires the user's pre-selected data and browsing data. The pre-selected data is a plurality of interest types selected by the user. The browsing data includes the video data browsed by the user, the duration of the video data, the video type, the duration of browsing the same type of videos and the number of negative type videos.

[0018] The data processing module selects first browsing data whose browsing time is greater than a first preset time from the browsing data, and obtains a plurality of first video data from the first browsing data; obtains first feature information of the first video data from a preset feature information library, and fuses each first feature information to obtain fused feature information; selects a plurality of second feature information including the fused feature information from the preset feature information library based on the fused feature information; obtains second video data based on the second feature information; and selects a set number of video data to be played from each second video data using a random sampling method.

[0019] The video determination module, when the browsed video type is a negative type, determines whether the duration of browsing the negative type video is greater than a second preset duration, and determines whether the number of negative type videos browsed is greater than a preset number; if all are no, the video data to be played is played in the order of update time; if one of them is yes, a video screening model is used to screen out video data that meets the preset rules from the video information library and play it.

[0020] The beneficial effects of the present invention are:

[0021] 1. The present invention uses a method of judging whether the duration of browsing negative type videos is greater than a second preset duration, and judging whether the number of negative type videos browsed is greater than a preset number when the type of video being browsed is a negative type; if all are negative, the video data to be played is played in the order of update time; if one of them is yes, a video screening model is used to screen out video data that meets the preset rules from the video information library and play the technical means, and positive type videos are played to relieve the user's viewing mood. Through the above technical means, the present invention can solve the problem that the existing video intelligent recommendation algorithm does not combine user interests, but continues to recommend the same type of videos to users. When users continue to watch negative type videos, they continue to fall into negative emotions, which seriously affects the user's viewing mood.

[0022] 2. When the present invention selects the video data to be played according to the browsing data, the first browsing data with a browsing time longer than the first preset time is selected from the browsing data, and a plurality of first video data in the first browsing data are obtained; the first feature information of the first video data is fused to obtain fused feature information; based on the fused feature information, a plurality of second feature information including the fused feature information is selected from the preset feature information library; the second video data is obtained based on the second feature information; and a set number of video data to be played is selected from each second video data by a random sampling method. When selecting a plurality of second feature information, the second feature information is selected according to the ratio of the fused feature information to the second feature information and the set ratio interval, so that there is a large difference between the selected second feature information, so that the user can see different videos of the same type.

[0023] 3. When screening positive video data, the present invention adopts two screening methods. The first method is based on the interest type of the positive type in the user's pre-selected data; based on the video information library corresponding to the positive type, the video data in the video information library is screened at least in accordance with the video length, video upload time, and video activity, to obtain multiple initial screened video data, and then a second preset number of video data is randomly selected and added to the list to be played. The second method is to screen the positive video data in the active video data based on the third preset number of active video data that ranks at the top of the current activity level to obtain the second initial video data; then, the second preset number of video data is screened from the second initial video data in order of activity from high to low, and added to the list to be played. The present invention can screen out positive video data for users through any of the above screening methods, so that the user's negative emotions are gradually alleviated and gradually restored to a pleasant mood. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1The present invention is a schematic flow chart of a multi-level data optimization method based on big data.

[0025] Figure 2 This is a module schematic diagram of a multi-level data optimization system based on big data in the present invention. DETAILED DESCRIPTION

[0026] The following will refer to the attached Figure 1 To Attachment Figure 2 The embodiments of the present invention are described in detail. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0027] A multi-level data optimization method based on big data, as shown in the attached Figure 1 As shown, the following steps are included:

[0028] Step S11: Obtain the user's pre-selected data and browsing data.

[0029] In step S11, the pre-selected data are multiple interest types selected by the user, and the browsing data includes the video data browsed by the user, the duration of the video data, the video type, the duration of browsing similar videos, and the number of negative type videos.

[0030] Preferably, the negative type display method includes but is not limited to color label marking, digital or character code marking. Video types can be classified into emotion, law, society, finance, sports, education, entertainment, diplomacy, etc. according to the theme classification. The types of video types divided according to the set evaluation rules include positive type, negative type and neutral type. According to the positive type and negative type keyword library, the videos uploaded by the user are classified into the corresponding video types.

[0031] Step S12: Filter out first browsing data whose browsing duration is greater than a first preset duration from the browsing data, and obtain a plurality of first video data from the first browsing data.

[0032] Preferably, the first preset duration is 10s, 15s, 30s, or the duration corresponding to when the user's browsing time accounts for 20% of the total video duration, so as to filter out multiple first video data that the user is interested in from the browsing data to facilitate subsequent analysis; and filtering multiple second video data can avoid contingency.

[0033] Step S13: obtaining first feature information of the first video data from a preset feature information library, and fusing each first feature information to obtain fused feature information.

[0034] In step S13, by fusing a plurality of first feature information, common features among the first video data can be found, so that the video data to be played can be found to be more in line with the user's viewing needs based on the fused feature information.

[0035] Step S14: based on the fused feature information, a plurality of second feature information including the fused feature information is screened out from a preset feature information library; and second video data is obtained based on the second feature information.

[0036] Step S15: A set number of video data to be played are selected from each second video data by a random sampling method.

[0037] Step S16: When the type of the browsed video is a negative type, determine whether the duration of browsing the negative type video is greater than a second preset duration; if not, execute step S17, and if so, execute step S19.

[0038] Step S17: Determine whether the number of negative type videos browsed is greater than a preset number, if so, execute step S19, if not, execute step S18.

[0039] Step S18: Play the video data to be played in the order of update time.

[0040] Step S19: Use the video screening model to screen out video data that meets preset rules from the video information library and play it.

[0041] Preferably, the screening method in this embodiment includes but is not limited to a condition screening algorithm, a keyword screening algorithm, a date screening algorithm, and a specified data range screening algorithm. The preset rules are the screening conditions of the video screening model.

[0042] In this embodiment, the present invention adopts the following method: when the type of video being browsed is negative, it is judged whether the duration of browsing negative type videos is greater than a second preset duration, and whether the number of negative type videos being browsed is greater than a preset number; if all are negative, the video data to be played is played in the order of update time; if one of them is yes, a video screening model is used to screen out video data that meets preset rules from a video information library and play the video, and positive type videos are played to relieve the user's viewing mood. Through the above technical means, the present invention can solve the problem that the existing video intelligent recommendation algorithm does not combine user interests, but continues to recommend the same type of videos to users. When users continue to watch negative type videos, the users continue to fall into negative emotions, which seriously affects the user's viewing mood.

[0043] Furthermore, when the present invention selects the video data to be played according to the browsing data, the first browsing data with a browsing time longer than the first preset time is selected from the browsing data, and a plurality of first video data in the first browsing data are obtained; the first feature information of the first video data is fused to obtain fused feature information; based on the fused feature information, a plurality of second feature information including the fused feature information is selected from the preset feature information library; the second video data is obtained based on the second feature information; and a set number of video data to be played is selected from each second video data by a random sampling method. When selecting a plurality of second feature information, the second feature information is selected according to the ratio of the fused feature information to the second feature information and the set ratio interval, so that there is a large difference between the selected second feature information, so that the user can see different videos of the same type.

[0044] In an optional embodiment of the present invention, a processing process for obtaining fused feature information in step S13 includes: determining whether each first feature information contains the same feature data, and if so, taking the same feature data as the first fused feature data; obtaining a first preset number of third feature information close to the current viewing time in each first feature information; obtaining feature data with a repetition degree in a set repetition interval in the overall data of each third feature information as the second fused feature data; integrating the first fused feature data and the second fused feature data to obtain initial fused feature information;

[0045] Determine whether the data amount of the initial fused feature information is greater than the set data amount. If so, use the feature data in the second fused feature data that is less than or equal to the set data amount as the fused feature information; if not, use the initial fused feature information as the fused feature information.

[0046] Preferably, the first feature information includes multiple feature data, and similarly, the second feature information, the fused feature information, and the third feature information also include multiple feature data.

[0047] In this embodiment, since the first feature information corresponding to the obtained first video data is large and the analysis time is long, after the first fused feature data is obtained, the feature information of a first preset number close to the current viewing time in multiple first feature information is searched according to the nearest principle to obtain the third feature information; then the proportion of repeated feature data in the overall data of each third feature information is compared to obtain the repetition degree of each feature data; and then the feature data with a repetition degree in the set repetition interval is obtained as the second fused feature data; the method of obtaining the second fused feature data in this embodiment avoids duplication with the first fused feature data and improves work efficiency.

[0048] Preferably, the data volume is set as the maximum upper limit of the initial fusion feature information, and the specific value is determined according to the actual situation to avoid affecting the work efficiency when the data volume of the fusion feature information exceeds the set data volume.

[0049] When the data amount of the second fused feature data is greater than the set data amount, some feature data is randomly removed. When the data amount of the second fused feature data is reduced to be equal to or less than the set data amount, the removal is stopped to obtain fused feature information.

[0050] Through the method of this embodiment, the present invention can obtain the fused feature data of the user watching the video, and then find the video data that fits the user's viewing needs according to the fused feature data, thereby meeting the user's video viewing needs.

[0051] In an optional embodiment of the present invention, another processing process for obtaining fused feature information in step S13 includes: determining the repetition of each feature data in the overall data of each first feature information, removing the feature data corresponding to the maximum value in the repetition to obtain the feature data to be arranged; adding the feature data to be arranged to the fused feature information in order from high to low in terms of the repetition of the feature data to be arranged; when the data amount of the fused feature information reaches the set data amount, the final fused feature information is obtained.

[0052] By obtaining the fused feature information in this embodiment, when subsequently screening multiple second feature information containing the fused feature information from a preset feature information library based on the fused feature information, the second feature information in the feature data corresponding to the maximum value of the recorded repetition degree among the multiple second feature information is excluded at the same time, so that the obtained multiple second feature information changes continuously, and thus the present invention can make the screened video data gradually change, so that users can watch different video data.

[0053] In an optional embodiment of the present invention, the processing process of the video screening model includes: obtaining the interest type of the positive type in the pre-selected data; and obtaining the video information library corresponding to the interest type; screening the video data in the video information library at least in turn according to the video length, video upload time, and video activity to obtain multiple initial screening video data; randomly extracting a second preset number of video data from each initial screening video data, and adding them to the to-be-played list in the order of upload time.

[0054] Preferably, in the video type of each video, there are at least a theme type and a judgment type, and the judgment type includes a positive type, a negative type and a neutral type, which are classified according to the keyword library. When it does not belong to the positive type and the negative type keyword library, it is classified into the neutral type.

[0055] Preferably, in the video information library, the preset feature information library is a part thereof, and the video information library at least further includes a video database storing video data corresponding to each preset feature information in the preset feature information library.

[0056] Preferably, the second preset number is any value between 5 and 10, which can be determined randomly.

[0057] In an optional embodiment of the present invention, the processing process of the video screening model includes: obtaining a third preset number of active video data that is currently ranked at the top in terms of activity; obtaining positive type video data from the active video data to obtain second initial video data; screening out a second preset number of video data from the second initial video data in order of activity from high to low, and adding them to a to-be-played list in order of activity from high to low.

[0058] Preferably, the value of the third preset number is 200 or 300, which can be adjusted according to actual conditions.

[0059] Preferably, the activity evaluation method includes but is not limited to evaluation based on the number of user visits, evaluation based on the number of user comments, or evaluation based on a method in which the number of user visits and user statistics each account for half of the weight.

[0060] When screening positive video data, the present invention adopts the screening methods of the above two optional embodiments. The first is based on the interest type of the positive type in the user's pre-selected data; based on the video information library corresponding to the positive type, the video data in the video information library is screened at least in accordance with the video length, video upload time, and video activity, to obtain a plurality of initial screened video data, and then a second preset number of video data is randomly selected and added to the list to be played. The second is to screen the positive video data in the active video data based on the third preset number of active video data that ranks top in current activity, to obtain the second initial video data; then, the second preset number of video data is screened from the second initial video data in order of activity from high to low, and added to the list to be played. The present invention can screen out positive video data for users through any of the above screening methods, so that the user's negative emotions are gradually alleviated and gradually restored to a pleasant mood.

[0061] In one embodiment of the present invention, a multi-level data optimization method based on big data also includes: after obtaining the initial video data uploaded by the user, obtaining all video frame image data and audio data of the initial video data, using a feature extraction model constructed based on the attention mechanism to perform feature extraction on each video frame data to obtain image feature information corresponding to each video frame image data, and using an audio feature extraction model to perform feature extraction on audio data to obtain audio feature information; using a clustering analysis model to perform cluster analysis on each image feature information and audio feature information to obtain the video type of the initial video data; and integrating each image feature information and audio feature information to obtain video feature information, and adding the video feature information to a preset feature information library.

[0062] In this embodiment, when the video frame data is text data, after the feature extraction model extracts the text information of the video frame data, it uses a keyword extraction algorithm to process and integrate the integrated text information to form image feature information.

[0063] When the video frame data is image data, the feature extraction model processes the image data by using an image feature extraction model constructed based on the attention mechanism and convolutional neural network. The processing flow is: first, the image data is converted into pixel data, and then the pixel data is analyzed and processed through the input layer, multiple convolutional layers and output layer to obtain the image feature information of the video frame image.

[0064] The cluster analysis model is constructed by using, but not limited to, a k-means method, a hierarchical analysis method, a mixed Gaussian model, etc., to implement cluster analysis and determine the video type of the initial video.

[0065] Through the implementation method of this embodiment, the present invention can analyze and process the initial video data uploaded by the user to obtain the video type and video feature information of the initial video data; and then add the video feature information to the appropriate position in the preset feature information library according to the video type and video feature information, so as to facilitate subsequent analysis and processing and improve the data processing speed.

[0066] In one embodiment of the present invention, the storage and analysis process of feature information in a preset feature information library includes: according to the video type of the initial video data, storing the video feature information of the initial video data in a to-be-inserted list in a sub-feature information library corresponding to the video type; according to the similarity between the video feature information of the initial video data and the existing video feature information in the sub-feature information library, inserting the video feature information of the initial video data in the to-be-inserted list into the next position of the feature information with the maximum similarity in the sub-feature information library.

[0067] Through the storage method of this embodiment, the present invention can insert the video feature information of the initial video data into the appropriate position in the sub-feature information library, so as to perform batch processing on the video data of the same video type later, such as batch screening, thereby improving data processing efficiency.

[0068] In one embodiment of the present invention, an audio feature extraction model is used to perform feature extraction on audio data to obtain audio feature information, including: preprocessing the audio data; converting the preprocessed audio data into text data using a speech recognition model constructed using a recurrent neural network; performing keyword analysis on the text data, and extracting key information from the text data based on a keyword library to form audio feature information.

[0069] Preferably, during preprocessing, the audio data is subjected to denoising, smoothing, etc., to improve the quality of the audio data so that the speech recognition model can perform recognition processing and improve the accuracy of speech recognition.

[0070] Through the implementation in this embodiment, the present invention can extract features from audio data through an audio feature extraction model to obtain audio feature information, so as to subsequently process and analyze the audio feature information and improve the speed of subsequent data processing.

[0071] In one embodiment of the present invention, a multi-level data optimization method based on big data also includes: when there is no preset feature information containing fused feature information in the preset feature information library, the ratio of repeated feature data in the preset feature information and the fused feature information to the total data of the fused feature information is used as the similarity between the preset feature information and the fused feature information, and multiple second feature information are screened out in order of similarity from high to low.

[0072] In this embodiment, when the preset feature information including the fused feature information cannot be found in the preset feature information library, the similarity between the preset feature information and the fused feature information is calculated, that is, the ratio of repeated feature data in the preset feature information and the fused feature information to the total data of the fused feature information is used as the similarity, and then multiple preset feature information is screened from the preset feature information library in order of similarity from high to low to obtain multiple second feature information. Then, the corresponding second video data is found according to the multiple second feature information, and subsequent selection processing is performed to obtain a set number of video data to be played.

[0073] Preferably, the number is set to any value between 5 and 10, which can be determined randomly.

[0074] Through the configuration of this embodiment, the present invention can match the appropriate second feature information when the preset feature information containing the fusion feature information cannot be found, and then obtain the appropriate video data to be played, which meets the viewing needs of the user.

[0075] In one embodiment of the present invention, as shown in the attached Figure 2 As shown, the present invention also provides a multi-level data optimization system based on big data, and the system includes a data acquisition module, a data processing module and a video determination module.

[0076] Among them, the data acquisition module is used to obtain the user's pre-selected data and browsing data, the pre-selected data is multiple interest types selected by the user, and the browsing data includes the video data browsed by the user, the length of the video data, the video type, the duration of browsing similar videos and the number of negative type videos.

[0077] The data processing module is used to filter out first browsing data whose browsing time is greater than a first preset time from the browsing data, and obtain multiple first video data from the first browsing data; obtain first feature information of the first video data from a preset feature information library, and fuse each first feature information to obtain fused feature information; based on the fused feature information, filter out multiple second feature information containing the fused feature information from the preset feature information library; obtain second video data based on the second feature information; and use a random sampling method to select a set number of video data to be played from each second video data.

[0078] The video determination module is used to determine whether the duration of browsing negative type videos is greater than a second preset duration, and whether the number of negative type videos browsed is greater than a preset number when the browsed video type is a negative type; if all are no, the video data to be played is played in the order of update time; if one of them is yes, a video screening model is used to screen out video data that meets the preset rules from a video information library and play it.

[0079] In this embodiment, the present invention can solve the problem that the existing video intelligent recommendation algorithm does not take into account the user's interests, but continues to recommend the same type of videos to the user through the mutual cooperation between the above-mentioned modules. When the user continues to watch negative videos, the user continues to fall into negative emotions, which seriously affects the user's viewing mood.

[0080] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0081] It should be noted that in the description of the present invention, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings, which are only for the convenience of description, and do not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0082] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0083] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0084] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0086] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0087] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0088] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A multi-level data optimization method based on big data, characterized in that: include: Acquire the user's pre-selected data and browsing data, wherein the pre-selected data is a plurality of interest types selected by the user, and the browsing data includes the video data browsed by the user, the duration of the video data, the video type, the duration of browsing the same type of videos, and the number of negative type videos; Filter out first browsing data whose browsing duration is greater than a first preset duration from the browsing data, and obtain a plurality of first video data in the first browsing data; Acquire first feature information of first video data from a preset feature information library, fuse each first feature information to obtain fused feature information; select multiple second feature information including the fused feature information from the preset feature information library according to the fused feature information; obtain second video data according to the second feature information; and select a set number of video data to be played from each second video data by a random sampling method; When the type of the browsed video is a negative type, determining whether the duration of browsing the negative type video is greater than a second preset duration, and determining whether the number of negative type videos browsed is greater than a preset number; If all are yes or no, then the video data to be played is played in the order of update time; If one of them is yes, the video screening model is used to screen out video data that meets the preset rules from the video information library and play it.

2. The multi-level data optimization method according to claim 1, characterized in that: The process of obtaining the fused feature information includes: determining whether each first feature information contains the same feature data, and if so, taking the same feature data as the first fused feature data; Obtain a first preset number of third feature information close to the current viewing time from each first feature information; obtain feature data with a repetition degree within a set repetition interval from the overall data of each third feature information as second fused feature data; integrate the first fused feature data and the second fused feature data to obtain initial fused feature information; Determine whether the data amount of the initial fused feature information is greater than the set data amount. If so, use the feature data in the second fused feature data that is less than or equal to the set data amount as the fused feature information; if not, use the initial fused feature information as the fused feature information.

3. The multi-level data optimization method according to claim 1, characterized in that: The process of obtaining the fused feature information includes: determining the repetition of each feature data in the overall data of each first feature information, removing the feature data whose repetition is greater than the maximum value in the first preset repetition, so as to obtain the feature data to be arranged; adding the feature data to be arranged to the fused feature information in order from high to low in terms of the repetition of the feature data to be arranged; and obtaining the final fused feature information when the data amount of the fused feature information reaches the set data amount.

4. The multi-level data optimization method according to claim 1, characterized in that: The processing process of the video screening model includes: obtaining the interest type of the positive type in the pre-selected data; and obtaining the video information library corresponding to the interest type; screening the video data in the video information library at least in turn according to the video length, video upload time, and video activity to obtain multiple initial screening video data; randomly extracting a second preset number of video data from each initial screening video data, and adding them to the to-be-played list in the order of upload time.

5. The multi-level data optimization method according to claim 1, characterized in that: The processing process of the video screening model includes: obtaining a third preset number of active video data that are currently ranked at the top in terms of activity; obtaining positive type video data from the active video data to obtain second initial video data; screening out a second preset number of video data from the second initial video data in order of activity from high to low, and adding them to a to-be-played list in order of activity from high to low.

6. The multi-level data optimization method according to claim 1, characterized in that: Also includes: After obtaining the initial video data uploaded by the user, all video frame image data and audio data of the initial video data are obtained, and a feature extraction model based on the attention mechanism is used to perform feature extraction on each video frame data to obtain image feature information corresponding to each video frame image data, and an audio feature extraction model is used to perform feature extraction on the audio data to obtain audio feature information; Using a cluster analysis model to perform cluster analysis on each image feature information and audio feature information to obtain the video type of the initial video data; And the image feature information and the audio feature information are integrated to obtain the video feature information, and the video feature information is added to the preset feature information library.

7. The multi-level data optimization method according to claim 6, characterized in that: The storage and analysis process of the feature information in the preset feature information library includes: according to the video type of the initial video data, storing the video feature information of the initial video data in a to-be-inserted list in the sub-feature information library corresponding to the video type; according to the similarity between the video feature information of the initial video data and the existing video feature information in the sub-feature information library, inserting the video feature information of the initial video data into the next position of the feature information with the maximum similarity.

8. The multi-level data optimization method according to claim 6, characterized in that: The method of using an audio feature extraction model to extract features from audio data to obtain audio feature information includes: preprocessing the audio data; converting the preprocessed audio data into text data using a speech recognition model constructed using a recurrent neural network; and performing keyword analysis on the text data to extract key information from the text data to form audio feature information.

9. The multi-level data optimization method according to claim 1, characterized in that: Also includes: When there is no preset feature information containing fused feature information in the preset feature information library, the ratio of repeated feature data in the preset feature information and the fused feature information to the total data of the fused feature information is used as the similarity between the preset feature information and the fused feature information, and multiple second feature information are screened out in order of similarity from high to low.

10. A multi-level data optimization system based on big data, characterized in that: include: A data acquisition module, which acquires the user's pre-selected data and browsing data, wherein the pre-selected data is a plurality of interest types selected by the user, and the browsing data includes the video data browsed by the user, the duration of the video data, the video type, the duration of browsing the same type of videos, and the number of negative type videos; A data processing module, which selects first browsing data whose browsing time is longer than a first preset time from the browsing data, and obtains a plurality of first video data in the first browsing data; Acquire first feature information of first video data from a preset feature information library, fuse each first feature information to obtain fused feature information; select multiple second feature information including the fused feature information from the preset feature information library according to the fused feature information; obtain second video data according to the second feature information; and select a set number of video data to be played from each second video data by a random sampling method; The video determination module determines, when the type of the browsed video is a negative type, whether the duration of browsing the negative type video is greater than a second preset duration, and determines whether the number of negative type videos browsed is greater than a preset number; If all are yes or no, then the video data to be played is played in the order of update time; If one of them is yes, the video screening model is used to screen out video data that meets the preset rules from the video information library and play it.

Images