Video data storage method, device, equipment and medium based on content recognition

Abstract

The embodiments of the present disclosure disclose a method, device, equipment and medium for storing video data based on content recognition. A specific implementation of the method includes: collecting a video data set to be labeled; for each video data to be labeled in the video data set to be labeled, performing the following labeling and storage steps: performing metadata labeling processing on the video data to be labeled; determining the resolution information corresponding to the resolution labeling information as the target resolution; performing image quality enhancement processing on the video frame set to be labeled in the video data to be labeled to obtain an enhanced video frame set to be labeled; performing content recognition processing on the enhanced video frame set to be labeled; based on the enhanced video frame set to be labeled, updating the video data to be labeled to obtain updated video data to be labeled; based on the metadata labeling record information and the content recognition identifier, labeling and storing the updated video data to be labeled in a preset storage device. This implementation improves the video quality of the stored video data.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, apparatus, device, and medium for storing video data based on content recognition. Background Art

[0002] Video data storage is a technology for storing video data. At present, when storing video data, the method usually adopted is: directly marking the format or author identification of the video as a tag of the video data and then storing it.

[0003] However, when the above method is used to store video data, the following technical problems often occur:

[0004] First, the video format or author ID is directly marked as a label of the video data before storage. The video format or author ID can only provide basic information about the video encoding method or the creator's identity identification, and cannot reflect the essence of the video content. When the video format or ID is used as a label to retrieve the stored video data, due to the single label information and lack of content relevance, the retrieval results may contain a large number of videos that do not match the user's needs. As a result, the retrieved videos have a low degree of match with user needs. At the same time, after the video format or author ID is marked as a label of the video data, the low-resolution video frame set is not enhanced during the storage process, resulting in poor video quality of the stored video data.

[0005] Continuing, in the process of adopting technical solutions to solve the above-mentioned technical problem 1, the following technical problems often occur: the conventional solution for image quality enhancement of a video frame set with a lower resolution is generally to convert each video frame in the video frame set with a lower resolution into a video frame set with a higher resolution through a video super-resolution model to obtain a video frame set with a higher resolution. However, the above-mentioned conventional solution converts each video frame in the video frame set with a lower resolution into a video frame set with a higher resolution through a video super-resolution model to obtain a video frame set with a higher resolution, but still has the following problems:

[0006] Second, each video frame in a low-resolution video frame set is converted into a high-resolution video frame set through a video super-resolution model. Although the resolution of each video frame in the video frame set is improved, the video is composed of continuous frames, and each frame contains dynamic information. Directly enlarging the resolution without considering the motion relationship between frames may cause the converted high-resolution video to have inter-frame discontinuities in dynamic scenes, causing fast-moving objects to appear blurred, resulting in a poor viewing experience of the video.

[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0008] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0009] Some embodiments of the present disclosure propose a video data storage method, device, electronic device, and computer-readable medium based on content recognition to solve one or more of the technical problems mentioned in the above background technology section.

[0010] In a first aspect, some embodiments of the present disclosure provide a method for storing video data based on content recognition, the method comprising: collecting, from each preset video data source, each video data to be annotated corresponding to each video as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data and metadata information; for each video data to be annotated in the above video data set to be annotated, performing the following annotation and storage steps: performing metadata annotation processing on the above video data to be annotated to obtain metadata annotation record information corresponding to the above video data to be annotated, wherein the above metadata annotation record information includes basic annotation information, video format annotation information, Resolution annotation information, duration annotation information; determining the resolution information corresponding to the above resolution annotation information as the target resolution; in response to determining that the above target resolution is less than or equal to the preset resolution, performing image quality enhancement processing on the video frame set to be annotated in the above video data to be annotated, so as to obtain an enhanced video frame set to be annotated; performing content recognition processing on the above enhanced video frame set to be annotated, so as to obtain a content recognition identifier corresponding to the above video data to be annotated; based on the above enhanced video frame set to be annotated, updating processing is performed on the above video data to be annotated, so as to obtain updated video data to be annotated; based on the above metadata annotation record information and the above content recognition identifier, the above updated video data to be annotated is labeled and stored in a preset storage device.

[0011] In a second aspect, some embodiments of the present disclosure provide a video data storage device based on content recognition, the device comprising: an acquisition unit, configured to collect each video data to be annotated corresponding to each video from each preset video data source as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data and metadata information; an annotation storage unit, configured to perform the following annotation storage steps for each video data to be annotated in the above video data set to be annotated: perform metadata annotation processing on the above video data to be annotated to obtain metadata annotation record information corresponding to the above video data to be annotated, wherein the above metadata annotation record information includes basic annotation information, Video format annotation information, resolution annotation information, and duration annotation information; determining the resolution information corresponding to the above resolution annotation information as the target resolution; in response to determining that the above target resolution is less than or equal to the preset resolution, performing image quality enhancement processing on the video frame set to be annotated in the above video data to be annotated to obtain an enhanced video frame set to be annotated; performing content recognition processing on the above enhanced video frame set to be annotated to obtain a content recognition identifier corresponding to the above video data to be annotated; based on the above enhanced video frame set to be annotated, updating processing is performed on the above video data to be annotated to obtain updated video data to be annotated; based on the above metadata annotation record information and the above content recognition identifier, the above updated video data to be annotated is labeled and stored in a preset storage device.

[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0014] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the video data storage method based on content recognition of some embodiments of the present disclosure, the video quality of the stored video data is improved. Specifically, the reasons for the poor video quality of the stored video data and the low matching degree between the retrieved video and the user's needs are: the video format or author identification is directly marked as a label of the video data and then stored. The video format or author identification can only provide basic information about the video encoding method or the creator's identity identification, and cannot reflect the essence of the video content. When the stored video data is retrieved using the video format or identification as a label, due to the single label information and lack of content relevance, the retrieval results may contain a large number of videos that do not match the user's needs. As a result, the retrieved video has a low matching degree with the user's needs. At the same time, after the video format or author identification is marked as a label of the video data, the video frame set with lower resolution is not enhanced during the storage process, resulting in poor video quality of the stored video data. Based on this, in some embodiments of the present disclosure, the video data storage method based on content recognition first collects each video data to be annotated corresponding to each video from each preset video data source as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data and metadata information. Thus, each video data to be annotated can be collected. Afterwards, for each video data to be annotated in the above video data set to be annotated, the following annotation storage step is performed: metadata annotation processing is performed on the above video data to be annotated to obtain metadata annotation record information corresponding to the above video data to be annotated, wherein the above metadata annotation record information includes basic annotation information (for example, author name), video format annotation information, resolution annotation information, and duration annotation information. Thus, metadata annotation processing can be performed on the video data to be annotated, and metadata annotation record information corresponding to the video data to be annotated is generated. The above metadata annotation record information can represent each basic information of the video data to be annotated (for example, author name, video format, resolution, duration). Afterwards, the resolution information corresponding to the above resolution annotation information is determined as the target resolution. Thus, the target resolution of the video data to be annotated can be determined. Next, in response to determining that the target resolution is less than or equal to the preset resolution, the video frame set to be labeled in the video data to be labeled is subjected to image quality enhancement processing to obtain an enhanced video frame set to be labeled. Thus, the image quality of the video frames to be labeled in the video data to be labeled can be enhanced, thereby improving the video quality of the video data to be labeled. Then, the enhanced video frame set to be labeled is subjected to content recognition processing to obtain a content recognition identifier corresponding to the video data to be labeled. Thus, the enhanced video frame set to be labeled that represents the video to be labeled after image quality enhancement can be subjected to content recognition to generate a content recognition identifier representing the video content corresponding to the video data to be labeled.Afterwards, based on the enhanced video frame set to be annotated, the video data to be annotated is updated to obtain updated video data to be annotated. Thus, updated video data to be annotated containing the enhanced video frame set to be annotated can be generated. Next, based on the metadata annotation record information and the content identification mark, the updated video data to be annotated is tagged and stored in a preset storage device. Thus, the updated video data to be annotated after the image quality is enhanced can be tagged according to the metadata annotation record information and the content identification mark representing the basic information of the video data to be annotated (for example, the author's name, the video format, the resolution, and the duration), so that the change information of the updated video data to be annotated is richer, including not only the basic information, but also the content identification mark representing the essence of the video content. Thus, the matching degree between each video obtained through user needs and label retrieval in the subsequent retrieval process and the user needs is improved. Also, because the image quality enhancement process is adopted for the video frame set to be annotated with a lower resolution during the storage process, the video quality of the stored video data is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0016] Figure 1 is a flowchart of some embodiments of the video data storage method based on content recognition according to the present disclosure;

[0017] Figure 2 is a schematic structural diagram of some embodiments of a video data storage device based on content recognition according to the present disclosure;

[0018] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 The process 100 of some embodiments of the video data storage method based on content recognition according to the present disclosure is shown. The video data storage method based on content recognition includes the following steps:

[0026] Step 101 : collect various video data to be labeled corresponding to various videos from various preset video data sources as video data sets to be labeled.

[0027] In some embodiments, the execution subject (e.g., video distribution server) of the content recognition-based video data storage method may collect the video data to be annotated corresponding to each video from each preset video data source through a streaming media transmission protocol as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data, and metadata information. The above preset video data sources may be the source of the video data to be annotated. For example, the above preset video data sources may include a video database, a media server, and a camera. The above video data to be annotated may represent each unannotated video. The above metadata information may be additional information attached to the video file. The above metadata information may include at least one of the following: basic information (e.g., title, author name), technical details information (e.g., resolution information (e.g., 720x480 pixels), frame rate, bit rate, video format information (e.g., H.264, HEVC)), video duration information (e.g., 4min), cover image (e.g., a thumbnail or cover picture of the video), and geographic location information (e.g., the geographic location coordinates of the video shooting (if any)).

[0028] Step 102: for each video data to be annotated in the video data set to be annotated, perform the following annotation and storage steps:

[0029] Step 1021 , performing metadata annotation processing on the video data to be annotated, and obtaining metadata annotation record information corresponding to the video data to be annotated.

[0030] In some embodiments, the execution subject may perform metadata annotation processing on the video data to be annotated to obtain metadata annotation record information corresponding to the video data to be annotated, wherein the metadata annotation record information includes basic annotation information, video format annotation information, resolution annotation information, and duration annotation information. The basic annotation information may indicate the field name and row identifier corresponding to the location where the basic information is stored in the preset metadata table. For example, the basic annotation information may be "field name: basic information (e.g., title, author name), row identifier information: 20". The basic information may indicate the title or author name of the video corresponding to the video data to be annotated. The video format annotation information indicates the field name and row identifier corresponding to the location where the resolution information is stored in the preset metadata table. The resolution information may indicate the resolution of the video corresponding to the video data to be annotated. The duration annotation information may indicate the field name and row identifier corresponding to the location where the video duration information is stored in the preset metadata table. The video duration information may indicate the playback duration of the video corresponding to the video data to be annotated.

[0031] In some optional implementations of some embodiments, the execution subject may perform metadata annotation processing on the video data to be annotated through the following steps to obtain metadata annotation record information corresponding to the video data to be annotated:

[0032] In the first step, the metadata information in the video data to be annotated is determined as the information to be extracted.

[0033] The second step is to extract basic information from the information to be extracted, and enter the basic information into a preset metadata table. The extracted basic information corresponds to a first preset field name and row identification information. In practice, the execution subject can extract basic information from the information to be extracted by named entity recognition (NER). The execution subject can map the extracted basic information to a cell corresponding to the basic information in the preset metadata table by data mapping technology. Optionally, the execution subject can execute an entry task corresponding to the preset entry information to enter the basic information into the preset metadata table. The preset entry information can be a programming language code corresponding to the basic information for storing the basic information in a cell corresponding to the basic information in the preset metadata table. As an example, the entry information can be "UPDATE metadata; SET author name = 'Xiao Ming'; WHERE author name ISNULL; ORDERBYid; LIMIT1;". The preset metadata table can be a data table for storing the extracted basic information, video format information, resolution information, and video duration information. The above-mentioned row identification information may be the row number corresponding to the cell of the preset metadata table in which the extracted information is entered.

[0034] The third step is to determine the row identification information corresponding to the first preset field name and basic information as basic marking information.

[0035] The fourth step is to extract the video format information from the above-mentioned information to be extracted, and enter the above-mentioned video format information into the above-mentioned preset metadata table, wherein the extracted video format information corresponds to the second preset field name and row identification information. In practice, the above-mentioned execution subject can extract the video format information from the above-mentioned information to be extracted through Named Entity Recognition (NER for short). Afterwards, the above-mentioned execution subject can traverse the vertical list corresponding to the second preset field name corresponding to the above-mentioned video format information in the above-mentioned preset metadata table from top to bottom, and determine the cell in the first vertical list with empty data traversed as the cell to be inserted. Then, the above-mentioned execution subject can enter the video format information into the cell to be inserted. Among them, the above-mentioned video format information can represent the encoding format of the video. For example, the above-mentioned video format information can be "H.264". The above-mentioned second preset field name can be "video format".

[0036] The fifth step is to determine the row identification information corresponding to the second preset field name and the video format information as the video format marking information.

[0037] The sixth step is to extract the resolution information from the above-mentioned information to be extracted, and enter the above-mentioned resolution information into the above-mentioned preset metadata table. The extracted resolution information corresponds to the third preset field name and row identification information. In practice, the above-mentioned execution subject can extract the resolution information from the above-mentioned information to be extracted through Named Entity Recognition (NER for short). After that, the above-mentioned execution subject can traverse the vertical list corresponding to the third preset field corresponding to the above-mentioned resolution information in the above-mentioned preset metadata table from top to bottom, and determine the cell in the first vertical list with empty data as the cell to be inserted. Then, the above-mentioned execution subject can enter the resolution information into the cell to be inserted. The above-mentioned resolution information can represent the resolution of the video. For example, the above-mentioned resolution information can be 720x480 pixels. The above-mentioned third preset field name can be "resolution".

[0038] In the seventh step, the row identification information corresponding to the third preset field name and the resolution information is determined as the resolution label information.

[0039] The eighth step is to extract the video duration information from the above-mentioned information to be extracted, and enter the above-mentioned video duration information into the above-mentioned preset metadata table, wherein the extracted video duration information corresponds to the fourth preset field name and row identification information. In practice, the above-mentioned execution subject can extract the video duration information from the above-mentioned information to be extracted through Named Entity Recognition (NER for short). Afterwards, the above-mentioned execution subject can traverse the vertical list corresponding to the fourth preset field corresponding to the above-mentioned resolution information in the above-mentioned preset metadata table from top to bottom, and determine the cell in the first traversed vertical list with empty data as the cell to be inserted. Then, the above-mentioned execution subject can enter the video duration information into the cell to be inserted. Among them, the above-mentioned video duration information can represent the playback duration of the video. The above-mentioned fourth preset field name can be "video duration".

[0040] In the ninth step, the row identification information corresponding to the fourth preset field name and the video duration information is determined as duration marking information.

[0041] In the tenth step, the basic annotation information, the video format annotation information, the resolution annotation information, and the duration annotation information are determined as metadata annotation record information.

[0042] Step 1022: determine the resolution information corresponding to the resolution annotation information as the target resolution.

[0043] In some embodiments, the execution entity may determine the resolution information corresponding to the resolution annotation information as the target resolution.

[0044] Step 1023 , in response to determining that the target resolution is less than or equal to the preset resolution, performing image quality enhancement processing on the to-be-annotated video frame set in the to-be-annotated video data to obtain an enhanced to-be-annotated video frame set.

[0045] In some embodiments, in response to determining that the target resolution is less than or equal to a preset resolution, the execution subject may perform image quality enhancement processing on the video frame set to be labeled in the video data to be labeled to obtain an enhanced video frame set to be labeled.

[0046] In some optional implementations of some embodiments, the execution subject may perform image quality enhancement processing on the video frame set to be annotated in the video data to be annotated through the following steps to obtain an enhanced video frame set to be annotated:

[0047] In the first step, the video frame set to be labeled is input into a pre-trained video super-resolution model to obtain a super-resolution video frame set. The super-resolution video frames in the super-resolution video frame set correspond to the video frames to be labeled in the video frame set to be labeled. A super-resolution video frame in the super-resolution video frame set can be a video frame obtained by converting the video frame to be labeled into a high-resolution video frame.

[0048] In the second step, the super-resolution video frame set is determined as a super-resolution video frame sequence to be divided.

[0049] The third step is to determine the first preset value as the sliding window size. For example, the sliding window size may be 2.

[0050] Step 4: Determine the second preset value as the sliding step length. For example, the sliding step length may be 1.

[0051] In the fifth step, based on the above sliding window size and the above sliding step size, the super-resolution video frame sequence to be divided is divided in a sliding window manner to obtain a super-resolution video frame group sequence. Wherein, each super-resolution video frame group in the super-resolution video frame group sequence includes a preceding super-resolution video frame and a succeeding super-resolution video frame. The preceding super-resolution video frame may be the first video frame in the super-resolution video frame group. The succeeding super-resolution video frame may be the second video frame in the super-resolution video frame group. As an example, the super-resolution video frame sequence to be divided may be {super-resolution video frame A to be divided, super-resolution video frame B to be divided, super-resolution video frame C to be divided}, then the super-resolution video frame sequence to be divided is divided in a sliding window manner, and the obtained identification groups to be checked may be {super-resolution video frame A to be divided, super-resolution video frame B to be divided}, {super-resolution video frame B to be divided, super-resolution video frame C to be divided}.

[0052] Step 6: For the first super-resolution video frame group in the super-resolution video frame group sequence, perform the following determination process:

[0053] In the first sub-step, optical flow estimation is performed on the super-resolution video frame group to obtain optical flow vector field information. The optical flow vector field information includes various vectors. The optical flow vector field information may represent the moving direction and speed of each pixel between the preceding super-resolution video frame and the succeeding super-resolution video frame in the super-resolution video frame group. Each vector represents the displacement of a pixel point in time, including horizontal and vertical components.

[0054] In the second sub-step, based on the optical flow vector field information, motion compensation is performed on the preceding super-resolution video frame in the super-resolution video frame group to obtain a motion-compensated predicted video frame. The motion-compensated predicted video frame may be a video frame generated by performing motion-compensated prediction on the preceding super-resolution video frame using the optical flow vector field.

[0055] In the third sub-step, a subsequent super-resolution video frame in the super-resolution video frame group is determined as a target super-resolution video frame.

[0056] The fourth sub-step is to generate a corresponding smooth video frame based on the motion compensation prediction video frame and the target super-resolution video frame. In practice, the above execution subject can perform weighted average processing on the motion compensation prediction video frame and the target super-resolution video frame through linear interpolation technology to obtain a smooth video frame.

[0057] In a fifth sub-step, the smoothed video frame is added to a position in the super-resolution video frame group that is located after the preceding super-resolution video frame and before the succeeding super-resolution video frame, so as to update the super-resolution video frame group.

[0058] In a sixth sub-step, the updated super-resolution video frame group is determined as the updated super-resolution video frame group.

[0059] In a seventh sub-step, the super-resolution video frame group is deleted from the super-resolution video frame group sequence to update the super-resolution video frame group sequence.

[0060] An eighth sub-step, in response to determining that the updated super-resolution video frame group sequence is not empty, performing the above determination step again according to the updated super-resolution video frame group sequence.

[0061] In the seventh step, in response to determining that the updated super-resolution video frame group sequence is empty, an enhanced video frame set to be labeled is generated based on the determined updated super-resolution video frame groups. In practice, the above-mentioned execution subject may sort and combine the super-resolution video frame groups corresponding to the updated super-resolution video frame groups in the order in the super-resolution video frame group sequence to obtain an updated super-resolution video frame group sequence, and then the above-mentioned execution subject may deduplicate the updated super-resolution video frames in the updated super-resolution video frame group sequence to obtain each deduplicated updated super-resolution video frame group sequence. Afterwards, the above-mentioned execution subject may arrange the order of the deduplicated updated super-resolution video frames in the deduplicated updated super-resolution video frame group sequence in the group sequence to obtain a deduplicated updated super-resolution video frame sequence. Finally, the above-mentioned execution subject may determine the deduplicated updated super-resolution video frame sequence as the enhanced video frame set to be labeled. As an example, the above-mentioned updated super-resolution video frame groups may be “{super-resolution video frame A to be divided, super-resolution video frame AB to be divided, super-resolution video frame B to be divided}, {super-resolution video frame B to be divided, super-resolution video frame BC to be divided, super-resolution video frame C to be divided}”. The above-mentioned deduplication updated super-resolution video frame group sequences may be “{super-resolution video frame A to be divided, super-resolution video frame AB to be divided, super-resolution video frame B to be divided}, {super-resolution video frame BC to be divided, super-resolution video frame C to be divided}”. The above-mentioned deduplication updated super-resolution video frame sequence may be “{super-resolution video frame A to be divided, super-resolution video frame AB to be divided, super-resolution video frame B to be divided, super-resolution video frame BC to be divided, super-resolution video frame C to be divided}”.

[0062] The above technical solution and its related contents, as an inventive point of an embodiment of the present disclosure, solve the second technical problem mentioned in the background technology, "Using a video super-resolution model to convert each video frame in a video frame set with a low resolution into a high-resolution video frame set to obtain a high-resolution video frame set. Although the resolution of each video frame in the video frame set is improved, the video is composed of continuous frames, and each frame contains dynamic information. Directly enlarging the resolution without considering the motion relationship between frames may cause the converted high-resolution video to have inter-frame discontinuity in dynamic scenes, causing fast-moving objects to appear blurred. This results in a poor viewing experience of the video." The factors that lead to the poor viewing experience of the video are often as follows: Using a video super-resolution model to convert each video frame in a video frame set with a low resolution into a high-resolution video frame set to obtain a high-resolution video frame set. Although the resolution of each video frame in the video frame set is improved, the video is composed of continuous frames, and each frame contains dynamic information. Directly enlarging the resolution without considering the motion relationship between frames may cause the converted high-resolution video to have inter-frame discontinuity in dynamic scenes, causing fast-moving objects to appear blurred. This results in a poor viewing experience of the video. If the above factors are solved, the effect of improving the viewing experience of the video can be achieved. In order to achieve this effect, first, the above-mentioned video frame set to be labeled is input into a pre-trained video super-resolution model to obtain a super-resolution video frame set. In this way, the resolution of the video frames in the video frame set to be labeled can be improved. Then, the above-mentioned super-resolution video frame set is determined as a super-resolution video frame sequence to be divided. In this way, a super-resolution video frame sequence to be divided for generating a super-resolution video frame group sequence can be obtained. Afterwards, the first preset value is determined as the sliding window size. In this way, the sliding window size for division processing can be obtained. Then, the second preset value is determined as the sliding step size. In this way, the sliding step size for division can be obtained. Afterwards, based on the above-mentioned sliding window size and the above-mentioned sliding step size, the above-mentioned super-resolution video frame sequence to be divided is divided in a sliding window manner to obtain a super-resolution video frame group sequence, wherein each super-resolution video frame group in the above-mentioned super-resolution video frame group sequence includes a preceding super-resolution video frame and a subsequent super-resolution video frame. In this way, a super-resolution video frame group sequence of the enhanced video frame set to be labeled can be obtained. Next, for the first super-resolution video frame group in the super-resolution video frame group sequence, the following determination process is performed: In the first sub-step, the super-resolution video frame group is subjected to optical flow estimation process to obtain optical flow vector field information. Thus, the motion information of each pixel point from the preceding super-resolution video frame in the super-resolution video frame group to the succeeding super-resolution video frame can be obtained. In the second sub-step, based on the optical flow vector field information, the preceding super-resolution video frame in the super-resolution video frame group is subjected to motion compensation process to obtain a motion compensated predicted video frame.Thus, a more accurate motion compensation prediction video frame can be generated by performing motion compensation on the preceding super-resolution video frame. Afterwards, the subsequent super-resolution video frame in the super-resolution video frame group is determined as the target super-resolution video frame. Then, based on the motion compensation prediction video frame and the target super-resolution video frame, a corresponding smooth video frame is generated. Thus, the generated smooth video frame takes into account the motion change between two consecutive frames, reduces motion blur and artifacts, makes the dynamic objects in the video clearer, and reduces the visual jump and incoherence. Then, the smooth video frame is added to the position after the preceding super-resolution video frame and before the subsequent super-resolution video frame in the super-resolution video frame group to update the super-resolution video frame group. Thus, the smooth video frame can be added to the super-resolution video frame group. Then, the updated super-resolution video frame group is determined as the updated super-resolution video frame group. Then, the super-resolution video frame group is deleted from the super-resolution video frame group sequence to update the super-resolution video frame group sequence. Next, in response to determining that the updated super-resolution video frame group sequence is not empty, the above determination step is performed again according to the updated super-resolution video frame group sequence. In response to determining that the updated super-resolution video frame group sequence is empty, an enhanced video frame set to be labeled is generated based on each determined updated super-resolution video frame group. Thus, an enhanced video frame set to be labeled with each smooth video frame added is generated, the motion change between two consecutive frames is taken into account, motion blur and artifacts are reduced, dynamic objects in the video are clearer, visual jumps and incoherence are reduced, and the viewing experience of the video is improved.

[0063] Step 1024 , performing content recognition processing on the enhanced video frame set to be labeled, and obtaining a content recognition identifier corresponding to the video data to be labeled.

[0064] In some embodiments, the execution subject may perform content recognition processing on the enhanced video frame set to be annotated to obtain a content recognition identifier corresponding to the video data to be annotated.

[0065] In some optional implementations of some embodiments, the execution subject may perform content recognition processing on the enhanced video frame set to be annotated through the following steps to obtain a content recognition identifier corresponding to the video data to be annotated:

[0066] In the first step, each enhanced video frame to be labeled in the enhanced video frame set to be labeled is input into a pre-trained image recognition model to obtain the object identification corresponding to the enhanced video frame to be labeled. The image recognition model can be a convolutional neural network (CNN) model that takes the enhanced video frame to be labeled as input data and the object identification as output data. The object identification can be the identification of each object in the enhanced video frame to be labeled. For example, the object identification can be "car", "river", etc.

[0067] In the second step, each obtained object identifier is deduplicated to obtain at least one deduplicated object identifier.

[0068] The third step is to determine the at least one object identifier as a content identification identifier.

[0069] Step 1025 , based on the enhanced video frame set to be labeled, update the video data to be labeled to obtain updated video data to be labeled.

[0070] In some embodiments, the execution subject may update the video data to be annotated based on the enhanced video frame set to be annotated to obtain updated video data to be annotated.

[0071] In some optional implementations of some embodiments, the execution subject may update the video data to be annotated based on the enhanced video frame set to be annotated through the following steps to obtain updated video data to be annotated:

[0072] In some optional implementations of some embodiments, the execution subject may update the video data to be annotated by the following steps to obtain updated video data to be annotated:

[0073] In the first step, the video frame set to be labeled is removed from the video data to be labeled, and the enhanced video frame set to be labeled is added to the video data to be labeled, so as to update the video data to be labeled.

[0074] In the second step, the updated video data to be labeled is determined as updated video data to be labeled.

[0075] Step 1026: Based on the metadata annotation record information and the content identification mark, the updated video data to be annotated is labeled and stored in a preset storage device.

[0076] In some embodiments, the execution subject may store the updated video data to be annotated in a label format in a preset storage device based on the metadata annotation record information and the content identification identifier.

[0077] In some optional implementations of some embodiments, the execution subject may store the updated video data to be annotated in a label format in a preset storage device based on the metadata annotation record information and the content identification identifier through the following steps:

[0078] The first step is to obtain basic information corresponding to the basic annotation information included in the metadata annotation record information from the preset metadata table, and determine the basic information as the basic information label of the updated video data to be annotated. In practice, the execution subject can obtain the basic information from the preset metadata table through the field name and row identification information in the basic annotation information.

[0079] The second step is to obtain video format information corresponding to the video format annotation information included in the metadata annotation record information from the preset metadata table, and determine the video format information as the video format label of the updated video data to be annotated.

[0080] The third step is to obtain, from the preset metadata table, resolution information corresponding to the resolution annotation information included in the metadata annotation record information, and determine the resolution information as the resolution tag of the updated video data to be annotated.

[0081] The fourth step is to obtain the video duration information corresponding to the duration annotation information included in the metadata annotation record information from the preset metadata table, and determine the video duration information as the video duration label of the updated video data to be annotated.

[0082] The fifth step is to store the basic information tag, the video format tag, the resolution tag and the video duration tag into a preset index table in a preset storage device.

[0083] Step 6: determine the content identification identifier as the content identification tag corresponding to the updated video data to be annotated.

[0084] The seventh step is to store the updated video data to be annotated corresponding to the basic information tag, the video format tag, the resolution tag, the content identification tag and the video duration tag in a preset storage file of a preset storage device. In practice, the execution entity may store the basic information tag, the video format tag, the resolution tag, the content identification tag and the video duration tag as attribute information of the updated video data to be annotated. Then, the execution entity may create a preset storage file with the attribute information as the file name in the preset storage device. Afterwards, the execution entity may store the updated video data to be annotated in the preset storage file of the preset storage device.

[0085] Optionally, the above execution entity may further perform the following steps:

[0086] The first step is to obtain a video viewing data set within a preset time corresponding to the above-mentioned client in response to receiving information sent by the client indicating that the client has started the target application, wherein each video viewing data in the above-mentioned video viewing data set corresponds to a video. Among them, the above-mentioned target application may be an application for playing videos. In practice, the above-mentioned execution subject may obtain the video viewing data set from a preset log file. Each video viewing data in the above-mentioned video viewing data set includes user viewing record data and user viewing behavior record data. The above-mentioned user viewing record data may be a video identifier of a video watched by a user. The above-mentioned user viewing behavior record data includes various operation record information. Each operation record information in the above-mentioned various operation record information may represent the user's operation behavior (for example, click, pause, fast forward, replay) during the user's viewing of the video corresponding to the above-mentioned video viewing data.

[0087] The second step is to determine each video corresponding to the video viewing data set as an interested video set, wherein the interested videos in the interested video set correspond one-to-one to the video viewing data in the video viewing data set.

[0088] The third step is to generate a user portrait based on the above-mentioned video set of interest and the above-mentioned video viewing data set. The number of operation record information included in each video viewing data in the above-mentioned video viewing data set is determined as the interest value. The individual video viewing data in the above-mentioned video viewing data set whose interest value is greater than or equal to the preset interest value is determined as each target video viewing data. The individual videos of interest corresponding to the above-mentioned target video viewing data in the above-mentioned video set of interest are determined as each target video of interest. Content recognition is performed on each video frame set corresponding to each of the above-mentioned target videos of interest to obtain each object identifier. Clustering processing is performed on the above-mentioned object identifiers to obtain each cluster of interested identification identifiers. The object identifier and the number of object identifiers of each cluster of interested identification identifiers in the above-mentioned clusters of interested identification identifiers are converted into a feature vector (a vector representing the object identifier and the number of object identifiers of the cluster of interested identification identifiers) through word embedding technology. The generated feature vectors are determined as user portraits.

[0089] In the fourth step, in response to receiving the event information sent by the client and including the target application playing the current video, the video corresponding to the event information is determined as the currently playing video.

[0090] In the fifth step, based on the user portrait and the currently played video, at least one retrieved video data corresponding to the user portrait and the currently played video is retrieved from the preset storage device. In practice, first, the execution subject may perform content recognition processing on the video frame set corresponding to the currently played video to obtain each object identifier as each interested object identifier. Then, the execution subject may determine each feature vector corresponding to the user portrait as each target feature vector. Afterwards, the execution subject may determine each target feature vector in which the number of object identifiers represented in each target feature vector is greater than a preset value as each target interested feature vector. Afterwards, each object identifier represented by each target interested feature vector may be determined as each interested object identifier. Afterwards, the execution subject may retrieve each file name corresponding to each stored preset storage file from the preset storage device. In response to determining that at least one file name among the various file names includes at least one interested object identifier among the determined interested object identifiers, at least one preset storage file corresponding to the at least one file name is determined as at least one target preset storage file. Afterwards, the execution subject may determine the video data stored in the at least one target preset storage file as at least one retrieved video data.

[0091] Step 6: Push the at least one retrieved video data to the client.

[0092] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the video data storage method based on content recognition of some embodiments of the present disclosure, the video quality of the stored video data is improved. Specifically, the reasons for the poor video quality of the stored video data and the low matching degree between the retrieved video and the user's needs are: the video format or author identification is directly marked as a label of the video data and then stored. The video format or author identification can only provide basic information about the video encoding method or the creator's identity identification, and cannot reflect the essence of the video content. When the stored video data is retrieved using the video format or identification as a label, due to the single label information and lack of content relevance, the retrieval results may contain a large number of videos that do not match the user's needs. As a result, the retrieved video has a low matching degree with the user's needs. At the same time, after the video format or author identification is marked as a label of the video data, the video frame set with lower resolution is not enhanced during the storage process, resulting in poor video quality of the stored video data. Based on this, in some embodiments of the present disclosure, the video data storage method based on content recognition first collects each video data to be annotated corresponding to each video from each preset video data source as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data and metadata information. Thus, each video data to be annotated can be collected. Afterwards, for each video data to be annotated in the above video data set to be annotated, the following annotation storage step is performed: metadata annotation processing is performed on the above video data to be annotated to obtain metadata annotation record information corresponding to the above video data to be annotated, wherein the above metadata annotation record information includes basic annotation information (for example, author name), video format annotation information, resolution annotation information, and duration annotation information. Thus, metadata annotation processing can be performed on the video data to be annotated, and metadata annotation record information corresponding to the video data to be annotated is generated. The above metadata annotation record information can represent each basic information of the video data to be annotated (for example, author name, video format, resolution, duration). Afterwards, the resolution information corresponding to the above resolution annotation information is determined as the target resolution. Thus, the target resolution of the video data to be annotated can be determined. Next, in response to determining that the target resolution is less than or equal to the preset resolution, the video frame set to be labeled in the video data to be labeled is subjected to image quality enhancement processing to obtain an enhanced video frame set to be labeled. Thus, the image quality of the video frames to be labeled in the video data to be labeled can be enhanced, thereby improving the video quality of the video data to be labeled. Then, the enhanced video frame set to be labeled is subjected to content recognition processing to obtain a content recognition identifier corresponding to the video data to be labeled. Thus, the enhanced video frame set to be labeled that represents the video to be labeled after image quality enhancement can be subjected to content recognition to generate a content recognition identifier representing the video content corresponding to the video data to be labeled.Afterwards, based on the enhanced video frame set to be annotated, the video data to be annotated is updated to obtain updated video data to be annotated. Thus, updated video data to be annotated containing the enhanced video frame set to be annotated can be generated. Next, based on the metadata annotation record information and the content identification mark, the updated video data to be annotated is tagged and stored in a preset storage device. Thus, the updated video data to be annotated after the image quality is enhanced can be tagged according to the metadata annotation record information and the content identification mark representing the basic information of the video data to be annotated (for example, the author's name, the video format, the resolution, and the duration), so that the change information of the updated video data to be annotated is richer, including not only the basic information, but also the content identification mark representing the essence of the video content. Thus, the matching degree between each video obtained through user needs and label retrieval in the subsequent retrieval process and the user needs is improved. Also, because the image quality enhancement process is adopted for the video frame set to be annotated with a lower resolution during the storage process, the video quality of the stored video data is improved.

[0093] Further references Figure 2 As an implementation of the methods shown in the figures, the present disclosure provides some embodiments of a video data storage device based on content recognition, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0094] like Figure 2As shown, some embodiments of the video data storage device 200 based on content recognition include: an acquisition unit 201 and an annotation storage unit 202. The acquisition unit 201 is configured to collect various video data to be annotated corresponding to each video from various preset video data sources as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data and metadata information; the annotation storage unit 202 is configured to perform the following annotation storage steps for each video data to be annotated in the above video data set to be annotated: perform metadata annotation processing on the above video data to be annotated, and obtain metadata annotation record information corresponding to the above video data to be annotated, wherein the above metadata annotation record information includes basic annotation information, video format annotation information, resolution annotation information , duration annotation information; determining the resolution information corresponding to the resolution annotation information as the target resolution; in response to determining that the target resolution is less than or equal to the preset resolution, performing image quality enhancement processing on the video frame set to be annotated in the video data to be annotated, and obtaining an enhanced video frame set to be annotated; performing content recognition processing on the enhanced video frame set to be annotated, and obtaining a content recognition identifier corresponding to the video data to be annotated; based on the enhanced video frame set to be annotated, performing update processing on the video data to be annotated, and obtaining updated video data to be annotated; based on the metadata annotation record information and the content recognition identifier, storing the updated video data to be annotated in a labeled manner in a preset storage device.

[0095] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units contained therein, and will not be described in detail here.

[0096] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0098] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0099] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the functions defined in the method of some embodiments of the present disclosure are executed.

[0100] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0101] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0102] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: collects each video data to be annotated corresponding to each video from each preset video data source as a video data set to be annotated, wherein each video data to be annotated in the above video data set to be annotated includes a video frame set to be annotated, audio data and metadata information; for each video data to be annotated in the above video data set to be annotated, the following annotation and storage steps are performed: metadata annotation processing is performed on the above video data to be annotated to obtain metadata annotation record information corresponding to the above video data to be annotated, wherein the above metadata annotation record information includes basic annotation information, video format annotation information, and video data format annotation information. information, resolution annotation information, and duration annotation information; determining the resolution information corresponding to the resolution annotation information as the target resolution; in response to determining that the target resolution is less than or equal to the preset resolution, performing image quality enhancement processing on the video frame set to be annotated in the video data to be annotated to obtain an enhanced video frame set to be annotated; performing content recognition processing on the enhanced video frame set to be annotated to obtain a content recognition identifier corresponding to the video data to be annotated; based on the enhanced video frame set to be annotated, updating processing is performed on the video data to be annotated to obtain updated video data to be annotated; based on the metadata annotation record information and the content recognition identifier, the updated video data to be annotated is labeled and stored in a preset storage device.

[0103] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit and a labeling storage unit. The names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the acquisition unit may also be described as "a unit for collecting each video data to be labeled corresponding to each video from each preset video data source as a video data set to be labeled".

[0106] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0107] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of technical features, but should also cover other technical solutions formed by any combination of technical features or their equivalent features without departing from the inventive concept. For example, a technical solution formed by replacing a feature with a technical feature with similar functions disclosed in the embodiments of the present disclosure (but not limited to).

Claims

1. A video data storage method based on content recognition, comprising: Collecting each video data to be labeled corresponding to each video from each preset video data source as a video data set to be labeled, wherein each video data to be labeled in the video data set to be labeled includes a video frame set to be labeled, audio data and metadata information; For each video data to be annotated in the video data set to be annotated, the following annotation and storage steps are performed: Performing metadata annotation processing on the video data to be annotated to obtain metadata annotation record information corresponding to the video data to be annotated, including: Determining metadata information in the video data to be annotated as information to be extracted; Extracting basic information from the information to be extracted, and entering the basic information into a preset metadata table, wherein the extracted basic information corresponds to a first preset field name and row identification information; Determine the row identification information corresponding to the first preset field name and the basic information as basic marking information; Extracting video format information from the information to be extracted, and entering the video format information into the preset metadata table, wherein the extracted video format information corresponds to a second preset field name and row identification information; Determine the row identification information corresponding to the second preset field name and the video format information as the video format marking information; Extracting resolution information from the information to be extracted, and entering the resolution information into the preset metadata table, wherein the extracted resolution information corresponds to a third preset field name and row identification information; Determine the row identification information corresponding to the third preset field name and the resolution information as the resolution label information; Extracting video duration information from the information to be extracted, and entering the video duration information into the preset metadata table, wherein the extracted video duration information corresponds to a fourth preset field name and row identification information; Determine the row identification information corresponding to the fourth preset field name and the video duration information as duration marking information; Determine the basic annotation information, the video format annotation information, the resolution annotation information, and the duration annotation information as metadata annotation record information; Determining the resolution information corresponding to the resolution annotation information as the target resolution; In response to determining that the target resolution is less than or equal to a preset resolution, performing image quality enhancement processing on the video frame set to be labeled in the video data to be labeled to obtain an enhanced video frame set to be labeled; Performing content recognition processing on the enhanced video frame set to be labeled to obtain a content recognition identifier corresponding to the video data to be labeled; Based on the enhanced video frame set to be labeled, updating the video data to be labeled to obtain updated video data to be labeled; Based on the metadata annotation record information and the content identification identifier, the updated video data to be annotated is labeled and stored in a preset storage device.

2. The method according to claim 1, wherein: The method further comprises: In response to receiving information sent by a client indicating that the client has started a target application, obtaining a video viewing data set within a preset time corresponding to the client, wherein each video viewing data in the video viewing data set corresponds to a video; Determine each video corresponding to the video viewing data set as an interested video set, wherein the interested videos in the interested video set correspond one-to-one to the video viewing data in the video viewing data set; Generate a user portrait based on the set of videos of interest and the video viewing dataset; In response to receiving the event information sent by the client and including the target application playing the current video, determining the video corresponding to the event information as the currently playing video; Based on the user portrait and the currently playing video, at least one retrieved video data corresponding to the user portrait and the currently playing video is retrieved from the preset storage device; The at least one retrieved video data is pushed to the client.

3. The method according to claim 1, wherein: The performing content recognition processing on the enhanced video frame set to be labeled to obtain a content recognition identifier corresponding to the video data to be labeled includes: Inputting each enhanced video frame to be labeled in the enhanced video frame set to a pre-trained image recognition model to obtain an object identifier corresponding to the enhanced video frame to be labeled; Deduplication processing is performed on each obtained object identifier to obtain at least one deduplication-processed object identifier; The at least one object identifier is determined as a content identification identifier.

4. The method according to claim 1, wherein: The updating process of the video data to be labeled based on the enhanced video frame set to be labeled to obtain updated video data to be labeled includes: Removing the video frame set to be labeled in the video data to be labeled, and adding the enhanced video frame set to be labeled to the video data to be labeled, so as to update the video data to be labeled; The updated video data to be labeled is determined as updated video data to be labeled.

5. The method according to claim 1, wherein: The step of labeling and storing the updated video data to be labeled in a preset storage device based on the metadata labeling record information and the content identification identifier includes: Acquire basic information corresponding to the basic annotation information included in the metadata annotation record information from the preset metadata table, and determine the basic information as the basic information tag of the updated video data to be annotated; Acquire video format information corresponding to the video format annotation information included in the metadata annotation record information from the preset metadata table, and determine the video format information as the video format label of the updated video data to be annotated; Acquire resolution information corresponding to the resolution annotation information included in the metadata annotation record information from the preset metadata table, and determine the resolution information as the resolution label of the updated video data to be annotated; Acquire video duration information corresponding to the duration annotation information included in the metadata annotation record information from the preset metadata table, and determine the video duration information as the video duration label of the updated video data to be annotated; Storing the basic information tag, the video format tag, the resolution tag, and the video duration tag in a preset index table in a preset storage device; Determining the content identification identifier as a content identification tag corresponding to the updated video data to be annotated; The updated video data to be annotated corresponding to the basic information tag, the video format tag, the resolution tag, the content identification tag and the video duration tag is stored in a preset storage file of a preset storage device.

6. A video data storage device based on content recognition, comprising: An acquisition unit is configured to collect each to-be-annotated video data corresponding to each video from each preset video data source as a to-be-annotated video data set, wherein each to-be-annotated video data in the to-be-annotated video data set includes a to-be-annotated video frame set, audio data, and metadata information; The annotation storage unit is configured to perform the following annotation storage steps for each video data to be annotated in the video data set to be annotated: perform metadata annotation processing on the video data to be annotated to obtain metadata annotation record information corresponding to the video data to be annotated, including: determining the metadata information in the video data to be annotated as information to be extracted; extracting basic information from the information to be extracted, and entering the basic information into a preset metadata table, wherein the extracted basic information corresponds to a first preset field name and row identification information; determining the first preset field name and the row identification information corresponding to the basic information as basic annotation information; extracting video format information from the information to be extracted, and entering the video format information into the preset metadata table, wherein the extracted video format information corresponds to a second preset field name and row identification information; determining the second preset field name and the row identification information corresponding to the video format information as video format annotation information; extracting resolution information from the information to be extracted, and entering the resolution information into the preset metadata table, wherein the extracted resolution information corresponds to a third preset field name and row identification information; determining the third preset field name and the row identification information corresponding to the resolution information is determined as the resolution annotation information; the video duration information is extracted from the information to be extracted, and the video duration information is entered into the preset metadata table, wherein the extracted video duration information corresponds to the fourth preset field name and the row identification information; the row identification information corresponding to the fourth preset field name and the video duration information is determined as the duration annotation information; the basic annotation information, the video format annotation information, the resolution annotation information, and the duration annotation information are determined as metadata annotation record information; the resolution information corresponding to the resolution annotation information is determined as the target resolution; in response to determining that the target resolution is less than or equal to the preset resolution, the video frame set to be annotated in the video data to be annotated is subjected to image quality enhancement processing to obtain an enhanced video frame set to be annotated; the enhanced video frame set to be annotated is subjected to content recognition processing to obtain a content recognition identifier corresponding to the video data to be annotated; based on the enhanced video frame set to be annotated, the video data to be annotated is updated to obtain updated video data to be annotated; based on the metadata annotation record information and the content recognition identifier, the updated video data to be annotated is labeled and stored in a preset storage device.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images