A video shot recommendation method

Abstract

The present invention discloses a video shot recommendation method, which includes the following steps: S1: video shot segmentation; S2: multi-dimensional labeling, usage statistics, and time statistics; S3: obtaining basic information data, frequency data, and freshness data of the video shots, and obtaining semantic similarity data, frequency coefficient, and freshness coefficient; S4: calculating and sorting comprehensive scores based on the semantic similarity data, usage frequency coefficient, and freshness coefficient to obtain a recommendation list. The video shots are segmented, multi-dimensionally labeled, and user usage frequency data is obtained by statistics on the shots. Time statistics are used to obtain shot freshness data, and semantic similarity, frequency coefficient, and freshness coefficient are obtained. Finally, the video shots are comprehensively sorted through a comprehensive score ranking calculation, taking into account factors such as semantic matching, usage, and time freshness. This ensures semantic accuracy, incorporates user usage preferences, and reflects the age of the videos, thereby achieving accurate video shot recommendations.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence video analysis and processing, and in particular to a video shot recommendation method. Background Art

[0002] With the rapid development of mobile internet in recent years, short videos have replaced traditional broadcast television as the primary means of communication for all major media outlets. However, for many short video production companies and individuals, the questions of "how to quickly find video footage that users will love," "how to find footage that is both relevant to the content and timely," and "how to accurately provide video footage" remain significant concerns.

[0003] The traditional method generally involves manual collection, organization, segmentation and manual labeling, and then screening and filtering from the collected footage during video editing. This method is not only inefficient, but also consumes a lot of manpower and material resources. The range of materials is very limited, and often valuable footage cannot be obtained. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a video shot recommendation method.

[0005] The purpose of the present invention is achieved through the following technical solutions: A video shot recommendation method comprises the following steps:

[0006] S1: Detect the boundary position of each shot in the video shot and segment the video according to the boundary position;

[0007] S2: Perform multi-dimensional labeling, usage statistics, and time statistics on the segmented video shots;

[0008] S3: Obtain basic information data of video shots by multi-dimensionally labeling them, and obtain semantic similarity data. Obtain frequency data of video shots by performing usage statistics on video shots, and obtain usage frequency coefficients. Obtain freshness data of video shots by performing time statistics on video shots, and obtain freshness coefficients.

[0009] S4: Calculate and sort the comprehensive scores based on the semantic similarity data, usage frequency coefficient, and freshness coefficient to obtain a list of recommended video shots.

[0010] Preferably, step S1 further includes the following steps:

[0011] S1.1: Calculate the difference between the red, green, and blue colors in the RGB color space between the two frames. The calculation formula is:

[0012]

[0013]

[0014]

[0015] Among them, H R Calculate the value of the histogram of R color, H G Calculate the value of the histogram of G color, H B Calculate the value of the histogram of B color, f i is the i-th image frame, f j is the jth image frame, D R (f i ,f j ) is the inter-frame difference of red in RGB color space between the i-th frame and the j-th frame, D G (f i ,f j ) is the inter-frame difference of green in RGB color space between the i-th frame and the j-th frame, D B (f i ,f j ) is the inter-frame difference of blue in RGB color space between the i-th frame and the j-th frame, and k is the bit resolution;

[0016] S1.2:

[0017] D(f i ,f j )=D R (f i ,f j )+D G (f i ,f j )+D B (f i ,f j );

[0018] Where D(f i ,f j ) is the inter-frame difference between the i-th frame and the j-th frame, when D(f i ,f j ) is greater than a preset threshold, the system determines that the frame is a shot cut point;

[0019] S1.3: Extract the video clip before the shot split point, perform preprocessing, and retain it as one shot;

[0020] S1.4: Store the lens file in the corresponding file system and enter the lens information into the database for indexing.

[0021] Preferably, in step S2, the multi-dimensional annotation includes speech recognition annotation, OCR text recognition annotation, face recognition annotation and object detection annotation.

[0022] Preferably, in step S2, basic information data is obtained after multi-dimensional annotation, and the basic information data is stored in a MySQL database. The binlog in MySQL is monitored through canal, and the monitored data changes are notified to the ES service, and the ES service updates the internal data.

[0023] Preferably, in step S3, semantic similarity values ​​are obtained through multi-field matching, and the video shots are sorted. The specific steps are:

[0024] S3.1.1: Create keyword-type shot_voice, shot_ocr, shot_person, and shot_things fields and synchronize data from MySQL.

[0025] S3.1.2: Configure the ES Chinese word segmentation plug-in;

[0026] S3.1.3: When setting ES to insert data, if the index name does not exist, it will automatically create the index;

[0027] S3.1.4: Use the ES built-in bool shouldquery scoring method to sort the four fields shot_voice, shot_ocr, shot_person, and shot_things to obtain the semantic similarity ranking and semantic similarity score. i .

[0028] Preferably, step S3 further includes the following steps:

[0029] S3.2.1: The frequency of lens usage is calculated by accumulating the number of times the lens is selected.

[0030] shot_usage i =∑shot i ;

[0031] After each shot is used, the system will i Update to the database;

[0032] S2.2.2: Store the frequency data in a MySQL database;

[0033] S2.2.3: Calculate the frequency coefficient k i ,

[0034]

[0035] Preferably, step S3 further includes the following steps:

[0036] S2.3.1: Count the video footage time and obtain the freshness data of the video footage.

[0037] D i =Day(Time now -shot_time i );

[0038] Among them, shot_time i Time is the shot time. now is the current query time;

[0039] S2.3.2: Calculate the freshness coefficient.

[0040]

[0041] Among them, K is the preservation constant.

[0042] Preferably, in step S4, the comprehensive score Score is calculated i ,

[0043] Score i =score i ×k i ×t i

[0044] Among them, score i is the semantic similarity score, k i is the frequency coefficient, t i It is the freshness coefficient.

[0045] Preferably, the data content is queried by establishing a reverse index.

[0046] The present invention has the following advantages: the present invention first divides the video shots, then labels the video shots through multiple dimensions, obtains shot usage frequency data by statistics on user shot usage, obtains shot freshness data through time statistics, and then obtains shot semantic similarity, shot usage frequency coefficient and shot freshness coefficient. Finally, the video shots are comprehensively ranked through comprehensive score ranking calculation, thereby comprehensively considering influencing factors such as semantic matching, usage and time freshness, ensuring semantic accuracy, including user usage preferences, and reflecting the newness or oldness of the video, thereby achieving accurate recommendation of video shots. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the structure of the video shot recommendation method process;

[0048] Figure 2 This is a structural diagram of basic information storage. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

[0051] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other.

[0052] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0053] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is typically placed when in use, or are the orientations or positional relationships commonly understood by those skilled in the art. These terms are intended only to facilitate the description of the present invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0054] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0055] In this embodiment, if Figure 1 As shown, a video shot recommendation method includes the following steps:

[0056] S1: Detect the boundary position of each shot in the video shot and segment the video according to the boundary position;

[0057] S2: Perform multi-dimensional labeling, usage statistics, and time statistics on the segmented video shots;

[0058] S3: Obtain basic information data of video shots by multi-dimensionally labeling them, and obtain semantic similarity data. Obtain frequency data of video shots by performing usage statistics on video shots, and obtain usage frequency coefficients. Obtain freshness data of video shots by performing time statistics on video shots, and obtain freshness coefficients.

[0059] S4: Calculate and sort the comprehensive scores based on semantic similarity data, usage frequency coefficient, and freshness coefficient to obtain a list of recommended video shots. First, the video shots are segmented, then labeled using multiple dimensions. User usage statistics are used to obtain lens usage frequency data, and time statistics are used to obtain lens freshness data. The semantic similarity, lens usage frequency coefficient, and lens freshness coefficient of the shots are then calculated. Finally, the video shots are ranked comprehensively using a comprehensive score ranking calculation. This comprehensively considers factors such as semantic matching, usage, and time freshness, ensuring semantic accuracy while also incorporating user preferences and reflecting the age of the videos, enabling accurate recommendations of video shots.

[0060] Furthermore, step S1 further includes the following steps:

[0061] S1.1: Calculate the difference between the red, green, and blue colors in the RGB color space between the two frames. The calculation formula is:

[0062]

[0063]

[0064]

[0065] Among them, H R Calculate the value of the histogram of R color, H G Calculate the value of the histogram of G color, H B Calculate the value of the histogram of B color, f i is the i-th image frame, f j is the jth image frame, D R (f i ,f j ) is the inter-frame difference of red in RGB color space between the i-th frame and the j-th frame, D G (f i ,fj ) is the inter-frame difference of green in RGB color space between the i-th frame and the j-th frame, D B (f i ,f j ) is the inter-frame difference of blue in the RGB color space between the i-th frame and the j-th frame, and k is the bit resolution. Specifically, k is the bit resolution, which refers to the number of bits for each color component in a pixel. In this embodiment, k is 8, that is, each color component has 256 possibilities.

[0066] S1.2:

[0067] D(f i ,f j )=D R (f i ,f j )+D G (f i ,f j )+D B (f i ,f j );

[0068] Where D(f i ,f j ) is the inter-frame difference between the i-th frame and the j-th frame, when D(f i ,f j ) is greater than a preset threshold, the system determines that the frame is a shot cut point;

[0069] S1.3: Extract the video clip before the shot split point, perform preprocessing, and retain it as one shot;

[0070] S1.4: Store the lens file in the corresponding file system and enter the lens information into the database for indexing.

[0071] In this embodiment, in step S2, multi-dimensional annotation includes voice recognition annotation, OCR text recognition annotation, face recognition annotation, and object detection annotation. Specifically, voice recognition annotation is achieved by extracting the sound of the video lens, then calling the voice recognition API to obtain the voice text content, realizing the voice recognition of the video lens, and storing it in the shot_voice field; OCR text recognition annotation specifies the corresponding text recognition area according to the different types of sources for the segmented video lens, and obtains the key information of the video lens by calling the OCR text recognition API, and stores it in the shot_ocr field. In another embodiment, in order to speed up recognition, only the first frame can be extracted for recognition according to actual conditions; face recognition annotation extracts the first ten frames of the video lens for face detection, and compares it with the preset key face database by calling the face recognition API. If there is data in the preset key face database, the name of the key person recognized is recorded and stored in the shot_person field; object detection annotation extracts the first ten frames of the lens, then calls the object detection API, records the detected object information, and stores it in the shot_things field.

[0072] Furthermore, in step S2, basic information data is obtained after multi-dimensional annotation, and the basic information data is stored in the MySQL database, and the binlog in MySQL is monitored through canal, that is, the MySQL binary log. The monitored data changes are notified to the ES service, and the ES service updates the internal data. Further, the data content is queried by establishing a reverse index. Specifically, since it is necessary to quickly query the data content, when the amount of data is large, it is necessary to establish a reverse index for fast query, and the MySQL database is not suitable. Therefore, it is necessary to synchronize the MySQL data to the ES (Elasticsearch) database to facilitate quick search. By adopting the MySQL and ES synchronization method in storage, MySQL ensures the normal operation of the business and the uniqueness of the data, and ES ensures fast and accurate retrieval.

[0073] In this embodiment, in step S3, semantic similarity values ​​are obtained through multi-field matching, and the video shots are sorted. The specific steps are as follows:

[0074] S3.1.1: Create keyword-type shot_voice, shot_ocr, shot_person, and shot_things fields and synchronize data from MySQL.

[0075] S3.1.2: Configure the ES Chinese word segmentation plug-in;

[0076] S3.1.3: When setting ES to insert data, if the index name does not exist, it will automatically create the index;

[0077] S3.1.4: Use the ES built-in bool should query scoring method to sort the four fields shot_voice, shot_ocr, shot_person, and shot_things to obtain the semantic similarity ranking and semantic similarity score. i Specifically, the bool should query scoring method is an existing method, which has not been improved here and will not be described in detail.

[0078] In this embodiment, step S3 further includes the following steps:

[0079] S3.2.1: The frequency of lens usage is calculated by accumulating the number of times the lens is selected.

[0080] shot_usage i =∑shot i ;

[0081] After each shot is used, the system will i Update to the database;

[0082] S2.2.2: Store the frequency data in a MySQL database;

[0083] S2.2.3: Calculate the frequency coefficient k i ,

[0084]

[0085] In this embodiment, step S3 further includes the following steps:

[0086] S2.3.1: Count the video footage time and obtain the freshness data of the video footage.

[0087] D i =Day(Time now -shot_time i );

[0088] Among them, shot_time i Time is the shot time. now is the current query time;

[0089] S2.3.2: Calculate the freshness coefficient.

[0090]

[0091] Specifically, since the system will continue to add new shots, the usage frequency of these new shots is 0 in the initial state, which will affect the ranking. Therefore, in order to ensure that the added new shots can also be recommended first, the recommendation algorithm is improved and the video shot freshness coefficient t is introduced. i Users can calculate the freshness based on K days according to actual conditions. For example, if K=7, the freshness is calculated based on a week. The freshness coefficient for shots within a week is between 1 and 2, and the freshness coefficient for shots over a week is 1.

[0092] Furthermore, in step S4, the comprehensive score Score is calculated. i ,

[0093] Score i =score i ×k i ×t i

[0094] Among them, score i is the semantic similarity score, k i is the frequency coefficient, t i It is the freshness coefficient.

[0095] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A video shot recommendation method, characterized by: The following steps are involved: S1: Detect the boundary position of each shot in the video shot and segment the video according to the boundary position; S2: Perform multi-dimensional labeling, usage statistics, and time statistics on the segmented video shots; S3: Obtain basic information data of video shots by multi-dimensionally labeling them, and obtain semantic similarity data. Obtain frequency data of video shots by performing usage statistics on video shots, and obtain usage frequency coefficients. Obtain freshness data of video shots by performing time statistics on video shots, and obtain freshness coefficients. S4: Calculate the comprehensive score based on the semantic similarity data, usage frequency coefficient and freshness coefficient, and sort them to obtain a list of recommended video shots; The step S1 further includes the following steps: S1.1: Calculate the difference between the red, green, and blue colors in the RGB color space between the two frames. The calculation formula is: ； ； ； in, Calculate the value for the histogram of R color, Calculate the value of the histogram of G color, Calculate the value for the histogram of B color, For the image frames, For the image frames, The red color in the RGB color space is Frame and The inter-frame difference between frames, Green in the RGB color space is Frame and The inter-frame difference between frames, The blue in the RGB color space is Frame and The inter-frame difference between frames, is the bit resolution; S1.2： ； in For the Frame and The inter-frame difference between frames, when When it is greater than the preset threshold, the system determines that the frame is a shot cut point; S1.3: Extract the video clip before the shot split point, perform preprocessing, and retain it as one shot; S1.4: Store the lens file in the corresponding file system and enter the lens information into the database for indexing.

2. The video shot recommendation method according to claim 1, wherein: In step S2, the multi-dimensional annotation includes speech recognition annotation, OCR text recognition annotation, face recognition annotation and object detection annotation.

3. The video shot recommendation method according to claim 2, wherein: In step S2, basic information data is obtained after multi-dimensional annotation, and the basic information data is stored in a MySQL database. The binlog in MySQL is monitored through canal, and the monitored data changes are notified to the ES service, and the ES service updates the internal data.

4. The video shot recommendation method according to claim 3, wherein: In step S3, semantic similarity values ​​are obtained through multi-field matching, and the video shots are sorted. The specific steps are as follows: S3.1.1: Create keyword-type shot_voice, shot_ocr, shot_person, and shot_things fields and synchronize data from MySQL. S3.1.2: Configure the ES Chinese word segmentation plug-in; S3.1.3: When setting ES to insert data, if the index name does not exist, it will automatically create the index; S3.1.4: Use the ES built-in bool should query scoring method to sort the four fields shot_voice, shot_ocr, shot_person, and shot_things to obtain the semantic similarity ranking and semantic similarity score. .

5. The video shot recommendation method according to claim 4, wherein: The step S3 further includes the following steps: S3.2.1: The frequency of lens usage is calculated by accumulating the number of times the lens is selected. ； After each lens is used, the system will Update to the database; S2.2.2: Store the frequency data in a MySQL database; S2.2.3: Calculate frequency coefficients , 。 6. The video shot recommendation method according to claim 5, wherein: The step S3 further includes the following steps: S2.3.1: Count the video footage time and obtain the freshness data of the video footage. ； in, For the lens time, is the current query time; S2.3.2: Calculate the freshness coefficient. ； in, To keep fresh constant.

7. The video shot recommendation method according to claim 6, wherein: In step S4, the comprehensive score is calculated , ； in, is the semantic similarity score, is the frequency coefficient, It is the freshness coefficient.

8. The video shot recommendation method according to claim 3, wherein: Query the data content by establishing a reverse index.

Images