A method for recommending different visual angles of a video ringback tone

Abstract

The application discloses a video ringback tone different-viewing-angle recommendation method, comprising the following steps: S1, after a calling number dials a called number, call information is triggered to a core network; S2, the core network forwards the call information to a corresponding ringback tone calling node; S3, the ringback tone calling node queries a video ringback tone ID to be played according to the call information; S4, the ringback tone calling node forwards the call information and the corresponding ringback tone ID to a BM; S5, the BM acquires user attributes and historical viewing records, calculates a calling viewing-angle preference, compares each video viewing angle corresponding to the ID, and calculates a sound file information played to the user; S6, the BM returns the sound file information of the preferred viewing angle to the ringback tone calling node; S7, the ringback tone calling node selects the sound file to play; and S8, the ringback tone calling node returns a media stream to the core network and plays the preferred video to the calling user. The application can generate the video ringback tone content to be played according to the calling user attributes and the video viewing records, and intelligently plays and views the recommended viewing angle.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of wireless information communication technology, and particularly relates to a video ringtone different angle recommendation method. BACKGROUND

[0002] With the technological upgrading of 5G+ultra-high definition+MR and cloud interaction, the presentation mode of the video ringtone in the future is no longer a single angle, but is increased by 5G+4K angle, panoramic angle, vertical screen straight shooting angle and the like, which brings more visual splendor to the calling user in the period of waiting for the called user to answer. By using the video watching record of the user in the video ringtone APP, the user's viewing angle preference for different types of videos is analyzed, and a more attractive video ringtone is provided for the calling user. Since the video ringtone has not yet been online with the different angle watching function, there is no complete technical solution for the angle recommendation method for this scene. Most of the existing video shot segmentation algorithms extract the semantic features of each frame in the candidate segment by using a feature extraction network, and then calculate the similarity between the continuous frames by using the semantic features, and then determine the shot boundary position segmentation. Based on the lack of the angle recommendation method, and the weak feature expression ability in the similar background leads to a high missing rate of the shot segmentation algorithm, therefore, a video ringtone recommendation method which can automatically select the playing angle according to the user's preference is to be developed. SUMMARY

[0003] The present application provides a video ringtone different angle recommendation method, which comprises the following steps:

[0004] A video ringtone different angle recommendation method comprises the following steps:

[0005] S1, after the calling number dials the called number, the call information is triggered to the core network;

[0006] S2, the core network forwards the call information to the corresponding ringtone call node;

[0007] S3, the ringtone call node queries the video ringtone ID to be played for the call information;

[0008] S4, the ringtone call node forwards the call information and the corresponding ringtone ID to the BM;

[0009] S5, the BM obtains the user attributes and the historical watching record, calculates the calling angle preference, compares the ID corresponding to each video angle, and calculates the sound file information to the user;

[0010] S6, the BM returns the preferred angle sound file information to the ringtone call node;

[0011] S7, the ringtone call node selects the sound file to play;

[0012] S8, the color ring calling node returns the media stream to the core network, and plays the preference video to the caller.

[0013] As a further illustration of the application, the step S5 comprises the following steps:

[0014] S501, calculating the classification corresponding to the video ring ID to be put and the perspective of each entity file;

[0015] S502, judging whether there is a video playing record in the video ring APP of the caller, if there is, then step S503 is performed, otherwise step S504 is entered;

[0016] S503, calculating the perspective preference of the user for different classification videos according to the video playing record of the caller, and entering step S505 after the calculation is completed.

[0017] S504, under the condition that there is no video playing record of the caller, calculating the perspective preference according to the caller attribute;

[0018] S505, combining the classification of the video ring ID to be put and the perspective of each video file calculated in step S501, recommending the video ring entity playback of the corresponding perspective to the caller according to the perspective preference of the caller for different classification videos calculated in step S503 or step S504.

[0019] Further, the step S503 comprises the following steps:

[0020] S5031, obtaining the video content and viewing period details of the user watching the video in the video ring APP;

[0021] S5032, classifying these video contents according to the video classification system;

[0022] S5033, combining the viewing period details of the user for each type of video to generate a single perspective user watching cut video record set under each type of video of the user;

[0023] S5034, classifying the video in the single perspective user watching cut video record set of the user under each video classification according to the perspective, and distinguishing the long shot and close-up perspective;

[0024] S5035, calculating the perspective preference of the user according to the viewing behavior (viewing details) of the user on each cut single perspective video of each type of video.

[0025] Further, in step S5033, after the video is divided into categories according to the pre-set categories, a deep learning method is used to segment the videos of each category, separate the video clips of different perspectives, and use a feature extraction network for a shot boundary detection framework.

[0026] Further, the algorithm process of the feature extraction network comprises: inputting a video, obtaining mutation boundary candidate segments and gradual change boundary candidate segments through preprocessing, the segments being extracted to strong expression ability features through CNN, and the features being used for subsequent mutation boundary detection and gradual change boundary detection.

[0027] Further, the preprocessing comprises removing non-shot frames and boundary distinguishing in advance; for a small continuous frame segment, if the first frame and the last frame of the segment have a very high similarity, all the frames in the segment are considered as non-shot frames and no longer participate in the subsequent shot detection process; for boundary distinguishing, the similarity between the first frame of the segment and the frames in the segment is judged to distinguish the mutation boundary and the gradual change boundary.

[0028] Further, the boundary distinguishing step comprises:

[0029] Calculating the brightness distance; the entire video is divided into small segments, each segment having a length of N frames, and the brightness distance D between the first frame and the last frame of each small segment is calculated.

[0030] Calculating the local threshold value; 10 small segments are divided into a group, each 10 groups forming a unit, and then the features (mean value and standard deviation) of each unit and group are calculated to calculate the local threshold value T of the nth small segment.

[0031] Candidate segment screening; whether the nth small segment may contain a shot boundary is judged by comparing the mean value and variance in the group; if D>T, the nth segment is a candidate segment; if D<T and the brightness distance is much larger than that of other adjacent segments, the nth segment is also considered as a candidate segment.

[0032] Mutation and gradual change shot distinguishing; the brightness distance D1 of the first half of the candidate segment and the brightness distance D2 of the second half of the candidate segment are calculated, if D1 / D2>1.5 and D1 / D>0.7, the first half contains a mutation shot, and the second half contains a mutation shot, and other cases are determined as gradual change boundary candidate segments.

[0033] Further, the network of the feature extraction network has 50 layers, ResNet50 has undergone 4 groups of Block, each group of Block comprising 3, 4, 6 and 3 Bottleneck, and the first one is a Bottleneck with different input and output sizes.

[0034] Further, in the step S5034, the user viewing time is normalized to obtain a user perspective preference score.

[0035] Further, in the step S504, the user perspective recommendation is performed in a machine learning manner.

[0036] The present application has the following beneficial effects:

[0037] The application realizes the function of automatically selecting and playing the view angle according to the user preference by using the user watching record and the label attribute, fills the gap of the different view angle watching function of the video ringtone, confirms the view angle and the playing scheme in the multi-view angle video ringtone playing process, accurately divides the video lens in the same scene, and calculates and learns the view angle preference of the user in different types of video according to the long video history record and the personal characteristics of the user and makes the recommendation; the convolutional neural feature extraction network of the application introduces the channel attention and the feature fusion module. The channel attention can be regarded as a self-attention module, which strengthens the connection of the channel dimension without increasing the calculation overhead, makes the network discard the unimportant features, and improves the calculation speed, and the feature fusion module fuses the feature maps of the last three layers, combines the advantages of the low resolution features and the high resolution features, and makes the output features contain more semantic information. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The flowchart of step S5 of the method of the application is shown in the figure;

[0039] Figure 2 The flow principle diagram of step S503 of the method of the application is shown in the figure;

[0040] Figure 3 The algorithm flowchart of step S5033 of the method of the application is shown in the figure;

[0041] Figure 4 The structural diagram of the feature extraction network of the application is shown in the figure. DETAILED DESCRIPTION

[0042] The embodiments of the application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments.

[0043] In the description of the application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer", "first", "second" and the like indicate the orientation or position or sequence relationship based on the orientation or position or sequence relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0044] The application provides a video ringtone different view angle recommendation method, which comprises the following steps:

[0045] S1, after the calling number dials the called number, the call information triggers the core network;

[0046] S2, the core network forwards the call information to the corresponding ringback tone call node;

[0047] S3, the ringback tone call node queries the call information to play the video ringback tone ID;

[0048] S4, the ringback tone call node forwards the call information and the corresponding ringback tone ID to the BM (called BM real-time decision software);

[0049] S5, the BM obtains the user attributes and the historical viewing record, calculates the calling perspective preference, compares the ID corresponding to each video perspective, and calculates the sound file information to the user;

[0050] S6, the BM returns the preferred perspective sound file information to the ringback tone call node;

[0051] S7, the ringback tone call node selects the sound file to play;

[0052] S8, the ringback tone call node returns the media stream to the core network and plays the preferred video to the caller.

[0053] The video ringback tone different perspective recommendation method can generate the video ringback tone content to be played according to the caller user attributes and the video viewing record, intelligently play the viewing perspective recommendation, can provide the caller with more attractive video ringback tone, solves the problem of how to portrait the perspective preference of new users in various video classifications, and solves the problem of finally recommending the visual ringback tone perspective.

[0054] Specifically, referring to the accompanying drawings Figure 1 When the user sound file information is played in the above step S5, the following steps are implemented:

[0055] S501, the classification corresponding to the video ringback tone ID to be played and the perspective of each entity file are calculated;

[0056] S502, it is judged whether the video playing record exists in the video ringback tone APP of the caller, if yes, step S503 is performed, otherwise, step S504 is entered;

[0057] S503, the perspective preference of the user for different classification video viewing is calculated according to the video playing record of the caller, and step S505 is entered after the calculation is completed. As a feasible implementation mode, referring to the accompanying drawings Figure 2 When the user perspective preference is calculated in the above step S503, the following method steps can be implemented:

[0058] S5031, the video content and viewing period details of the user watching the video in the video ringback tone APP are obtained.

[0059] S5032, classify these video contents according to a video classification system, for example, the classification in the embodiment includes music, basketball, football, animation, concert and party and government, etc.

[0060] S5033, generate a single perspective user watching cut video record set of the user under each type of video in combination with the user watching time period details of each type of video; after the video is divided according to the pre-set category (for example, the category such as music, football, etc.), the video is segmented by using a deep learning method, and the video segments of different perspectives are separated. Generally, a shot only includes one perspective, and the segmentation of the perspective is converted into the segmentation of the shot boundary. The video shot boundary can be divided into: a sudden change boundary and a gradual change boundary. The length of the sudden change boundary is one frame, and the sudden change is abrupt between two frames (between two shots), the characteristic value changes sharply, and the sudden change boundary detection is relatively simple. The length of the gradual change boundary is several frames to tens of frames. Because the transition is smoother, the inter-frame similarity of the gradual change boundary is high, and the detection difficulty is great.

[0061] The application adopts a new feature extraction network for a shot boundary detection framework, and the algorithm process is shown in the accompanying Figure 3 The principle process is as follows: input a video, obtain sudden change boundary candidate segments and gradual change boundary candidate segments through preprocessing, the segments are extracted to strong expression ability features by using CNN, and then the features are used for subsequent sudden change boundary detection and gradual change boundary detection.

[0062] Specifically, for the preprocessing, in order to improve the rate of shot detection, the application removes a large number of non-shot frames in advance before video processing. For a small number of continuous frames, if the first frame and the last frame have a very high similarity, it is considered that all the frames in the segment are non-shot frames and do not participate in the subsequent shot detection process. As for the boundary division, the similarity between the first frame and the frame in the segment is judged in the specific embodiment of the application to realize the division of the sudden change boundary and the gradual change boundary, and the specific steps are as follows:

[0063] Calculate the brightness distance. The whole video is divided into several small segments, each segment has a length of N frames, and the brightness distance D between the first frame and the last frame of each small segment is calculated.

[0064] Calculate the local threshold value. 10 small segments are divided into 1 group, and each 10 groups form a unit. Then the features (the mean and standard deviation of D) in each unit and group are counted to calculate the local threshold value T of the nth small segment.

[0065] Candidate segment screening. Whether the nth small segment may include a shot boundary is judged by comparing the mean and variance in the group. If D>T, the nth segment is a candidate segment. If D

[0066] Mutant and gradual lens distinction. The luminance distance D1 of the first half of the candidate segment and the luminance distance D2 of the second half are calculated, if D1 / D2>1.5, and D1 / D>0.7, the first half contains a mutant lens, and the second half contains a mutant lens, and other cases are determined as gradual boundary candidate segments.

[0067] Specifically, for feature extraction, the feature extraction network of the application is based on ResNet50, the last three layers of features are fused to strengthen the semantic expression of the features, and channel attention is embedded to correct the channel features by modeling the relationship between channels and improve the representation ability of the neural network. The network structure is shown in the accompanying Figure 4 The network has 50 layers, and ResNet50 has gone through 4 groups of Block, each group of Block contains 3, 4, 6, and 3 Bottleneck, and the first one is a Bottleneck with different input and output sizes. Because the convolutional neural network needs to convert the image into a very small but deep feature map little by little, its main purpose is to change the dimension of the feature vector. There are three convolutions in each Bottleneck, which are 11, 33, and 11. The input of the network is (224, 224), and after 77 convolutions, 33 maximum pooling, and twice downsampling, the output is one-fourth of the original, and then enters the first Block. Channel attention is added in the last Bottleneck. First, the feature map size is compressed to 11C by using global pooling, and then input into the fully connected layer to reduce the dimension to 11C / r to capture the relationship between channels. Then, respectively pass through the ReLu activation layer, the fully connected layer and the Sigmoid activation layer to restore the dimension. The obtained value can be regarded as the weight of the corresponding channel, and finally, the feature map is multiplied by the corresponding weight, and the useless feature tends to 0. The same operation is adopted in the following Block, and the output feature map of the last Bottleneck is upsampled and concatenated with the output of Block3 in the channel dimension, and then 33 and 11 convolutions are performed again. The output is concatenated with the output of Block2 after upsampling, and so on, to obtain a feature map with a size of 2828. Finally, the obtained feature map is mapped into a feature vector by a fully connected layer, and a detection algorithm is used for detection.

[0068] For the above edge detection, the boundary mentioned in the application can be divided into two types: mutation and gradual change. For mutant lens boundary detection, a mutant boundary candidate segment with a length of N is given , , which represents the depth feature vector corresponding to each frame. We use the modified cosine distance to measure the inter-frame similarity: . , which represents the average similarity in the candidate segment.

[0069]

[0070] where t= 0,…,N-2, k is a parameter between 0 and 1, is a preset threshold.

[0071] For the candidate segment, if conditions (1), (2) and (3) are met in turn, the t+1 frame is a mutation boundary. If condition (1) is not met, the segment is discarded because it is impossible to contain a mutation boundary. Conditions (2) and (3) are important features reflecting the mutation boundary, that is, the similarity between the mutation boundary and the previous frame is extremely low, and the similarity between other continuous frames in the two segments is very high. If the two conditions cannot be met at the same time, it means that the excessive change is not obvious enough, and the segment may be part of a gradual transition boundary. The segment needs to be extended (5 frames are added to the beginning and end of the segment) and then gradual transition boundary detection is performed.

[0072] Considering that the gradual transition boundary is basically the superposition of the contents of the two shots before and after the boundary transition, the first half of the transition boundary has a high similarity with the content of the shot before the transition and a low similarity with the content of the shot after the transition, and vice versa. We use the absolute distance difference: to represent the possibility that the frames in the gradual transition boundary belong to the shots before and after the transition. Wherein represents the last frame before the boundary transition and the first frame after the transition. The last frame before the transition and the first frame after the transition of an ideal gradual transition boundary should belong to different shots, and thus the similarity between the two frames should be very low.

[0073]

[0074] Similar to the mutation shot, if the gradual transition boundary candidate segment meets conditions (4), (5) and (6) at the same time, it is considered to be a gradual transition shot boundary.

[0075] After the above three steps of preprocessing, feature extraction and boundary detection, the video shots can be well cut. Compared with the existing video shot cutting method, the new method proposed in the patent cuts the boundaries of the shots more completely and cleanly.

[0076] S5034, classifying the perspective of the user in each video classification under a single perspective of the user watching the cut video record set, and distinguishing between long shot and close-up perspective; for example, the statistical perspective data of the application is as follows:

[0077] The user watching time is normalized:

[0078]

[0079] The score of each type of perspective preference is calculated as:

[0080] Near view preference score = (N2 / M2+ N4 / M4+…) / number of near view video watching

[0081] Far view preference score = (N1 / M1+ N3 / M3+…) / number of far view video watching

[0082] Determine the size of the near view preference score / far view preference score of the user, and the one with more score is the user's view preference.

[0083] S5035, calculate the user's view preference according to the user's watching behavior (watching details) on each type of video in each cutting single view video.

[0084] S504, in the case where the caller does not exist video playing record, the view preference is calculated according to the caller attribute; in step S503, a large number of user's video preference is accumulated, that is, the system learns the view preference of different users for different types of video, and the different view video ringtone recommendation can be played for the user without playing record (age, gender, hobby, region, community, etc.) according to the user attribute classification, and the user view recommendation is carried out through machine learning.

[0085] Extract the age, gender and user visual preference attribute in the user portrait, and construct a label matrix, wherein the gender label is represented by table 1, the age label is represented by table 2, and the user preference is represented by table 3:

[0086] The age label is calculated by extracting the birth year field filled in the user registration information, and the label system is labeled with corresponding label according to age.

[0087]

[0088] The user's hobby can also be labeled in the same way, and the user's hobby can be roughly divided into several categories and labeled.

[0089]

[0090] Wherein n is determined according to the last user's hobby category.

[0091] The above three labels are constructed into a multi-dimensional matrix-user-label matrix, which can be simply represented as:

[0092]

[0093] If a new user needs to be recommended visual ringtone at this time, the label value of the new user can be added to the above table, and then the similarity between this row and each row is calculated, and finally a series of similarity values are obtained, and the most similar user preference visual is recommended to the user.

[0094] The similarity calculation method usually adopts Euclidean distance, Manhattan distance and cosine similarity, etc. The similarity calculation between user tags adopts Euclidean distance calculation.

[0095] The calculation is as follows:

[0096] wherein represents the similarity between the new user u and the q user, q=(1, 2, 3, …, N) represents the user in the user cluster, the label similarity of the new user and other users is calculated and composed into a one-dimensional matrix, then the matrix is sorted in ascending order, the user with the closest similarity and the assigned video perspective preference are assigned to the user.

[0097] S505, according to the calling for different classification video perspective preference calculated in step S503 or step S504 combined with the classification of the video ringback tone ID calculated in step S501, the perspective of each video file is recommended to the caller.

[0098] The above only describes the preferred embodiments of the present application, but cannot be understood as the limitation of the claims. The present application is not limited to the above embodiments, and the specific structure allows changes. In general, all kinds of changes made within the protection scope of the independent claims of the present application are within the protection scope of the present application.

Claims

1. A method for recommending video ringback tones from different perspectives, characterized in that, Includes the following steps: S1, after the calling number dials the called party, the call information is triggered to the core network; S2, the core network forwards the call information to the corresponding ringback tone call node; S3, the ringback tone calling node queries the video ringtone ID that should be played in the call information; S4, the ringback tone calling node forwards the call information and the corresponding ringtone ID to the called party's real-time decision-making software; S5, the called party real-time decision-making software obtains user attributes and historical viewing records, calculates the caller's perspective preference, compares the ID with each video perspective, and calculates the audio file information to be played to the user. S6, the called party real-time decision-making software returns the preferred perspective audio file information to the ringback tone calling node; S7, the ringback tone calling node selects a playback file for playback; S8, the ringback tone calling node returns a media stream to the core network and plays the preferred video to the caller; Step S5 includes the following steps: S501, Calculate the category and perspective of each entity file corresponding to the ringback tone ID of the video to be played; S502. Determine whether there is a video playback record in the video ringback tone APP of the caller. If there is, proceed to step S503; otherwise, proceed to step S504. S503. Calculate the user's viewing perspective preference for different categories of videos based on the caller's video playback history. After the calculation is completed, proceed to step S505. S504. In the case that the caller does not have a video playback record, calculate the perspective preference based on the caller's attributes; S505. Based on the caller's preference for different video viewpoints calculated in step S503 or step S504, combined with the classification of the video ringback ID to be played calculated in step S501 and the viewpoint of each video file, recommend the corresponding viewpoint video ringback entity to be played to the caller. Step S503 includes the following steps: S5031. Obtain details of the video content and viewing time period of the video watched by the user in the video ringback tone APP; S5032. Classify these video contents according to the video classification system; S5033. Combine user viewing time details for various types of videos to generate a single-perspective user viewing segment video record set for each type of video; S5034. Classify the perspectives of users watching video clips from a single viewpoint under each video category, and distinguish between long-range and close-range perspectives. S5035. Calculate user perspective preference based on user viewing behavior in various types of videos and in each segmented single-view video. In step S5033, after the video is divided into categories according to a pre-defined set of categories, a deep learning method is used to segment the video of each category, separating video segments from different perspectives, and a feature extraction network is used for the lens boundary detection framework. The algorithm flow of the feature extraction network includes: inputting a video segment, obtaining candidate segments for mutation boundaries and candidate segments for gradual boundary through preprocessing, these segments will be processed by the feature extraction network to extract features with strong expressive power, and then these features will be used for subsequent mutation boundary detection and gradual boundary detection; The preprocessing includes removing non-shot frames and boundary discrimination; for a short segment of consecutive frames, if the frames at the beginning and end of the segment have extremely high similarity, all frames in the segment are considered non-shot frames and will no longer participate in the subsequent shot detection process; for boundary discrimination, by judging the similarity between the first frame and the frames in the segment, the mutation boundary and the gradual change boundary are distinguished. The steps of the boundary discrimination include: Calculating the brightness distance; the entire video is divided into several small segments, each segment having a length of N frames, and the brightness distance D between the first and last frames of each small segment is calculated. Calculating the local threshold; 10 small segments are divided into 1 group, and every 10 groups form a unit. Then, the features within each unit and group are statistically calculated to obtain the local threshold T of the nth small segment. Candidate segment screening; by comparing the mean and variance within the group, it is judged whether the nth small segment may contain a shot boundary; if D>T, then the nth segment is a candidate segment; if D<T and the brightness distance is much larger than that of other adjacent segments, it is also considered a candidate segment. Distinguishing mutation and gradual change shots; calculating the brightness distance D1 of the first half of the candidate segment and the brightness distance D2 of the second half. If D1 / D2>1.5 and D1 / D>0.7, then the first half contains a mutation shot, otherwise the second half contains a mutation shot. Other cases are judged as candidate segments for gradual change boundaries.

2. The method for recommending video ringback tones from different perspectives according to claim 1, characterized in that: The network of the feature extraction network has a total of 50 layers.

3. The method for recommending video ringback tones from different perspectives according to claim 1, characterized in that: In step S5034, the user viewing duration is normalized to obtain the user's various perspective preference scores.

4. The method for recommending video ringback tones from different perspectives according to claim 1, characterized in that: In step S504, user perspective recommendation is performed through machine learning.

Images