Method, electronic device and server for recommending video clips

Abstract

The present application provides a method, electronic device and server for recommending video clips. The electronic device can be a mobile phone, a smart screen, etc. The server can perform target detection, content recognition, content understanding, natural language understanding and other processing on each frame of the video, determine the label of each frame, aggregate the video into multiple clips, extract and store the label information of each video and the label information of each clip; secondly, the method can identify the user identity, and / or identify the current scene, and combine the label of the video and the label of each clip to recommend videos that match the user identity and the current scene to the user; in addition, when the user selects and plays a video, the method can also query the label information of each video clip included in the video, and match the user with clips that meet the user's viewing needs according to the current user identity, so as to finely control the playback effect of the video and improve the user experience.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a method for recommending video clips, an electronic device, and a server. Background Art

[0002] With the rapid development of Internet products and the emergence of massive videos, different users may have different viewing needs. In the process of users using video software, different types of videos are generally recommended to users. Different user groups may have different viewing needs. For example, different users expect video software to recommend videos of different types and contents. For example, parents generally expect to distinguish between videos recommended for minors and videos recommended for adults. For minors, it is necessary to more strictly control the video level and more finely control the type and content of recommended videos to avoid recommending videos related to violent themes, bloody themes or sexual themes to minors.

[0003] In one possible implementation, users can set the working mode of the electronic device to match the usage needs of different users. For example, when a child user is using a tablet to watch a video, the parent can set the tablet to child mode. When the child user finishes using the tablet, the parent can switch the tablet to normal mode to avoid recommending videos that are not suitable for children, such as violent or bloody themes, to the child user.

[0004] In this implementation, the children's mode cannot guarantee that the recommended videos meet the viewing needs of children, and its effectiveness is low. Secondly, for an electronic device, during the use of the electronic device, the electronic device can generally only be set to one mode. If the user wants to switch the working mode of the electronic device, a series of operations are required to achieve the switch. The switching process is cumbersome and the user experience is poor.

[0005] In addition, users may have different viewing needs in different scenarios. For example, users may have different viewing demands in different scenarios such as home and work. How to recommend videos that meet users' current viewing needs for different user groups and different scenarios, and improve users' viewing experience, is an urgent problem to be solved. Summary of the invention

[0006] The present application provides a method, electronic device and server for recommending video clips. The method can recommend videos that match the user's identity and the current scene to the user. In addition, when the user selects and plays a video, the method can also match the user with clips that meet the user's viewing needs based on the current user identity, thereby finely controlling the video playback effect and improving the user experience.

[0007] In a first aspect, a method for recommending video segments is provided, the method comprising: an electronic device displays a video list, the video list comprising one or more videos; the electronic device receives an operation by a user to play a first video, and in response to the operation, the electronic device sends a request to play the first video to a server, the request to play the first video comprising first identification information, the first video being any one of the one or more videos; the electronic device receives a response message sent by the server, the response message being used to indicate a target segment and a filtered segment of the first video, the target segment being associated with the first identification information, the target segment being a segment for playing to the user, and the filtered segment being a segment not for playing to the user; the electronic device plays the target segment of the first video according to the response message.

[0008] It should be understood that the "play request for the first video" can be used to request the address of the playback content (content Identification, content ID) of the first video. The video application of the electronic device can obtain the image data, audio data, etc. corresponding to the first video from the address of the playback content, thereby realizing the normal playback of the first video. It will not be repeated here.

[0009] It should also be understood that in the embodiments of the present application, "matching" can be understood as "association", that is, the category to which the label information of the target segment belongs is associated with the identity information of the current user, the current scene information, etc.

[0010] In combination with the first aspect, in some implementations of the first aspect, the first identification information includes identity information of the user and / or scene information where the electronic device is currently located.

[0011] Through the above method, when the user requests to play the video through the video application, the electronic device can collect user characteristics, accurately determine the user identity and create a user portrait based on the collected user characteristics, and then combine the label information of each video clip and the user's preferences, behavioral habits, etc. to provide the current user with refined personalized videos, or match the target clip in a video for the current user.

[0012] For example, if the current user is a child user or a minor user, the method can control the electronic device to automatically switch to the child mode, and control the permission range of the video that can be played by the electronic device in the child mode, and can also block the video clips in a video that are not suitable for children to watch (filter the clips). This process does not require manual setting of the electronic device to switch to the child mode, and can achieve the effect of finely controlling the video playback in the child mode, thereby improving the user experience.

[0013] Alternatively, the above method can also be combined with the life scene in which the electronic device is located, and according to the scene recognition results, different video content can be recommended to the user from the video content library. For example, for the kitchen scene, videos related to food and food making can be recommended to the user; for the study scene, videos related to teaching can be recommended to the user; for the living room scene, movies and TV series suitable for family members to watch together can be recommended to the user; for the balcony scene, videos related to home and cleaning can be recommended to the user. This method can recommend videos that match the current scene for the user to choose from, thereby improving the user experience.

[0014] In combination with the first aspect and the above-mentioned implementations, in some implementations of the first aspect, the response message also includes information of a video list, the information of the video list includes information of at least one second video, and the second video matches the first identification information. The method also includes: the electronic device updates the video list according to the information of the video list and displays the at least one second video in the video list.

[0015] During the above process, the electronic device can update the videos displayed in the video recommendation menu of the electronic device according to the information of the video list sent by the server, so that the videos displayed in the video recommendation menu are videos that match the current user identity and scene for the user to choose, thereby improving the user's viewing experience.

[0016] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the playback request of the first video also includes the account information of the electronic device logged in, and the tag information of the second video is a video among one or more videos stored in the server that matches the historical playback record corresponding to the identity of the user and the account of the electronic device logged in.

[0017] Through the above method, the user's historical playback records and other information can be obtained according to the user's account information, and videos can be recommended to the user or the video playback process can be controlled. This method can accurately recommend video content that suits the current scene, user habits and hobbies to the user, and can recommend videos or video clips that better match the user's viewing needs, thereby improving the user's viewing experience.

[0018] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, when the first identification information includes the identity information of the user, before the electronic device receives the response message sent by the server, the method also includes: the electronic device collects user characteristics and determines the identity of the user based on the user characteristics; or the electronic device collects user characteristics, the electronic device sends the user characteristics to the server, and receives the recognition results of the user characteristics sent by the server to determine the identity of the user; wherein the user characteristics include one or more of facial features, age characteristics, height characteristics, clothing characteristics, and occupational characteristics.

[0019] Optionally, the electronic device can collect user features through one or more collection and detection modules such as a camera, a fingerprint sensor, a touch sensor, an infrared sensor, etc. The camera is not limited to the front camera, the rear camera or the under-screen camera of the electronic device.

[0020] In one possible implementation, as the data processing and computing capabilities of electronic devices are improved, and algorithms such as image detection and recognition are enriched, electronic devices can independently identify user identities, thereby reducing the interaction process between electronic devices and servers and accelerating the rate of user identity identification.

[0021] In another possible implementation, the server can receive user feature data reported by the electronic device, create user portraits based on the massive user feature data, and encrypt and store a large amount of data of different user portraits. After subsequently receiving the user information reported by the electronic device, the server can quickly query the large amount of stored user portrait data and quickly determine the identity of the current user, thereby improving the rate of user identity recognition and reducing the amount of data processing. In addition, the accuracy and robustness of user identity recognition can also be improved.

[0022] Through the above method, if the electronic device recognizes that the current user is a different user identity such as the owner user, parent user, child user, etc., it can further recommend videos according to the current user identity. For example, educational videos can be recommended for parent users; for the owner user, the video with the same type or similar subject content as the owner user's historical playback record can be matched in combination with the owner user's historical playback record; for the child user, children's animation and learning videos can be recommended, etc., which are not limited in the present embodiment of the application.

[0023] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, it includes: the electronic device periodically collects user characteristics; and / or the electronic device detects the user's operation of running a video application, and collects user characteristics in response to the operation.

[0024] In one possible implementation, when a user turns on the personalized recommendation of video clips function of the electronic device's video application and turns on the "allow intelligent detection of identity information" function, the electronic device can trigger the collection and detection module of the electronic device to collect user information when the user clicks the icon of the video application (i.e., opens the video application) and enters the running interface of the video application.

[0025] Alternatively, the collection and detection module of the electronic device can periodically collect user information according to a certain period (for example, 10 times per minute). For example, the electronic device can periodically collect the current user information during the user's use of the video application to determine the identity of the current user. The embodiment of the present application does not limit the timing of the electronic device collecting user information.

[0026] Alternatively, the collection and detection module of the electronic device may not collect user information, and the user may manually set the current user identity of the electronic device.

[0027] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, after the electronic device determines the identity of the user, the method also includes: the electronic device displays a first window, the first window displays first prompt information, and the first prompt information is used to prompt the user that the electronic device can recommend videos to the user based on the identity of the user.

[0028] For example, when the current user is identified as a child user (or underage user), the mobile phone can automatically switch to the child mode, display the first window and display a prompt message in the first window: Dear user, you are identified as an underage user, and some video clips will be blocked. The user can choose whether to receive the identification result of the user identity according to their needs.

[0029] Through the above method, the electronic device or video application can accurately determine the identity or age information of the current user, for example, the current user is: a child user or a minor user; a parent user or an adult user; an elderly user, etc., and in the subsequent process of the user requesting to play the video, provide different services to meet the usage needs of different users.

[0030] In combination with the first aspect and the above-mentioned implementation manner, in some implementation manners of the first aspect, the electronic device plays the target segment of the first video according to the response message, including: displaying a playback progress bar on the playback interface of the first video, the playback progress bar including a first area corresponding to the target segment and a second area corresponding to the filtered segment, the second area of ​​the playback progress bar being displayed in grayscale; or

[0031] A playback progress bar is displayed on the playback interface of the first video, and the playback progress bar only includes a first area corresponding to the target segment, and does not include a second area corresponding to the filtered segment.

[0032] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, when the playback progress bar includes the first area and the second area, during the process of playing the target segment of the first video, when the current playback time displayed on the playback progress bar is located at the starting position of the second area, the electronic device displays a second window, and the second window displays second prompt information, and the second prompt information is used to prompt the user that the electronic device will skip the filtered segment and continue to play the target segment of the first video.

[0033] In combination with the first aspect and the above-mentioned implementation manners, in some implementation manners of the first aspect, when the first identification information includes scene information in which the electronic device is currently located, before the electronic device receives a response message sent by the server, the method further includes: the electronic device collects current scene features and determines the current scene information based on the scene features; or the electronic device collects the current scene features, the electronic device sends the current scene features to the server, and receives the recognition results of the scene features sent by the server to determine the current scene information.

[0034] Optionally, the above process may have different occurrence times and implementation methods. For example, for household devices with relatively fixed locations such as smart screens, the smart screen can be triggered to start collecting information about life scenes when the user uses and turns on the smart screen for the first time, and the scene in which the smart screen is located may not change for a long time. Alternatively, the collection and detection module of the electronic device can periodically collect life scene information according to a certain period, which is not limited in the embodiments of the present application.

[0035] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the playback request of the first video also includes the account information of the electronic device logged in, and the target segment is a segment in the first video that matches the historical playback record corresponding to the first identification information and the account of the electronic device logged in.

[0036] The above process can combine the current life scene of the electronic device, the identity of the current user, the account of the user logged in by the electronic device, the historical browsing records corresponding to the user account, and other information, and combine the tag information of each video from the video content library to recommend one or more videos to the user. This method can accurately recommend video content that matches the current scene, user habits and hobbies to the user, thereby improving the user's viewing experience.

[0037] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the response message also includes metadata of the target segment, and the metadata of the target segment includes one or more of the playback address, image data, audio data and text data of the target segment.

[0038] In summary, in the process of users using video applications, electronic devices can collect and identify the identity of users. In the process of the electronic device requesting the server to obtain the first video, the user's identity information can be reported, and the server can query the tag information of each video clip included in the video, and match the user with clips that meet the user's viewing needs based on the current user identity. For example, when it is identified that the current user is a child user or a minor user, the method can also control the electronic device to switch to child mode, and control the range of movie permissions that can be played by the electronic device in child mode. If the child user or minor user plays a selected video, the video provided to the electronic device by the server can also be a filtered video, that is, the video clip content in the video that is not suitable for child users to watch is blocked. This process does not require manual setting of the electronic device to switch to child mode, and can achieve fine control of the video playback effect in child mode, thereby improving the user experience.

[0039] In a second aspect, a method for recommending video segments is provided, which is applied to a server, wherein the server stores one or more videos and tag information of each of the one or more videos, and the method comprises: the server receives a play request for a first video sent by an electronic device, the play request for the first video comprises first identification information, the first video is any one of the one or more videos, and the first video comprises one or more segments; the server queries the tag information of the first video according to the play request for the first video, the tag information of the first video comprises tag information of each of the one or more segments; the server determines a target segment and a filtered segment of the first video according to the first identification information and the tag information of each of the one or more segments, the target segment being associated with the first identification information, the target segment being a segment for playing to the user, and the filtered segment being a segment not for playing to the user; the server sends a response message to the electronic device, wherein the response message is used to indicate the target segment and the filtered segment.

[0040] In combination with the second aspect, in some implementations of the second aspect, the first identification information includes identity information of the user and / or scene information in which the electronic device is currently located.

[0041] Through the above method, the server can receive user identity information and / or current scene information sent by the electronic device. After determining the user identity and / or the current scene, the server can combine the tag information of the video in the server's video content library to recommend videos that match the user identity and / or the current scene from the video content library.

[0042] Exemplarily, if the server determines that the current user is a child user, it can query the tag information of each video segment included in the first video, and then determine the tag that matches the child user, and then determine the target segment and the corresponding playback start time and end time of the target segment based on the association between the tag information stored for each video segment and the playback progress of each video segment, that is, the target segment that matches the child user and can be played for the child user from the multiple video segments included in the first video.

[0043] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, when the first identification information includes the identity information of the user, the method further includes: the server receives user characteristics sent by the electronic device, identifies the identity of the user based on the user characteristics, and sends the identification result of the user identity to the electronic device, the user characteristics include one or more of facial features, age characteristics, height characteristics, clothing characteristics, and occupational characteristics; and / or when the first identification information includes scene information in which the electronic device is currently located, the method further includes: the server receives scene characteristics sent by the electronic device, identifies the current scene based on the scene characteristics, and sends the scene recognition result to the electronic device.

[0044] Through the above method, the server can receive the user feature data reported by the electronic device, and combine the stored massive user feature data to create a user profile for the current user, quickly determine the identity of the current user, improve the speed of user identity recognition, and improve the accuracy of user identity recognition. After determining the user's identity, the server can combine the tag information of the video in the video content library of the server to recommend videos that match the user's identity.

[0045] Alternatively, the server can identify the life scene in which the electronic device is located, and recommend different video content to the user from the video content library based on the scene recognition results. This method can be used for scenarios where multiple people in the family use multiple large-screen devices such as smart screens, or when multiple family members use the same smart screen or other device, or when different users use large-screen devices such as smart screens in split screens. The process can intelligently identify the current life scene in which the electronic device is located, the identity of the current user, the account of the user logged in to the electronic device, the historical browsing records corresponding to the user account, and other information, and recommend one or more videos to the user from the video content library in combination with the tag information of each video. This method can accurately recommend video content that matches the current scene, user habits, and hobbies to users, thereby improving the user's viewing experience.

[0046] In combination with the second aspect and the above-mentioned implementation manner, in some implementation manners of the second aspect, the method also includes: the server determines, from the one or more videos, a video list matching the first identification information based on the first identification information and the tag information of each video in the one or more videos, the video list including at least one second video; the server sends information of the video list to the electronic device, the information of the video list including one or more of the playback address, image data, audio data and text data of the at least one second video.

[0047] In combination with the second aspect and the above-mentioned implementation manner, in some implementation manners of the second aspect, the method also includes: the server obtains the first video and determines the multiple frames included in the first video; the server detects each frame of the multiple frames included in the first video and identifies the content included in each frame; the server determines the tag information of each frame of the multiple frames based on the content included in each frame; the server divides the first video into one or more segments based on the tag information of each frame; the server determines the tag information of each of the one or more segments based on the tag information of each frame, and determines the tag information of the first video.

[0048] It should be understood that the operator of the video application can upload one or more videos to the server corresponding to the video application, and the server will perform intelligent frame-by-frame analysis on each video.

[0049] Optionally, the server may perform intelligent frame-by-frame analysis on the first video based on a variety of media artificial intelligence (AI) algorithms, deep learning algorithms, etc., to generate metadata of the first video and tag information of the first video.

[0050] In one possible implementation, after the server analyzes the first video frame by frame to obtain the label information of each frame, it can aggregate the labels of the video segments according to the label similarity between adjacent frames, and then divide the first video into video segments, for example, dividing the first video into multiple video segments, and each of the multiple video segments can correspond to different label information.

[0051] Optionally, each frame may include multiple tags, and the tag information of the video segment may be determined according to any tag of each frame among the multiple frames included in the video segment.

[0052] Through the above method, after the operator uploads the video file to the server, the frame-by-frame detection and intelligent analysis process of the video can be triggered based on AI algorithms, etc., to obtain the multiple fragmented video clips included in the video and the label information of each video clip. In the process of dividing the video into multiple fragmented video clips and determining the label information of each video clip, the dimension of label extraction can realize the intelligent classification and extraction of video labels based on the behavioral characteristics of the user input. On the one hand, this process avoids the problems of filming efficiency, label accuracy, and label effectiveness caused by manual operation; on the other hand, the label information of each video clip detected by the server can be further determined or adjusted by the operator, and the adjustment result can update or correct the existing frame-by-frame detection algorithm. This process can form an internal loop link in the form of "label detection and extraction-intelligent optimization of label detection algorithm-video clip push with different labels-user experience effect" in the full process, thereby improving the accuracy of generating label information.

[0053] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the server divides the first video into one or more segments according to the tag information of each frame, including: the server divides the first video into the one or more segments according to the similarity between the tag information of adjacent frames.

[0054] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the server determines the tag information of each of the one or more fragments, and determines the tag information of the first video based on the tag information of each frame, including: the server determines the tag information that appears most repeatedly in the multiple frames included in each fragment as the tag information of the fragment based on the tag information of each frame; and the server determines the tag information that appears most repeatedly in the tag information of the one or more fragments as the tag information of the first video based on the tag information of each fragment.

[0055] Optionally, the server may determine the tag with the most repetitive appearances in the multiple frames included in the video segment as the tag information of the video segment. Each video segment may have multiple tags.

[0056] In one possible implementation, in the process of aggregating the first video into video segments according to the label similarity between adjacent frames, there may be a few frames between multiple consecutive frames of the first video whose labels are different from those of adjacent consecutive frames. In this scenario, a noise threshold can be set. The noise threshold can be used to mark the critical number M of frames with discontinuous labels, where M is greater than or equal to 1.

[0057] By setting the noise threshold, when a judgment error occurs in the label information of a certain frame during the detection process of one or more dimensions such as target detection and content recognition, the noise threshold can ignore the impact of the error on the aggregation of video segments, thereby improving the accuracy of the judgment of the label information of the video segment and the accuracy of dividing the video segments.

[0058] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the playback request of the first video also includes the account information of the electronic device logged in, and the server determines the target segment and the filter segment of the first video based on the first identification information and the tag information of each of the one or more segments, including: the server obtains the historical playback record of the electronic device based on the account information of the electronic device logged in; the server determines the target segment based on the first identification information, the historical playback record of the electronic device and the tag information of each of the one or more segments.

[0059] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the response message also includes metadata of the target segment, and the metadata of the target segment includes one or more of the playback address, image data, audio data and text data of the target segment.

[0060] In a third aspect, an electronic device is provided, comprising: a display screen; one or more processors; one or more memories; a module with multiple applications installed; the memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes a method as described in any one of the first aspect and the implementation method of the first aspect.

[0061] In a fourth aspect, a graphical user interface system on an electronic device is provided, wherein the electronic device has a display screen, one or more memories, and one or more processors, wherein the one or more processors are used to execute one or more computer programs stored in the one or more memories, and the graphical user interface system includes a graphical user interface displayed when the electronic device executes the method as described in any one of the first aspect and the first aspect, and the method as described in any one of the second aspect and the second aspect.

[0062] In a fifth aspect, a server is provided, comprising: one or more processors; one or more memories; the memories storing one or more programs, and when the one or more programs are executed by the processors, the server executes a method as described in any one of the second aspect and the implementation manner of the second aspect.

[0063] In a sixth aspect, a system is provided, comprising an electronic device and a server, wherein the electronic device is capable of executing a method as described in any one of the first aspect and the implementation manner of the first aspect, and the server is capable of executing a method as described in any one of the second aspect and the implementation manner of the second aspect.

[0064] In the seventh aspect, a device is provided, which is included in an electronic device or a server, and the device has the function of implementing the electronic device behavior in the method described in any one of the first aspect and the implementation of the first aspect, and the function of the server behavior in the method described in any one of the second aspect and the implementation of the second aspect. The function can be implemented by hardware, or by hardware executing corresponding software implementation. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a display module or unit, a detection module or unit, a processing module or unit, etc.

[0065] In an eighth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method as described in any one of the first aspect and the first aspect, and the method as described in any one of the second aspect and the second aspect.

[0066] In the ninth aspect, a computer program product is provided. When the computer program product runs on an electronic device, the electronic device executes any possible method of the first aspect and the implementation method of the first aspect, as well as any method of the second aspect and the implementation method of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a user interface diagram of a process in which a user watches a video through a video application installed on a mobile phone.

[0068] Figure 2 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0069] Figure 3 It is a software structure block diagram of the electronic device according to an embodiment of the present application.

[0070] Figure 4 This is a schematic diagram of an interface for a user to enable a personalized recommendation of video clips, as provided in an embodiment of the present application.

[0071] Figure 5 This is a schematic flowchart of a method for personalized recommendation of video clips provided in an embodiment of the present application.

[0072] Figure 6 This is a schematic diagram of the results of frame-by-frame analysis of a video provided in an embodiment of the present application.

[0073] Figure 7 This is a schematic diagram of an interface for recommending video clips to users on a mobile phone, provided in an embodiment of the present application.

[0074] Figure 8 This is another schematic diagram of a process for recommending video clips to a user on a mobile phone provided in an embodiment of the present application.

[0075] Fig. 9 This is a schematic flowchart of another method for personalized recommendation of video clips provided in an embodiment of the present application.

[0076] Fig.10 This is another example of an interface diagram for recommending video clips to users on a smart screen provided in an embodiment of the present application.

[0077] Fig.11 This is another example of an interface diagram for recommending video clips to users on a smart screen provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] Figure 1 The figure is a graphical user interface (GUI) diagram of a user watching a video through a video application installed on a mobile phone.

[0079] For example, Figure 1 Figure (a) shows the interface 101 currently output by the mobile phone in unlock mode, and the interface 101 displays the weather clock component and multiple applications (applications, Apps) and other contents. Among them, the application may include a browser, phone, video, settings, etc. It should be understood that the interface 101 may also include other more applications, which is not limited in the embodiment of the present application.

[0080] like Figure 1 As shown in Figure (a), the user clicks the icon of the video application. In response to the user's click operation, the mobile phone runs the video application and displays the following Figure 1 FIG. (b) shows the main interface 102 of the video application. The main interface 102 of the video application can display different functional areas and menus, such as a video search box, a setting control 10, and multiple videos recommended to the user.

[0081] Optionally, under different menu categories such as “daily recommendation”, “TV series”, and “movies”, the interface 102 may display different types and numbers of video clips recommended for the user. Figure 1 As shown in Figure (b), the interface 102 is the interface corresponding to the "Daily Recommendation" menu, and the interface 102 displays multiple different videos currently recommended to the user, such as video 1, video 2, video 3, video 4, video 5, etc.

[0082] Users can click on the icon of a video they want to play according to their preferences and viewing needs, triggering the phone to start playing the video. Figure 1 As shown in Figure (b) of the figure, the user clicks the icon of "Video 2". In response to the user's click operation, the mobile phone displays the following Figure 1 The interface 103 shown in FIG. (c) starts playing the video 2. In addition, the video application can continue to play the 7th episode of the video 2 according to the user's historical viewing record, which will not be described in detail here.

[0083] In a possible implementation, the multiple videos recommended by the video application to the user may belong to different categories, or the multiple videos may correspond to different themes. Figure 1 In the interface 102 shown in FIG. 8 (b), video 1 may correspond to a violent theme, video 2 may correspond to a bloody theme, video 3 may correspond to a love theme, video 4 may correspond to a teaching theme, video 5 may correspond to an inspirational theme, and so on.

[0084] Alternatively, each video may include multiple different types of video clips. For example, the 7th episode of the video 2 being played by the user may include different clips such as a violent clip, a bloody clip, a romantic clip, a family clip, and a funny clip.

[0085] With the emergence of massive videos, different users may have different viewing needs. For example, for adults and minors, male users and female users, middle-aged users and elderly users, different user groups expect video applications to recommend videos of different types or different themes. In other words, Figure 1 In the interface 102 shown in FIG. 8 (b), different users may desire to obtain videos of different types or different themes.

[0086] Furthermore, users may have different viewing needs in different scenarios. For example, with the increasing number of electronic devices, users may use different electronic devices such as mobile phones, tablets, personal computers, smart screens, etc. to watch videos in different scenarios, such as home scenes, work scenes, etc. For different scenarios, users' viewing demands may also change, and users' playback demands may also vary during the process of watching videos. For example, users may want to play certain video clips, want to skip certain video clips, etc.

[0087] In view of the above situation, an embodiment of the present application will provide a method for recommending video clips, so as to recommend video clips that meet the current viewing needs of users for different users and different scenes, so as to improve the user's viewing experience.

[0088] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0089] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0090] The method for personalized recommendation of video clips provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of electronic devices.

[0091] For example, Figure 21 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0092] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0093] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0094] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0095] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0096] In the embodiment of the present application, the processor 110 stores a program or instruction corresponding to the method of recommending video clips to users in the embodiment of the present application, which can trigger the electronic device 100 to collect and identify the identity of the user, collect information about the current scene, and upload the collected identity of the user and the current scene to the server. The electronic device 100 can also be controlled to receive information about one or more videos recommended by the server, as well as the clips and tag information included in each video.

[0097] When a user selects and plays a video, the processor 110 can also query the tag information of each video clip included in the video, and match the clip that meets the user's viewing needs according to the current user identity. For example, when it is determined that the current user is a child user or a minor user, the processor 110 can control the electronic device 100 to switch to the child mode and control the range of movie permissions that the electronic device 100 can play in the child mode. If the child user or minor user plays a selected video, the electronic device 100 is controlled to play only video clips suitable for child users.

[0098] In addition, the processor 110 can also determine the identity of the user based on the collected user characteristics, and determine the current scene based on the collected scene information, which will not be repeated here.

[0099] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0100] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0101] The I2S interface can be used for audio communication. The PCM interface can also be used for audio communication to sample, quantize and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering a call via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0102] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0103] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0104] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0105] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0106] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0107] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0108] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0109] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0110] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0111] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0112] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0113] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0114] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0115] In the embodiment of the present application, the electronic device 100 can communicate with the server through the mobile communication module 150 or the wireless communication module 160. For example, the electronic device 100 can communicate with the server through the mobile communication module 150 or the wireless communication module 160. For example, the electronic device 100 can communicate with the server through the Wi-Fi network or through the wireless communication such as 2G / 3G / 4G / 5G, which is not limited in the embodiment of the present application.

[0116] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0117] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLED, MicroLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0118] In the embodiment of the present application, when the electronic device 100 is Figure 1 In the case of the mobile phone shown, the display screen 194 of the mobile phone can receive the user's operation, triggering the mobile phone to start collecting user features, or start collecting current scene information.

[0119] When one or more video information is received from the server, a corresponding video list interface may be displayed on the display screen 194 of the mobile phone.

[0120] When a user clicks to play a video, the playback interface of the video can be displayed to the user on the display screen 194, and when a certain segment of the video is filtered out, some prompt information or prompt windows with different contents can be displayed to prompt the user of the current playback progress.

[0121] In addition, the user can also perform operations such as clicking, double-clicking, and sliding on the display screen 194. The content displayed on the display screen 194 of the electronic device 100 responds to the user's operations, which will not be repeated here.

[0122] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0123] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0124] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0125] In the embodiment of the present application, the camera 193 of the electronic device 100 may be a front camera, a rear camera, or an under-screen camera, etc. The camera 193 may receive instructions issued by the processor 110, collect user features, or collect current scene information, etc., which will not be described in detail here.

[0126] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0127] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0128] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0130] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0131] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0132] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0133] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0134] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0135] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0136] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0137] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0138] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting.

[0139] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0140] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0141] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0142] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0143] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0144] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0145] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0146] In the embodiment of the present application, the fingerprint sensor 180H can obtain the user's fingerprint information and transmit the user's fingerprint information to the processor 110, which compares it with the existing fingerprint information to verify and confirm the identity.

[0147] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0148] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0149] In the embodiment of the present application, the touch sensor 180K can detect the user's touch, click, double-click and other operations, and transmit the user operation to the processor 110, and the processor 110 responds.

[0150] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0151] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0152] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0153] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0154] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0155] It should be understood that the electronic device 100 may include Figure 2 Part or all of the structures described in Figure 2 More or less structures, the embodiment of the present application does not limit the hardware structure of the electronic device 100.

[0156] It should also be understood that the server may also include Figure 2 Part or all of the structures described in Figure 2 More or less structures, the embodiment of the present application does not limit the structure of the server.

[0157] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplified.

[0158] Figure 31 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime ( runtime) and system libraries, as well as the kernel layer and network transport layer. The application layer can include a series of application packages.

[0159] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0160] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0161] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0162] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether the screen has a status bar, or participate in operations such as locking the screen and taking screenshots.

[0163] Content providers are used to store and retrieve data and make it accessible to applications. The stored data may include video data, image data, audio data, etc. It may also include call log data, user browsing history and bookmarks, etc., which will not be described in detail here.

[0164] For example, in the embodiment of the present application, when the electronic device 100 plays a video for the user, the user starts the video application and clicks to play a video. The window manager can determine the size of the video playing window on the display screen according to the size of the display screen of the electronic device 100, for example, playing the video for the user in a full screen or half screen manner.

[0165] At the same time, the electronic device 100 receives the data of the video sent by the server, and the content provider can obtain the data of the video, such as image data, audio data, etc., and draw the video screen according to the image data, audio data, etc., and display the video screen in the playback window determined by the window manager. In addition, the content provider can also obtain the browsing history data, browsing preferences and other data included in the user account, so that after the server obtains the data, it can recommend personalized videos to the user or match video clips according to the data, which will not be repeated here.

[0166] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0167] Exemplarily, the video playback interface displayed on the display screen of the electronic device 100, the play controls, pause controls, next episode controls, setting controls, etc. on the interface can all be provided based on the visual controls of the viewing system.

[0168] The phone manager is used to provide communication functions of the electronic device 100, such as the management of call status (including call connection, hang-up, etc.).

[0169] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0170] The notification manager enables applications to display notification information in the status bar of the screen, which can be used to convey messages to users. The notification information can automatically disappear after a short stay in the status bar, without the need for the user to perform interactive processes such as closing operations. For example, the notification manager can inform users of messages such as download completion. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of an application running in the background; or, the notification manager can also be a notification that appears on the screen in the form of a dialog window, such as a text message prompt in the status bar, etc.; or, the notification manager can also control the electronic device to emit a prompt sound, the electronic device to vibrate, the electronic device's indicator light flashes, etc., which will not be repeated here.

[0171] The runtime includes the core library and the virtual machine. The runtime is responsible for scheduling and management of the Android system.

[0172] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0173] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0174] The system library may include multiple functional modules, such as a surface manager, a media library, a three-dimensional (3D) graphics processing library (such as OpenGL ES), a two-dimensional (2D) graphics engine, etc.

[0175] The surface manager is used to manage the display subsystem of the electronic device and provides the fusion of 2D and 3D layers for multiple applications.

[0176] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0177] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0178] The network transmission layer may include a communication module, and a transmission channel may be established between the communication module of the electronic device 100 and the communication module of the server 200. Exemplarily, the communication module of the electronic device 100 and the communication module of the server 200 may communicate based on a Wi-Fi channel; or, the communication module of the electronic device 100 and the communication module of the server 200 may communicate based on 2G / 3G / 4G / 5G / 6G wireless communication, etc., which is not limited in the embodiments of the present application.

[0179] also, Figure 3 Also shown are software modules that the server 200 may have. For example, Figure 3 As shown, the server 200 may include at least a communication module, a distribution module, a media service module, a content service module, and a user service module.

[0180] Among them, the media service module can analyze each video uploaded by the operator frame by frame, for example, perform one or several dimensions of processing such as target detection, content recognition, content understanding, and natural language understanding on each frame of each video, so as to determine the label information of each video and the label information of each frame. Secondly, the media service module can also perform label aggregation based on the label information of each frame, so as to divide the video into multiple video segments, and each of the multiple video segments can correspond to different label information. In addition, the media service module can also analyze and extract the label information of each video or different segments of each video, and dynamically generate the label information of the video segments.

[0181] The content service module can provide different services to different users based on the metadata of each video segment included in the video, such as recommending a certain video to the user, determining a certain video segment of the video that can be played for the user, and reviewing a certain video segment of the video. Specifically, the content service module can obtain the user identity information sent by the electronic device 100, and request the distribution module to match a target segment that can be played for the user from multiple video segments, etc., which will not be described in detail here.

[0182] The user service module can receive user feature data reported by the electronic device 100, and create a user portrait based on the user feature data, and encrypt and store a large amount of data of different user portraits. After receiving the user information reported by the electronic device 100, the user service module can quickly query the stored large amount of user portrait data to quickly determine the identity of the current user. In addition, the user service module can also receive scene information reported by the electronic device 100. For example, the user service module can receive a life scene picture reported by the electronic device 100, and determine the scene corresponding to the life scene picture based on the picture detection and recognition algorithm.

[0183] The distribution module can obtain and store the tag information of different videos of the media service module or the content service module, or the tag information of different segments of a video. The distribution module can also obtain the recognition result of the user identity from the user service module, or obtain the recognition result of the current scene, and recommend one or more videos that meet the current viewing needs of the user based on the user identity, scene information, etc., as well as the target segments in each video that can be played by the current user, etc., to achieve personalized and intelligent video recommendations.

[0184] The communication module is used to communicate with the electronic device 100 . Please refer to the above-mentioned related description and will not be repeated here.

[0185] It should be noted here that the functions and execution steps of each module can be found in the detailed description of the subsequent embodiments, and will not be introduced one by one this time.

[0186] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 2 Take the electronic device with the structure shown as an example, such as a mobile phone, based on Figure 3 The software architecture shown, combined with the accompanying drawings and application scenarios, specifically describes the method for personalized recommendation of video clips provided in an embodiment of the present application.

[0187] Optionally, the method provided in the embodiment of the present application can be pre-configured in the video application by the developer of the video application. During the process of the user using the video application, the video clips can be recommended to the user by default and automatically according to the method provided in the present application; or, through the setting menu of the video application, a shortcut control or option can be provided to the user, and the user can use the shortcut control or option to enable the function of the video application to personalize the recommendation of video clips for the user, that is, to recommend video clips to the user according to the method provided in the present application; or, the shortcut control or option can be integrated in the menu of a system-level application such as a setting application, and the user can use the shortcut control or option included in the setting application and other system-level applications to enable the function of personalizing the recommendation of video clips for the user, and the embodiment of the present application is not limited to this.

[0188] In one possible manner, the user may enable the personalized recommendation of video clips function of the electronic device through the setting function of the video application.

[0189] It should be understood that the "video application" in the embodiments of the present application can be Huawei Video provided by Huawei, or other third-party video applications such as Youku Video, Tencent Video, iQiyi Video, Mango TV, etc. This method can be applied to any application with a video recommendation function, and the embodiments of the present application are not limited to this.

[0190] Figure 4 : is a schematic diagram of an interface for a user to enable a personalized video clip recommendation function provided in an embodiment of the present application. Figure 4 Figure (a) shows that when the mobile phone is unlocked, the screen of the mobile phone displays the main interface 401 of the mobile phone currently output, such as Figure 4 As shown in FIG. 4(a), the user clicks the video application icon on the main interface 401. In response to the user's click operation, the mobile phone displays the following Figure 4 FIG. 4 is a main interface 402 of the video application as shown in FIG. 4 (b).

[0191] like Figure 4 As shown in FIG. 1 (b), the user clicks the "settings" control 10 on the main interface 402 of the video application. In response to the user's click operation, the mobile phone may display the following Figure 4 The setting interface 403 shown in FIG. (c) of FIG. 403 may include multiple different setting menus and options. For example, Figure 4As shown in Figure (c) in FIG. 403 , the setting interface 403 may include an “Account Settings” menu, an “Appearance Settings” menu, a “Play Settings” menu, a “Download Settings” menu, etc., and an “Other Settings” menu. Among them, the “Account Settings” menu may include options such as Account Security Center, Personal Data, and Regional Settings, the “Appearance Settings” menu may include options such as Dark Mode, and the “Play Settings” menu may include options such as Skip Title and End, Continuous Play, etc., which will not be described in detail in the present embodiment of the application.

[0192] In a possible implementation, a quick control for enabling the personalized recommendation of video clips may be included in an interface corresponding to the "Other Settings" menu. Figure 4 As shown in Figure (c) of FIG. 1 , the user clicks on the “Other Settings” menu. In response to the user’s click operation, the mobile phone further displays the following Figure 4 The interface 404 corresponding to the "Other Settings" menu shown in FIG. (d) in FIG. The interface 404 may include a variety of quick controls (or switches), such as a "allow recommendation based on history" control, a "allow display of advertisements" control, a "allow non-WiFi autoplay" control, a "allow intelligent detection of identity information" control, a "allow scene recognition" control, and a "allow matching of video clips based on user identity" control.

[0193] Optionally, the user can click on some or all of the shortcut controls (or switches) to enable the corresponding functions according to their needs. Figure 4 As shown in Figure (d) in the figure, the user can click the quick control (or switch) to turn on the "Allow recommendation based on history" function, the "Allow intelligent detection of identity information" function, the "Allow scene recognition" function, and the "Allow matching of video clips based on user identity" function of the mobile phone's video application.

[0194] Through the above Figure 4 The method described here assumes that the user has enabled the personalized recommendation function of the video application of the mobile phone, that is, the method described in the subsequent embodiments can be used to recommend videos that match the user's identity, are close to the scene where the user is, and meet the user's viewing needs. The method for personalized recommendation of video clips will be described in detail below with reference to the accompanying drawings.

[0195] Figure 5 FIG. 1 is a schematic flow chart of a method for personalized video clip recommendation provided in an embodiment of the present application. Figure 5 As shown, the method 500 can be applied to Figure 3 In the illustrated system including the electronic device 100 and the server 200, the method 500 may be divided into three different stages: a video preprocessing stage, a user identity determination stage, and a video matching stage.

[0196] The “video preprocessing stage” can be understood as: after the operator of the video application uploads one or more videos to the server 200 corresponding to the video application, the server 200 first preprocesses each video.

[0197] The "user identity determination stage" can be understood as: after the user runs the video application, the electronic device 100 collects user information, and the electronic device 100 or the server 200 determines the user's identity based on the collected user information, such as an adult user, a minor user, a parent user, an owner user, a child user, other users of the family, etc.

[0198] The "matching video stage" can be understood as: when a user uses a video application, the video application can request the server 200 to obtain video resources, and the server 200 matches one or more videos for the user from the massive video resources and recommends them to the user on the running interface of the video application; or when a user plays a certain video, the target segment of the video is matched and played for the user. The following is a detailed introduction to the three stages.

[0199] [Phase 1: Video preprocessing stage]

[0200] 501 , an operator of a video application uploads a first video file to a server 200 corresponding to the video application, and a media service module of the server 200 obtains the first video file.

[0201] It should be understood that the "first video file" in the embodiment of the present application can also be referred to as the "first video". The operator uploads one or more videos to the server 200, and the "first video" can be any one of the one or more videos. For simplicity, the embodiment of the present application will take the first video as an example to introduce the processing process of the first video.

[0202] It should also be understood that the "server 200" in the embodiment of the present application can be a server corresponding to a video application, for example, Huawei Video, the server 200 is the server corresponding to Huawei Video, if it is Youku Video, the server can be the server corresponding to Youku Video. For the sake of convenience, it is uniformly referred to as server 200. The embodiment of the present application does not limit the video application.

[0203] 502, the media service module of the server 200 analyzes the first video frame by frame to generate metadata of the first video and tag information of the first video.

[0204] During step 502, the media service module of the server 200 can perform intelligent frame-by-frame analysis on the first video based on a variety of media artificial intelligence (AI) algorithms to generate metadata of the first video and tag information of the first video.

[0205] Exemplarily, assuming that the first video consists of N frames, after analyzing each of the N frames of the first video, the label information corresponding to each frame can be obtained, and then according to the label information of each frame, the N frames of the first video are aggregated into multiple fragmented video segments, such as video segment 1, video segment 2, video segment 3, etc.

[0206] It should be understood that the "metadata of the first video" in the embodiment of the present application can be understood as: the content data corresponding to the fragmented video segments obtained by dividing according to certain principles with the label information of each frame of the first video as the dimension. Exemplarily, if the first video includes video segment 1, video segment 2, and video segment 3, the metadata of the first video may include the metadata of video segment 1, the metadata of video segment 2, and the metadata of video segment 3, wherein the metadata of video segment 1 may include basic content information such as image data, audio data, and text data of video segment 1, as well as the media stream address of video segment 1. The embodiment of the present application does not limit the content included in the metadata.

[0207] In one possible implementation, after the operator uploads the video, the media service module can perform one or more dimensions of detection, such as target detection, content recognition, content understanding, and natural language understanding, on each frame based on the media AI algorithm, and then determine the label information of each frame. Then, based on the deep learning algorithm and the label information of each frame, the first video is aggregated according to the segment dimension.

[0208] Optionally, the media service module stores a variety of video analysis algorithms, such as media AI algorithms, deep learning algorithms, etc., and step 502 can be performed by the media service module.

[0209] Alternatively, step 502 may also be performed by other remote processing modules with stronger data processing capabilities, such as other media detection platforms or training platforms. Exemplarily, the media service module of server 200 may synchronize the first video to other remote processing modules such as media detection platforms or training platforms, and use more AI algorithms and training models stored in other media detection platforms or training platforms to perform accurate frame-by-frame video analysis. After other media detection platforms or training platforms process the first video, the metadata of the analyzed first video and the label information of the first video are returned and stored in the media service module, and the media service module manages and stores the metadata of the first video and the label information of the first video. The embodiments of the present application are not limited to this.

[0210] Optionally, each frame of the first video may correspond to multi-dimensional label information. For example, for the first frame of the first video, the media service module detects the content of the first frame and determines that the first frame includes a fighting scene, and at the same time determines that the first frame also includes bleeding content based on content understanding, natural language understanding, etc., then the label information corresponding to the first frame may include fighting and / or bleeding.

[0211] Figure 6 FIG. 1 is a schematic diagram of the result of a frame-by-frame analysis of a video provided in an embodiment of the present application. Figure 6 As shown, it is assumed that the first video includes N frames, where N is greater than or equal to 1. The media service module can detect the content of each frame frame by frame, determine the content included in each frame, and determine the tag information corresponding to the frame according to the content included in each frame.

[0212] For example, taking the first frame to the fifteenth frame of the first video as an example, the media service module detects that the first frame includes fighting content and also detects that the first frame includes bleeding content, then the label information corresponding to the first frame can be generated according to the following multiple methods:

[0213] (1) The content of the first frame is determined as the label information corresponding to the first frame. For example, if the content of the first frame includes "fighting, bleeding", the label information corresponding to the first frame may be "fighting and / or bleeding", which is recorded as "fighting / bleeding".

[0214] (2) Determine the label information corresponding to the first frame of the image based on the type of content included in the detected first frame of the image. For example, the content of the first frame of the image includes "fighting, bleeding", where "fighting" belongs to the "action" type and "bleeding" belongs to the "bloody" type. Then the label information that the first frame of the image can carry is: "action", and further carries label information: "fighting", recorded as "action>fighting". It should be understood that the type to which a certain frame of the image belongs can refer to the existing classification types of videos, such as action, love, family affection, bloody, etc., and the embodiments of the present application do not limit this.

[0215] (3) Similar to the above method (2), according to the type of the content included in the detected first frame, it is determined that the label information that the first frame can carry is any one of "action" and "blood".

[0216] By analogy, the media service module detects the pictures from the 1st frame to the Nth frame of the first video, and determines the label information corresponding to each frame.

[0217] In one possible case, according to the above label generation principle, the same frame may have labels of multiple dimensions. Optionally, the labels of multiple dimensions of the same frame may correspond to different priority orders, for example, determining which label of the multiple dimensions is the priority label based on the labels of adjacent multiple frames.

[0218] For example, Figure 6 Taking the first frame in the video as an example, the first frame corresponds to the "fighting" label and the "bleeding" label. It can be determined that the "fighting" label appears more times according to the labels of the adjacent second to third frames. The "fighting" label can be used as a priority label, so that the "fighting" label and the "bleeding" label correspond to different priority orders, and the priority of the "fighting" label is higher than that of the "bleeding" label. This embodiment of the present application is not limited to this.

[0219] In one possible implementation, after analyzing the first video frame by frame to obtain label information for each frame, the media service module may aggregate labels of video segments according to label similarities between adjacent frames, and then divide the first video into video segments, for example, dividing the first video into multiple video segments, and each of the multiple video segments may correspond to different label information.

[0220] For example, according to Figure 6 Table 1 lists several possible ways of dividing the video segments. Optionally, according to the above-listed multiple ways of generating labels, the first video can be divided into multiple video segments according to different label information.

[0221] Optionally, the tag that appears most repeatedly in the multiple frames included in the video segment can be determined as the tag information of the video segment. Each video segment can have multiple tags.

[0222] Exemplarily, in the video segment division method 1 shown in Table 1, each frame from the 1st frame to the 4th frame included in segment 1 corresponds to the label information of "action>fighting", that is, "action>fighting" is the label with the most repeated occurrences in the 1st frame to the 4th frame included in segment 1, and "action>fighting" is determined as the label information of segment 1. Similarly, in the 5th frame to the 11th frame included in segment 2, "emotion>hug" is repeated the most times, and "emotion>hug" can be determined as the label information of segment 2; in the 12th frame to the 15th frame included in segment 3, "emotion>love" is repeated the most times, and "emotion>love" can be determined as the label information of segment 3.

[0223] Optionally, each frame may include multiple tags, and the tag information of the video segment may be determined according to any tag of each frame among the multiple frames included in the video segment.

[0224] Exemplarily, in the video segment division method 2 shown in Table 1, each frame from the 1st to the 7th frame included in segment 1 corresponds to two kinds of label information, and the "bloody" corresponding to the bleeding scene that appears most frequently can be used as the label information of segment 1; each frame from the 8th to the 11th frame included in segment 2 corresponds to two kinds of label information, and any one of "speeding" can be used as the label information of segment 2; each frame from the 12th to the 15th frame included in segment 3 corresponds to one kind of label information, and "emotion" can be used as the label information of segment 3.

[0225] Or, exemplarily, as shown in Table 1, in the video segment division method 3, each frame from the 1st frame to the 4th frame included in segment 1 corresponds to two types of label information, and any one of the "actions" can be used as the label information of video segment 1, and each frame from the 5th frame to the 15th frame included in segment 2 includes an "emotion" label, and "emotion" can be used as the label information of video segment 2, which will not be repeated here.

[0226] Table 1

[0227]

[0228]

[0229] In another possible implementation, in the process of aggregating the first video into video segments according to the label similarity between adjacent frames, there may be a few frames between multiple consecutive frames of the first video whose labels are different from those of adjacent consecutive frames. In this scenario, a noise threshold can be set. The noise threshold can be used to mark the critical number M of frames with discontinuous labels, where M is greater than or equal to 1.

[0230] Optionally, when the media service module determines that the labels of more than K consecutive frames are different from the labels of other adjacent frames, if K is greater than or equal to M, the first frame of the K consecutive frames is used as the dividing line to segment the video segments; if K is less than M, the influence of noise can be ignored, and the K consecutive frames and the previous frame adjacent to the K frames are divided into the same video segment.

[0231] For example, assuming that the noise threshold is 3, the label of the 3rd frame of the first video in Table 1 is "fighting / bad guy", which is different from the labels of the 1st, 2nd and 4th frames, but the number 1 is less than the critical number 3, so the "fighting / bad guy" label of the 3rd frame can be ignored, and the 3rd frame can continue to be divided into video segment 1 (including the 1st to 4th frames), and the label of video segment 1 is "action> bloody", which will not be repeated here.

[0232] By setting the noise threshold, when a judgment error occurs in the label information of a certain frame during the detection process of one or more dimensions such as target detection and content recognition, the noise threshold can ignore the impact of the error on the aggregation of video segments, thereby improving the accuracy of the judgment of the label information of the video segment and the accuracy of dividing the video segments.

[0233] In another possible implementation, after determining the division method of the video segments of the first video and the label of each video segment, the media service module can determine the playback progress corresponding to each video segment, that is, determine the playback start time and end time corresponding to each video segment. In other words, the media service module can determine the association between the label information of each video segment and the playback progress of the video segment.

[0234] It should be understood that the "playback progress of each video clip" here can be understood as the corresponding time period between the start time and the end time occupied by the video clip within the complete length of the first video (long video).

[0235] For example, taking the video segment division method 1 in Table 1 above as an example, the video segment 1 includes the 1st to 4th frames, and the display time of each frame is known, for example, 16.67 milliseconds (ms), then the start time (for example, 00:00) and end time (for example, 00:09) corresponding to the video segment 1 can be accurately determined. Similarly, the media service module can accurately determine the playback start time and end time of each video segment such as video segment 2, video segment 3, etc., which will not be repeated here.

[0236] With respect to the process introduced above, the media service module of the server 200 can generate metadata of the first video and tag information of the first video after analyzing the first video frame by frame. Optionally, Table 2 lists the parameter content that may be included in the tag information of a possible video clip. Exemplarily, as shown in Table 2, for the first video, the first video corresponds to a unique "address of the content to be played (content Identification, content ID)" and "tags data". Among them, the "address of the content to be played (content Identification, content ID)" is the media stream address or playback address of the first video introduced above; the parameters corresponding to the tags are represented by "TagInfo", that is, a list of tag information of multiple video clips included in the first video.

[0237] It should be understood that the parameters listed in Table 2 may be mandatory or optional (M / O) information. In the embodiment of the present application, the ID (content ID) and tags (tags) of the playback content in Table 2 are both mandatory information.

[0238] Table 2

[0239] Parameter name Parameter Type M / O Parameter length (bytes) Parameter Description content ID string M 128 ID of the content being played tags List <taginfo> < / taginfo> M Long video tag information list

[0240] The parameter "TagInfo" corresponding to the tag (tags) of the first video in Table 2 may further include more core parameters, and Table 3 lists a possible core parameter content of TagInfo. As shown in Table 3, combined with the aforementioned content, the media service module may generate tag information corresponding to each video segment included in the first video, the start time and end time occupied by each video segment, and other information. Specifically, the tag information corresponding to each video segment may include tag semantics (theme ID) and tag name (tag name), etc.

[0241] Exemplarily, in combination with the tag information listed in Table 1, taking the "action>fighting" tag as an example, the tag semantics (themeID) may indicate that the tag information includes "action", and indicates that the category (secondary classification) that the tag information may further include is "fighting"; or, taking "emotion>hug" as an example, the tag semantics (theme ID) may indicate that the tag information includes "emotion", and indicates that the category (secondary classification) that the tag information may further include is "hug", which will not be repeated here.

[0242] Table 3

[0243] Parameter name Parameter Type M / O Parameter length Parameter Description theme ID int M / Tag semantics and the secondary classification to which it belongs tag name int M / Tag Name start time string M / The start time corresponding to the playback progress end time string M / End time corresponding to the playback progress

[0244] Through the process introduced above, in step 502, the media service module stores and manages the metadata of the first video and the tag of the first video, which can be understood as: the media service module stores and manages the metadata of each video segment included in the first video and the tag information of each video segment, and stores the association between the tag information of each video segment and the playback progress of the video segment.

[0245] 503. The media service module sends the metadata of the first video to the content service module.

[0246] Optionally, according to the process of step 502 introduced above, in step 503, the media service module can send the metadata of each video segment included in the first video to the content service module, so that when the user uses a video application to play the first video, combined with the user's query, search and other operations, the content service module can provide different services for different users based on the metadata of each video segment included in the first video, such as recommending a certain video to the user, determining a certain video segment of the video that can be played for the user, and reviewing a certain video segment of the video.

[0247] 504. The media service module sends the tag information of the first video to the distribution module.

[0248] Optionally, according to the process of step 502 introduced above, in step 504, the media service module can send the label information of each video segment included in the first video to the distribution module, as well as the association between the label information of each video segment and the playback progress of the video segment, that is, the various parameters listed in Tables 2 and 3 above, which will not be repeated here.

[0249] In a possible implementation, after analyzing the first video frame by frame, the media service module can store the tag information of one or more video segments of the first video, and provide a preview view for the operator to confirm and adjust. If the operator makes manual adjustments, the adjustment results can be synchronized in the media service module, that is, the media service module can update some parameters in Table 2 and Table 3, and save the updated parameters.

[0250] Optionally, if step 502 is sent by the media service module to other remote processing modules with stronger data processing capabilities, such as other media detection platforms or training platforms, the other media detection platforms or training platforms will return the metadata of the first video and the tag information of the first video obtained by analysis and store them in the media service module. If the operator makes manual adjustments to the media service module, the adjustment results can be synchronized to other media detection platforms or training platforms for deep learning to correct its own detection and other processes according to the adjustment results of the operator, which will not be repeated here.

[0251] Through the process of steps 501-504 above, the massive amount of videos uploaded by the operator can be processed, and the metadata of each video in the massive amount of videos and the tag information of each video can be stored, which will not be described one by one this time.

[0252] [Phase 2: User identity confirmation]

[0253] 505 , the user runs a video application, triggering the collection and detection module of the electronic device 100 to collect user information.

[0254] In one possible implementation, assuming that the user has already Figure 4 The process shown in Figures (a) to (d) in the figure enables the personalized recommendation of video clips function of the video application of the electronic device 100 (mobile phone), and enables the "allow intelligent detection of identity information" function, or enables the above functions through other shortcut operations or preset gestures, etc., which is not limited to the embodiments of the present application.

[0255] Optionally, when the personalized recommendation video clip function of the video application of the electronic device 100 (mobile phone) is turned on and the "allow intelligent detection of identity information" function is turned on, the electronic device 100 can be opened when the user clicks the icon of the video application (i.e., opens the video application) and enters the video application. Figure 1 When the running interface 102 of the video application shown in Figure (b) is displayed, the collection and detection module of the electronic device 100 can be triggered to collect user information.

[0256] Alternatively, the collection and detection module of the electronic device 100 can periodically collect user information according to a certain period (for example, 10 times per minute). For example, the electronic device 100 can periodically collect the current user information to determine the identity of the current user during the user's use of the video application. The embodiment of the present application does not limit the timing of the electronic device 100 collecting user information.

[0257] Alternatively, the collection and detection module of the electronic device 100 may not collect user information, and the user may manually set the current user identity of the electronic device 100.

[0258] For example, the user can manually set the user in the current usage period as a child user in the settings application or video application, and the collection and detection module of the electronic device 100 can no longer collect user information, and recommend videos to the child user or control the video playback process according to the user's settings. Alternatively, the user can associate the user account currently logged in to the electronic device 100 or the account logged in to the video application with the child user, and the collection and detection module of the electronic device 100 can no longer collect user information, recommend videos to the child user or control the video playback process. The embodiment of the present application does not limit the method of determining the user identity.

[0259] The following introduces several implementation processes of the collection and detection module of the electronic device 100 to collect the current user information.

[0260] Optionally, the acquisition and detection module of the electronic device 100 may include one or more devices such as a camera, a fingerprint sensor, a touch sensor, an infrared sensor, etc., wherein the camera is not limited to the front camera, rear camera or under-screen camera of the electronic device 100.

[0261] In addition, the user information collected by the collection and detection module may include one or more information such as the user's facial features, skin condition, fingerprint information, height information, etc., which is not limited to the embodiments of the present application.

[0262] Exemplarily, taking a mobile phone as an example, the collection and detection module may be a camera of the mobile phone, which may be used to obtain information such as the user's facial features and skin condition; or, the collection and detection module may be a fingerprint sensor or a touch sensor of the mobile phone, which may be used to collect the user's fingerprint information; or, the collection and detection module may be an infrared sensor of the mobile phone, which may be used to collect the user's facial features based on the reflection of infrared light on the user's face, etc. The embodiments of the present application are not limited to this.

[0263] Optionally, after the collection and detection module of the electronic device 100 collects the user information, the collected user information can be transmitted to the processor 110 of the electronic device 100, and the identity of the user can be determined by the processor 110. Alternatively, after the collection and detection module of the electronic device 100 collects the user information, the collected user information can be uploaded to the user service module or distribution module of the server 200, and the user service module or distribution module of the server 200 can further determine the identity of the user. The following describes these two possible methods respectively.

[0264] Method 1

[0265] 506 , the collection and detection module of the electronic device 100 collects user information, and the processor 110 of the electronic device 100 determines the identity of the user based on the collected user information.

[0266] Optionally, the camera 193 of the electronic device 100 can collect facial information of the current user and transmit the facial information to the processor 110. The processor 110 performs feature recognition and comparison based on the facial information to determine the user's age, height, clothing, occupation and other information, for example, the current user is: a child user or a minor user; an adult user; an elderly user, etc.

[0267] Alternatively, the camera 193 of the electronic device 100 can collect facial information of the current user and pass the facial information to the processor 110. The processor 110 can compare the facial information with the facial information of the owner of the electronic device 100, and determine whether the current user is the owner, parent, child, etc. through feature comparison, etc., or determine whether the current user is the user corresponding to the user account logged in to the electronic device 100, etc., which will not be repeated here.

[0268] It should be understood that if the electronic device 100 recognizes that the current user is a different user identity such as the owner user, the parent user, or the child user, it can further recommend videos based on the current user identity. For example, educational videos can be recommended for the parent user; for the owner user, videos with the same type or similar subject content as the owner user's historical playback records can be matched; for the child user, children's animation and learning videos can be recommended, etc., which are not limited in the present embodiment of the application.

[0269] It should also be understood that with the improvement of the data processing capability and computing power of the electronic device 100, and the enrichment of algorithms such as image detection and recognition, the electronic device 100 can independently realize the identification of user identities, thereby reducing the interaction process between the electronic device 100 and the server 200 and accelerating the rate of user identity identification.

[0270] 507 , the collection and detection module of the electronic device 100 returns the user identity information to the video application.

[0271] Exemplarily, when it is determined in step 506 that the current user is a child user, the collection and detection module may return the identity recognition result to the video application, and the video application determines that the user who is currently using the video application to request to play a video is a child user.

[0272] Optionally, the electronic device 100 can encrypt and store the identity recognition result, for example, by associating the identity recognition result with the facial features of the current user and saving them in encrypted form. If the facial features are detected again next time, the identity recognition result can be directly called without the need to perform feature matching and comparison processes to determine the user identity. This process can simplify the recognition process and reduce the detection workload and power consumption of the electronic device.

[0273] Method 2

[0274] 508, the collection and detection module of the electronic device 100 collects user information.

[0275] 509 , the collection and detection module of the electronic device 100 reports the collected user information to the user service module and / or distribution module of the server 200 through the communication module.

[0276] It should be understood that the communication module here is a module that both the electronic device 100 and the server 200 have. The communication module of the electronic device 100 and the communication module of the server 200 can be based on a variety of possible communication methods, such as WIFI, 5G / 6G and other communication methods, etc., and the embodiments of the present application are not limited to this.

[0277] 510. The user service module of the server 200 further determines the identity of the user according to the received user information, and encrypts and stores the identity information of the user.

[0278] 511 , the user service module of the server 200 returns the identity recognition result to the video application of the electronic device 100 through the communication module.

[0279] Optionally, the camera 193 of the electronic device 100 can collect facial information of the current user and transmit the facial information to the server 200. The server 200 performs feature recognition based on the facial information, such as comparison of skin texture, etc., to determine the user's age, height, clothing, occupation and other information, and judge whether the current user is: a child user or a minor user; an adult user; an elderly user, etc.

[0280] Alternatively, the camera 193 of the electronic device 100 can collect facial information of the current user and transmit the facial information to the server 200. The server 200 performs facial feature comparison, etc. to determine whether the current user is the owner of the device, or whether the current user is the user corresponding to the user account logged in to the electronic device 100, etc., which will not be repeated here.

[0281] It should be understood that the user service module of the server 200 can receive user feature data reported by the electronic device 100, create user portraits based on massive user feature data, and encrypt and store a large amount of data of different user portraits. After subsequently receiving the user information reported by the electronic device 100, it can quickly query the large amount of stored user portrait data and quickly determine the identity of the current user, thereby improving the rate of user identity recognition and reducing the amount of data processing. In addition, it can also improve the accuracy and robustness of user identity recognition.

[0282] Optionally, the electronic device 100 can determine the identity information of the current user through any one of the above-mentioned methods one and two, or it can be based on the processes in method one and method two at the same time. For example, method one can be used to detect whether the current user is the owner of the electronic device, etc., and method two can be used to determine the identity information such as the age of the current user. This application example is not limited to this.

[0283] 512, a window automatically pops up on the electronic device 100 to remind the user that the current electronic device is about to switch working modes.

[0284] 513, the user performs an operation allowing the working mode to be switched in the pop-up window, and in response to the operation allowing the working mode to be switched, the electronic device 100 switches the working mode.

[0285] Optionally, the pop-up window on the electronic device 100 may automatically disappear after receiving the user's permission operation; or, if the user does not perform the permission operation within a preset time period, the pop-up window may disappear actively after being displayed for a preset time period (for example, 5 seconds), and it is assumed that the current user identity is correctly identified, and video content can be requested according to the user identity. This embodiment of the present application is not limited to this.

[0286] Optionally, steps 512 and 513 are executed only when it is detected that the user identity does not match the current working mode. Exemplarily, when the current user is determined to be a child user according to the above method 1 or method 2, and the current working mode of the electronic device 100 matches the parent user or the adult user, the process of steps 512 and 513 will be executed, and a window will automatically pop up on the electronic device 100 to prompt the user that the current electronic device 100 is about to switch the working mode.

[0287] It should be understood that when it is detected in step 507 or step 511 that the user identity matches the current working mode, the processes of step 512 and step 513 may not be executed, that is, the user is not required to confirm whether to switch the working mode of the electronic device 100, and the electronic device 100 can continue to recommend videos to the user or control the video playback process according to the current user identity. Figure 7 This is a schematic diagram of an interface for recommending video clips to users on a mobile phone, provided in an embodiment of the present application.

[0288] For example, taking a mobile phone as an example, the mobile phone can display the following when it is unlocked: Figure 7 In the interface 701 shown in FIG. (a), the user clicks the icon of the video application on the interface 701. In response to the user's click operation, the mobile phone runs the video application and displays the following Figure 7 (b) shows the main interface 702 of the video application. When the mobile phone starts to run the video application and displays the main interface 702 of the video application, the mobile phone can be triggered to perform the operation of collecting user information in step 505. Further, the mobile phone can determine the identity of the current user according to the process of steps 506-507 or steps 508-511 described above.

[0289] After the mobile phone 100 determines the user's identity, a pop-up window such as Figure 7 The prompt window 20 shown in Figure (c) can display prompt information to tell the user the currently recognized user identity and that the working mode of the mobile phone will be switched next, and match videos or video clips for the user according to the currently recognized user identity.

[0290] Exemplarily, when the current user is identified as a child user (or underage user), the mobile phone can automatically switch to child mode, and a prompt message can be displayed in the prompt window 20: Dear user, you are identified as a underage user, and some video clips will be blocked from display.

[0291] The user can choose whether to receive the recognition result of the user identity in the prompt window 20 according to his own needs. Figure 7 In the operation shown in FIG. (c), click the "OK" option in the prompt window 20. In response to the user's click operation, the prompt window 20 disappears and the mobile phone resumes displaying the following Figure 7 In the interface 704 shown in FIG. 7 (d), the mobile phone switches to the children's mode and further enters the stage of matching videos.

[0292] In one possible situation, an error may occur in the user identification process, and the current user may be mistakenly identified as a minor user; or, even if the current user is a minor user, the minor user still wishes to play the entire content of the first video, then the user can click the "Cancel" option in the prompt window 20.

[0293] Exemplarily, when the user clicks the "Cancel" option in the prompt window 20, in response to the user's click operation, the prompt window 20 disappears, and the mobile phone resumes displaying the following Figure 7 In the interface 704 shown in Figure (d), an identity authentication window can pop up on the mobile phone to verify the user's identity. The verification method is not limited to fingerprint verification, digital password verification, face verification, etc., and the embodiment of the present application does not limit this.

[0294] When the user's identity verification is passed, the mobile phone will not switch to the child mode, that is, it will maintain the current normal mode, recommend videos or video clips to the user normally in the existing way, and play the complete content of the video 2, etc., which will not be repeated here.

[0295] Optionally, the prompt window 20 may automatically disappear after the user clicks the "OK" option; or, if the user does not click the "OK" option within a preset time period, the prompt window 20 may actively disappear after displaying for a preset time period (e.g., 5 seconds), and it is assumed that the identity of the current child user is correctly identified, and relevant videos matching the child user may be recommended according to the identity of the current child user. This embodiment of the present application is not limited to this.

[0296] Through the process introduced above, the electronic device 100 or the video application can accurately determine the identity or age information of the current user, for example, the current user is: a child user or a minor user; a parent user or an adult user; an elderly user, etc., and in the subsequent process of the user requesting to play a video, different services are provided to meet the usage needs of different users.

[0297] [Phase 3: Video Matching Phase]

[0298] 514, the user performs an operation of playing the first video.

[0299] For example, Figure 7 As shown in FIG. 7(d), the user can find video 2 (first video) on the running interface 704 of the video application and click the icon of video 2 (first video). In response to the user's click operation, the electronic device 100 displays the following Figure 7 The interface 705 shown in FIG. 8 (e) starts playing the video 2 (the first video) on the interface 705.

[0300] 515 , in response to the user's play operation, the video application of the electronic device 100 sends a play request for the first video to the content service module of the server 200 , and the play request for the first video carries the user identity information.

[0301] It should be understood that the playback request of the first video can be used to request the address of the playback content (content Identification, content ID) of the first video. The video application of the electronic device 100 can obtain the image data, audio data, etc. corresponding to the first video from the address of the playback content, thereby realizing the normal playback of the first video, which will not be repeated here.

[0302] Exemplarily, Table 4 lists a possible parameter content that may be included in the playback request of the first video. Exemplarily, as shown in Table 4, the playback request of the first video sent by the electronic device 100 may include information such as the user's age mode information, the video operation column ID (category ID) and the playback address (mv ID) of the first video. Optionally, when the user manually sets the current user identity in the settings application or video application, for example, setting the current user to a child user, the playback request of the first video may also include the parameter content listed in Table 4, or the playback request of the first video may also include other user characteristic parameters for indicating that the current user is a child user, which is not limited in the embodiments of the present application.

[0303] Exemplary, combined Figure 4 As shown in Figure (b), the video operation column ID (category ID) can be used to determine the content of the interface 402 corresponding to the "Daily Recommendation" menu, or to determine the content of the interface corresponding to the "TV Series" menu, "Movie" menu, etc.; the playback address (mv ID) of the first video can be used to determine the playback address of any video (such as video 2) on the interface 402 corresponding to the "Daily Recommendation" menu, which will not be repeated here.

[0304] Table 4

[0305] Parameter name Parameter Type M / O Parameter length (bytes) Parameter Description age mode int / / Is it a child mode? category ID string / / Video operation column ID mv ID string / / Video content ID

[0306] 516 , the content service module of the server 200 determines the first video selected for playback by the user according to the playback request of the first video, and requests the distribution module of the server 200 to query the tag information of the first video.

[0307] 517, the distribution module of the server 200 obtains the user identity information, and determines the filtered segments of the first video according to the metadata of the first video, the tag information of the first video and the user identity information, that is, determines the video segments that need to be filtered out or shielded for child users among the multiple video segments included in the first video, or determines the target segments that can be played for child users among the multiple video segments included in the first video.

[0308] Specifically, the content service module of the server 200 can obtain the user identity information included in the play request of the first video, send the user identity information to the distribution module, and request the distribution module to find a target segment that matches the child user and can be played for the child user from the multiple video segments included in the first video. The distribution module of the server 200 determines the target segment that matches the child user identity in the tag information storing the first video.

[0309] Exemplarily, according to the process of step 502 to step 504 of the video preprocessing stage described above, the first video can be divided into one or more video segments after frame-by-frame analysis, and each video segment can include the tag information shown in Table 2 and Table 3. When the distribution module of the server 200 determines that the current user is a child user, the tag information of each video segment included in the first video can be queried to further determine the tag matching the child user, and then the target segment and the corresponding playback start time and end time of the target segment can be determined based on the association between the tag information stored for each video segment and the playback progress of each video segment.

[0310] Optionally, in the embodiment of the present application, the “filtered segment of the first video” can be understood as: Figure 7 As shown in Figure (f), the video segment within the duration t is skipped, and when the video is played to the starting position of the gray progress bar, the segment corresponding to the duration t is skipped and continued to be played; or it can be understood that: the metadata sent by the server 200 to the electronic device 100 is the metadata of the first video remaining after deleting the video metadata within the duration t corresponding to the gray progress bar, and the embodiments of the present application are not limited to this.

[0311] In another possible implementation, the play request of the first video in the above step 515 may also include information such as the user's account information and / or the user's historical play records.

[0312] Exemplarily, in step 516 and step 517, if the play request of the first video includes the user's account information, the content service module requests the distribution module to query the tag information of each video, and informs the distribution module of the user's account information. The distribution module can match the user's historical play records based on the user account information, and draw a behavioral characteristic profile of the user based on the historical play records. After completing the behavioral characteristic profile of the user, one or more videos are matched for the user in combination with the tag information of each video from the video content library.

[0313] Furthermore, the label of each video segment included in the first video can be queried according to the user identity information, and the target segment in the first video that can be played for the user can be determined. If the current user is a child user, the content service module also needs to determine the video segments that need to be filtered out or shielded for the child user among the multiple video segments included in each of the one or more videos, or in other words, the content service module also needs to determine the target segment that can be played for the child user among the multiple video segments included in each of the one or more videos, and then determine the response message, which will not be repeated here.

[0314] Through the above method, the process of recommending videos to users or controlling video playback can be combined with the user's account information and / or the user's historical playback records to recommend videos or video clips that better match the user's viewing needs, thereby improving the user's viewing experience.

[0315] 518, the distribution module of the server 200 sends the filtered first video metadata to the video application of the electronic device 100. Here, "filtered first video metadata" can be understood as metadata of the target segment included in the first video.

[0316] Exemplarily, Table 5 lists a possible parameter content that may be included in the metadata of the target segment in the first video obtained after filtering. Exemplarily, as shown in Table 5, the metadata of the target segment in the first video after filtering sent by the server 200 to the electronic device 100 can be identified by the parameter "mvInfos", that is, it indicates the target segment of the first video after filtering and the video list after the video segment.

[0317] Table 5

[0318]

[0319] The parameter "mvInfos" corresponding to the information of the video list after filtering videos and video clips in Table 5 may further include more core parameters, and Table 6 lists a possible core parameter content of mvInfos. The information for controlling the playback progress of the first video in the child mode may be identified by the parameter "age modeTimeInfos".

[0320] Table 6

[0321]

[0322] The parameter "agemodeTimeInfos" corresponding to the information for controlling the playback progress of the first video in the child mode in Table 6 may further include more core parameters. Table 7 lists a possible core parameter content of agemodeTimeInfos. It can be seen from Table 7 that the parameter "agemodeTimeInfos" corresponding to the information for controlling the playback progress of the first video may further include the playback start time and playback end time corresponding to the filtered target segment that can be played for child users, which will not be elaborated one by one this time.

[0323] Table 7

[0324] Parameter name Parameter Type M / O Parameter length Parameter Description start time string M / The start time corresponding to the playback progress end time string M / End time corresponding to the playback progress

[0325] Through the above process, the distribution module of the server 200 sends the metadata of the filtered target segments listed in Tables 5 to 7 above to the video application of the electronic device 100. The video application of the electronic device 100 can accurately play the target segment of the first video for the child user based on the metadata of the filtered target segment.

[0326] 519, the electronic device 100 plays the filtered partial segment of the first video through the video application, that is, only plays the target segment in the first video that matches the child user.

[0327] For example, if the current user is a child user and the mobile phone has switched to the child mode, the mobile phone will only play the filtered video clips of the video 2 (the first video) for the user during the video 2 (the first video). Figure 7 As shown in Figure (e), the playback progress bar of Video 2 (the first video) includes a black area, a white area and a gray area, wherein the black area is the part that has been played, the white area is the part that has not been played, and the gray area is the unplayable part (the filtered out segment). It is assumed that the gray area of ​​the progress bar corresponds to a video segment with a duration of t.

[0328] Exemplarily, in combination with the parameter contents in Table 7, the start time (start time) corresponding to the playback progress and the end time (end time) corresponding to the playback progress can be used to determine the video segment with a duration t indicated by the gray area of ​​the progress bar.

[0329] It should be understood that in the embodiment of the present application, the gray area of ​​the progress bar is used to indicate the non-playable video segment, and the black area and the white area are used to indicate the playable video segment, which will not be described in detail later.

[0330] and Figure 1 Compare with the progress bar shown in Figure (c) in Figure 1 The progress bar shown in Figure (c) is black, which means that all the video clips included in Video 2 can be played. Figure 7 In (e) of the figure, video 2 is matched with the child user, and video clips with violent themes, bloody themes, etc. that are not suitable for child users to watch are filtered out for the child user. Only the video clip corresponding to the black progress bar can be played, that is, only the video clip that meets the child user's viewing level can be played.

[0331] In addition, when the playback progress of video 2 continues to reach the critical point, you can Figure 7 As shown in FIG. (f) of FIG. 1 , the mobile phone automatically pauses the playing process of the first video, for example, displays a pause icon 30 on the video playing screen, and further automatically pops up a prompt window 40, which can prompt the user that the video clip is restricted from playing in the children's mode. The critical point is the boundary between the playable video clip and the unplayable video clip, that is, the boundary between the black progress bar and the gray progress bar.

[0332] In another possible implementation, when the electronic device 100 plays the filtered partial segment of the first video through a video application, it may only display a progress bar corresponding to the target segment matched for the child user in the first video.

[0333] For example, Figure 7 As shown in FIG. 1( g ), during the playing of the first video, the filtered video segment with a duration of t may not be displayed on the progress bar. Figure 7 The playback operation shown in Figure (d) corresponds to the user's playback operation. The mobile phone can display Figure 7 In the interface 707 shown in Figure (g), assuming that the original total length of video 2 (first video) is 50 seconds, and the length of the filtered video segment is t = 20 seconds, then during the playback of video 2 (first video), the video length displayed on the progress bar is 50-20 = 30 seconds. The embodiment of the present application does not limit the display method of the progress bar.

[0334] In another possible implementation, the server 200 can not only filter some segments of the first video for the child user and only play the target segments that match the child user's viewing for the child user, but also match the recommended videos for the child user according to the child user's identity, and further send the matching video information to the electronic device 100, and the electronic device 100 updates the video list displayed in the video recommendation menu according to the matching video information. Specifically, the process can also include the following steps:

[0335] 520. The content service module of the server 200 obtains the user identity information included in the playback request of the first video, and determines one or more videos according to the user identity.

[0336] 521 , the content service module of the server 200 sends information of one or more videos to the electronic device 100 through the communication module.

[0337] 522 , the video application of the electronic device 100 updates the video list recommended for the user based on the information of one or more videos.

[0338] For example, if the user Figure 7 Figure (f) in returns to Figure 7 If the interface 704 corresponding to the "Daily Recommendation" menu shown in FIG. (d) is changed, the video content included in the video list displayed on the interface 704 may change. For example, the electronic device 100 may determine that the original videos 1, 2, 3, 4, and 5 on the interface 704 corresponding to the "Daily Recommendation" menu do not match the child user, and may update the video content on the interface 704 to multiple videos that match the child user, such as replacing the original videos 1, 2, 3, 4, and 5 with videos 6, 7, 8, 9, and 10; or, when the original videos 1, 2, 3, 4, and 5 match the user identity, the video 2 may continue to be displayed, which will not be repeated here.

[0339] Optionally, step 520 and step 516 are performed simultaneously, and step 518 and step 521 can be the same step, that is, the server 200 can send the metadata of the filtered first video and the information of one or more videos recommended to the user to the video application of the electronic device 100 at one time. Exemplarily, when step 518 and step 521 can be the same step, the parameter "mvInfos" corresponding to the metadata of the target segment in Table 5 can also include one or more video information.

[0340] Alternatively, step 518 and step 521 may be different steps, for example, when the user returns to Figure 7When the interface 704 corresponding to the "Daily Recommendation" menu shown in Figure (d) is displayed, the process of steps 521 and 522 is executed. The embodiment of the present application does not limit the execution order and timing of the steps.

[0341] In another possible scenario, after the user runs the video application, he may quickly perform the operation of playing the first video. In response to the user's play operation, the electronic device 100 can directly and quickly play the first video. In this case, the electronic device 100 may not have completed the process of collecting user information and determining the user's identity.

[0342] Figure 8 This is another example of a process for recommending video clips to a user on a mobile phone provided by an embodiment of the present application. Figure 8 This section describes the possible implementation process and interface in this scenario.

[0343] For example, taking a mobile phone as an example, the mobile phone can display the following when it is unlocked: Figure 8 In the interface 801 shown in FIG. (a), the user clicks the icon of the video application on the interface 801. In response to the user's click operation, the mobile phone runs the video application and displays the following Figure 8 The main interface 802 of the video application is shown in FIG. (b). The user quickly finds the video 2 (the first video) on the running interface 802 of the video application and executes the following steps: Figure 8 In the operation shown in FIG. 1 (b), the icon of the video 2 (the first video) is clicked. In response to the user's click operation, the electronic device 100 displays the following Figure 8 The playback interface 803 of the video 2 (the first video) is shown in FIG. 8 (c), and the video 2 (the first video) starts to be played on the interface 803.

[0344] It should be understood that when the mobile phone starts running the video application and displays the main interface 802 of the video application, it can trigger the mobile phone to execute the operation of collecting user information in step 505. It is possible that after the mobile phone starts playing video 2 (the first video), the mobile phone determines the identity of the current user according to the process of steps 506-508 or steps 508-511 described above.

[0345] If the mobile phone determines the user identity after the mobile phone starts playing video 2 (the first video), for example, after the mobile phone determines that the current user is a child user (or a minor user), a pop-up message such as Figure 8 The prompt window 20 shown in Figure (c) can display prompt information to tell the user the currently identified user identity and that the working mode of the mobile phone will be switched to child mode next, and videos or video clips will be matched for child users, which will not be repeated here.

[0346] The user can choose whether to receive the recognition result of the user identity in the prompt window 20 according to his own needs. Figure 8 In the operation shown in FIG. (c), click the "OK" option in the prompt window 20. In response to the user's click operation, the prompt window 20 disappears and the mobile phone resumes displaying the following Figure 8 The interface 804 shown in FIG. (d) in FIG. 1 is displayed, and the mobile phone switches to the children's mode and further enters the stage of matching videos. At this time, the mobile phone plays only the filtered video clips of the video 2 (the first video) for the user during the video 2 (the first video). Figure 8 As shown in Figure (d) in FIG. 1 , the progress bar of video 2 (the first video) further displays a gray area. The progress bar of the gray area corresponds to a video clip with a duration of t. Please refer to Figure 7 For the sake of brevity, some descriptions in are not repeated here.

[0347] Furthermore, when the playback progress of video 2 continues to reach the critical point, Figure 8 As shown in Figure (e), the mobile phone automatically pauses the playback process of the first video, for example, a pause playback icon 30 is displayed on the video playback screen, and a prompt window 40 is further automatically popped up, and the prompt window 40 can prompt the user: the video clip is restricted from playback in children's mode.

[0348] In another possible scenario, a user may receive a video or a link to each video in other chat applications such as WeChat, and when the user clicks the video or the link to the video, the video can be directly triggered to play. In this scenario, the playback process of the video can also be controlled according to the method provided in the embodiment of the present application.

[0349] For example, if a mobile phone receives a video link, the user can directly trigger the start of playing the video after clicking the video link, that is, the video can be jumped to display Figure 8 At the same time, the mobile phone can send a request to the server to play the video corresponding to the video link. The server determines the tags of multiple segments included in the video, filters the video for the user, and returns the video filtering result to the mobile phone. The mobile phone can then Figure 8 The process shown in Figures (d) and (e) continues to play the video for the user and will not be repeated here.

[0350] In summary, through the above method, after the operator uploads the video file to the server, the frame-by-frame detection and intelligent analysis process of the video can be triggered based on AI algorithms, etc., to obtain the multiple fragmented video clips included in the video and the label information of each video clip. In the process of dividing the video into multiple fragmented video clips and determining the label information of each video clip, the dimension of label extraction can realize the intelligent classification and extraction of video labels based on the behavioral characteristics of the user input. On the one hand, this process avoids the problems of filming efficiency, label accuracy, and label effectiveness caused by manual operation; on the other hand, the label information of each video clip detected by the server can be further determined or adjusted by the operator, and the adjustment result can update or correct the existing frame-by-frame detection algorithm. This process can form an internal loop link in the form of "label detection and extraction-intelligent optimization of label detection algorithm-video clip push with different labels-user experience effect" in the full process, thereby improving the accuracy of generating label information.

[0351] When a user requests to play a video through a video application, user information can be collected, and the user identity and user profile can be accurately determined based on the collected user information. Combined with the label information of each video clip and the user's preferences, behavioral habits, etc., a refined personalized video can be provided for the current user, or a target clip in a video can be matched for the current user.

[0352] If the current user is a child user or a minor user, the method can control the electronic device to switch to the child mode, and control the range of movie permissions that can be played by the electronic device in the child mode, and can also block video clips in a certain video that are not suitable for children to watch. This process does not require manual setting of the electronic device to switch to the child mode, and can achieve the effect of finely controlling video playback in the child mode, thereby improving the user experience.

[0353] In another possible scenario, the electronic device 100 may be a device with a relatively fixed position, such as a smart screen or a vehicle-mounted device. The electronic device with a relatively fixed position, such as a smart screen or a vehicle-mounted device, may be used by different users at different stages, or by multiple users at the same time. For example, the following possible scenarios are possible:

[0354] (1) The bedroom, study, living room, and kitchen of the home may all be equipped with smart screens and other home devices. Users may use the smart screen in the bedroom, study, living room, and kitchen at different stages;

[0355] (2) Multiple users in a family, such as parents, children, and the elderly, may use the smart screen in the living room to watch movies, TV series, variety shows, etc. at night;

[0356] (3) When a family car is used by different family members, different family members may expect the audio and video service on the car device to recommend different content.

[0357] For the different scenarios listed above, if you only follow Figure 5 The process introduced in method 500 collects user information to identify the identity of the user and recommends different videos or other audio-visual content to the user, which cannot meet the usage needs of multiple users.

[0358] For example, for the above scenario (1), the same user uses the smart screen in the bedroom, study, living room, and kitchen respectively. Videos are recommended to the user only based on the user's identity, and videos of the same type and content may be recommended to the user; or, for the above scenario (2), when parents, children, the elderly, etc. use the smart screen in the living room together, different user groups cannot meet everyone's viewing needs at the same time, and videos that better meet the current viewing needs of different user groups are recommended.

[0359] Therefore, the embodiment of the present application also provides another method for personalized recommendation of video clips, so as to recommend videos that are suitable for the current scene and the user's habits to the user for different scenes.

[0360] Fig. 9 FIG. 1 is a schematic flow chart of another method for personalized video clip recommendation provided in an embodiment of the present application. Fig. 9 As shown, the method 900 can be applied to Figure 3 In the system shown including the electronic device 100 and the server 200, the method 900 can be divided into two different stages: a scene determination stage and a video content recommendation stage for the user.

[0361] Among them, the "determining scene stage" can be understood as: the electronic device 100 obtains the scene information of the current user, and the electronic device 100 or the server 200 performs scene recognition to further determine the scene of the current user.

[0362] The "stage of recommending video content to the user" can be understood as: the electronic device 100 requests the server 200 to obtain one or more video information, and the server 200 recommends personalized videos to the user according to the scene the user is currently in. The following is a detailed introduction to the two stages.

[0363] [Phase 1: Determine the scenario]

[0364] 901, the user triggers the collection and detection module of the electronic device 100 to collect life scene information.

[0365] It should be understood that step 901 may have different occurrence times and implementation methods. Optionally, for household devices with relatively fixed locations such as smart screens, the smart screen can be triggered to start collecting information about life scenes when the user uses and turns on the smart screen for the first time, and the scene in which the smart screen is located may not change for a long time. Alternatively, the collection and detection module of the electronic device 100 can periodically collect life scene information according to a certain period, which is not limited in the embodiments of the present application.

[0366] It should also be understood that the electronic device 100 in the method 900 may also be configured in accordance with Figure 4 The process introduced in the above is to enable the personalized recommendation of video clips of the electronic device 100 (such as a smart screen) through the setting menu of the electronic device 100 (such as a smart screen), and to enable the function of "allowing collection of life scene information", or to enable the above functions through other quick operations or preset gestures. For the sake of simplicity, the process will not be repeated here.

[0367] Optionally, the acquisition and detection module of the electronic device 100 may include one or more devices such as a camera. For example, taking a smart screen as an example, the acquisition and detection module may be a camera of the smart screen, and the smart screen may obtain current life scene pictures through the camera.

[0368] Optionally, after the collection and detection module of the electronic device 100 collects the life scene pictures, the collected life scene pictures can be passed to the processor 110 of the electronic device 100, and then the processor 110 determines the current scene information. Alternatively, after the collection and detection module of the electronic device 100 collects the life scene pictures, the collected life scene pictures can be uploaded to the user service module or distribution module of the server 200, and the user service module or distribution module of the server 200 can further determine the current scene information. The following introduces these two possible methods respectively.

[0369] Method 1

[0370] 902, the collection and detection module of the electronic device 100 collects life scene pictures, and the processor 110 of the electronic device 100 determines the current scene information based on the collected life scene pictures.

[0371] 903, the collection and detection module of the electronic device 100 returns the current scene information to the video application.

[0372] It should be understood that with the improvement of the data processing capability and computing power of the electronic device 100, and the enrichment of algorithms such as image detection and recognition, the electronic device 100 can independently realize scene recognition, thereby reducing the interaction process between the electronic device 100 and the server 200 and accelerating the rate of scene recognition.

[0373] Method 2

[0374] 904, the collection and detection module of the electronic device 100 collects life scene information.

[0375] Exemplarily, the “life scene information” may be obtained by acquiring life scene pictures. In other words, the electronic device 100 may collect life scene pictures through a camera.

[0376] 905, the collection and detection module of the electronic device 100 reports the collected life scene information to the user service module of the server 200 through the communication module.

[0377] Exemplarily, Table 8 lists a possible parameter content that may be included in the life scene information. Exemplarily, as shown in Table 8, if the life scene information sent by the electronic device 100 is in the form of a life scene picture, the life scene information may include parameter name (image), device ID (device ID) and other information. Among them, the life scene picture can be a picture file compressed to a resolution of 480p, which is not limited in the embodiment of the present application.

[0378] Table 8

[0379]

[0380] 906. The user service module of the server 200 further determines the scene in which the electronic device 100 is currently located based on the life scene pictures collected by the electronic device 100, and encrypts and stores the scene data in which the electronic device 100 is currently located, that is, encrypts and stores the recognition result of the scene.

[0381] It should be understood that the user service module of the server 200 can receive life scene pictures reported by the electronic device 100, and determine the scene corresponding to the life scene picture based on picture detection and recognition algorithms.

[0382] For example, in the above-mentioned method 1 and method 2, when the electronic device 100 or the server 200 detects the appearance of objects such as cabinets, stoves, ovens, refrigerators, and range hoods in the life scene picture, it can be determined that the current scene is a kitchen scene; when objects such as beds and wardrobes are detected in the life scene picture, it can be determined that the current scene is a bedroom scene; when objects such as bookcases and desks are detected in the life scene picture, it can be determined that the current scene is a study scene. Examples will not be given one by one this time.

[0383] In one possible implementation, the server 200 may store a large number of life scene pictures of different categories. The server 200 compares the currently uploaded life scene picture with the large number of life scene pictures in the database. The scenes with high similarity are of the same category. In addition, the server 200 may also store the currently uploaded life scene picture in a database of life scenes of the same category to enrich the content of the database. After receiving a new life scene picture reported by the electronic device 100, it may quickly compare the large number of life scene pictures already stored in the database to quickly determine the current scene, thereby increasing the rate of scene recognition and reducing the amount of data processing. In addition, the accuracy and robustness of scene recognition may also be improved.

[0384] Exemplarily, Table 9 lists a possible parameter content that may be included in the recognition result of the scene. Exemplarily, as shown in Table 9, the server 200 determines the recognition result of the scene based on the life scene picture. The recognition result of the scene may include information such as parameter name (image), device ID (device ID), etc. Among them, the life scene picture can be a picture file compressed to a resolution of 480p, which is not limited in the embodiment of the present application.

[0385] Table 9

[0386] Parameter name Parameter Type M / O Parameter length Parameter Description scenarioInfos List <scenarioinfo> < / scenarioinfo> M / Scene recognition results

[0387] Among them, the parameter "ScenarioInfo" corresponding to the recognition result of the scene in Table 9 may further include more core parameters, and Table 10 lists a possible core parameter content of ScenarioInfo. Exemplarily, as shown in Table 10, "ScenarioInfo" may include recognizing that the scene type described by the electronic device 100 includes any one of a living room, a kitchen, a balcony, and a study.

[0388] Table 10

[0389]

[0390] 907, the user service module of the server 200 sends the recognition result of the scene to the electronic device 100 (or the video application of the electronic device 100) through the communication module. Specifically, the user service module of the server 200 can send the parameter contents of Tables 9 and 10 listed above to the electronic device 100, and the electronic device 100 can determine the current scene according to the parameter contents of Tables 9 and 10.

[0391] 908. A window automatically pops up on the electronic device 100 to remind the user of the current scene of the electronic device.

[0392] 909, the user can perform an operation in the pop-up window to allow recommendation of video content according to the current scene. In response to the allowing operation, the electronic device 100 closes the pop-up window.

[0393] Optionally, the pop-up window on the electronic device 100 may automatically disappear after receiving the user's permission operation; or, if the user does not perform the permission operation within a preset time period, the pop-up window may actively disappear after being displayed for a preset time period (for example, 5 seconds), and it is assumed that the current scene is correctly identified, and video content can be requested according to the current scene. This embodiment of the present application is not limited to this.

[0394] Optionally, the electronic device 100 may store the result of scene recognition in encrypted form, for example, by associating the current scene information with the current electronic device 100 and storing the result in encrypted form.

[0395] Exemplarily, the smart screen in the kitchen saves kitchen scene information, and the smart screen in the study saves study scene information. When the smart screen in the kitchen or the smart screen in the study requests video content from server 200, server 200 can recommend different video content for the smart screen in the kitchen or the smart screen in the study. The subsequent process will be introduced in detail.

[0396] Through the above process, each electronic device 100 can determine the current scene of the electronic device. When the user requests to obtain video content through the video application of the electronic device 100, the process corresponding to the "video content recommendation stage for users" can continue to be executed.

[0397] Fig.10 This is another example of an interface diagram for recommending video clips to users on a smart screen provided in an embodiment of the present application.

[0398] For example, taking the smart screen as an example, when the smart screen 100 determines the current scene according to the process of steps 901-903 or steps 904-907 described above, a pop-up window such as Fig.10 The prompt window 50 shown in Figure (a) can display prompt information to tell the user the currently recognized scene and that a video or video clip will be matched for the user based on the currently recognized scene.

[0399] For example, when the current scene is identified as a kitchen scene, the smart screen can select the "Kitchen" option in the prompt window 50 and display a prompt message: Dear user, your life scene is identified as follows (kitchen scene), and video clips of related scenes will be recommended for you to experience.

[0400] The user can choose whether to receive the recognition result of the user identity in the prompt window 50 according to his own needs. Fig.10 In the operation shown in Figure (a), click the "OK" option in the prompt window 50. In response to the user's click operation, the prompt window 50 disappears and the smart screen displays as shown in Figure (a). Fig.10 The interface 1002 shown in Figure (b) further recommends related videos matching the kitchen scene to the user based on the current kitchen scene, such as Food Video 1, Food Video 2, and Food Video 3 recommended on the interface 1002.

[0401] It should be understood that in the embodiment of the present application, if the electronic device 100 has a touch screen, the user's operation on the electronic device 100 can be to directly use a finger to perform click, double-click, long press and other operations. If the electronic device 100 does not have a touch screen, the user can perform corresponding operations through a stylus, a remote control, etc., and the embodiment of the present application is not limited to this. For example, if the electronic device 100 is a mobile phone, a car-mounted device, etc., the user can directly use a finger to perform click, double-click, long press and other operations. If the electronic device 100 is a smart screen, the user can select a certain video through the remote control and trigger the smart screen to start playing the video. The subsequent embodiments will not be repeated.

[0402] In a possible implementation, when the smart screen recognizes and selects the "kitchen" option in the prompt window 50, but the user does not expect to obtain videos related to the kitchen scene according to the kitchen scene, the user can modify the recognition result in the prompt window 50. For example, the user selects the "living room" option, clicks the check box before the "kitchen" option to cancel the "kitchen" option, and then clicks the "OK" option. In response to the user's click operation, the prompt window 50 disappears, and the smart screen displays the relevant videos matching the living room scene that are re-recommended for the user on the interface 1002, which will not be repeated here.

[0403] Optionally, the prompt window 50 may automatically disappear after the user clicks the "OK" option; or, if the user does not click the "OK" option within a preset time period, the prompt window 50 may actively disappear after displaying for a preset time period (e.g., 5 seconds), and it is assumed that the current kitchen scene is correctly identified, and related videos matching the kitchen scene may be recommended to the user according to the current kitchen scene. This embodiment of the present application is not limited to this.

[0404] Through the process introduced above, the electronic device 100 can accurately determine the current scene, such as the kitchen scene, living room scene, driving scene, study scene, etc., and subsequently recommend relevant videos matching the current scene to the user, thereby providing different video services for different scenes to meet the user's usage needs in different scenes.

[0405] [Phase 2: Recommending video content to users]

[0406] 910, the user runs a video application.

[0407] 911, the electronic device 100 sends a request for obtaining video content to the server 200, where the request for obtaining video content carries scene information.

[0408] It should be understood that for large-screen devices such as smart screens, step 910 may not be included, and the process of step 911 may be triggered when the user starts the smart screen; or, the process includes step 910. For example, different video applications such as Huawei Video and Youku Video may be installed on the smart screen. When the user clicks the Huawei Video icon to run the Huawei Video application, step 911 may be triggered to send a request for obtaining video content to the server 200 corresponding to Huawei Video. The embodiments of the present application are not limited to this.

[0409] Optionally, the request for obtaining video content may be used to request obtaining information of one or more videos, and the electronic device 100 may display one or more video lists on the interface of the electronic device 100 according to the information of the one or more videos.

[0410] Exemplarily, Table 11 lists a possible parameter content that may be included in the request for obtaining video content. Exemplarily, as shown in Table 11, the request for obtaining video content sent by the electronic device 100 may include parameters such as scene information (image), video operation column ID (category ID) and the playback address of the video content (mv ID). Optionally, when the request for obtaining video content may also include other scene parameters for indicating the current scene information, the embodiment of the present application does not limit the parameters or content included in the request for obtaining video content.

[0411] Table 11

[0412]

[0413] Exemplary, combined Fig.10 In Figure (a), the scene information (image) can be used to determine that the current scene is a kitchen scene; the video operation column ID (category ID) can be used to determine the content of the interface 1002 corresponding to the "Home" menu, or to determine the interface content corresponding to the "Member Video" menu, "Daily Recommendation" menu, "TV Series" menu, etc.; the video content ID (mv ID) can be used to determine the playback address of any video (such as Food Video 2) on the interface 1002 corresponding to the "Member Video" menu, which will not be repeated here.

[0414] 912 , the content service module of the server 200 determines the current scene according to the request for obtaining the video content, and requests the distribution module of the server 200 to query the tag information of the video.

[0415] 913, the distribution module of the server 200 obtains the current scene information, and determines the information of one or more videos and the target segment of each video according to the scene information.

[0416] 914. The distribution module of the server 200 sends a response message to the electronic device 100 (or the video application of the electronic device 100). The response message includes information of one or more videos and target segment information of each video.

[0417] It should be understood that method 900 may include the process of the video preprocessing stage introduced in method 500. In other words, the massive videos stored in the video content library of server 200 in method 900 are also processed through frame-by-frame analysis, etc. The content service module, distribution module, etc. of server 200 store the label information of each video, as well as the label information of multiple segments included in each video, etc., which will not be repeated here.

[0418] Optionally, “querying the tag information of the video” here may include different implementation processes.

[0419] In a possible implementation, the content service module may request the distribution module to query the global tag of the video, and the global tag may be used to identify the category information of different long videos, for example, the operator uploads the first video, the second video, and the third video to the server 200, and each video may correspond to a different global tag, for example, the first video belongs to the food category, the second video belongs to the emotion category, and the third video belongs to the family category. When the content service module can first request to query the global tag of the video, find the first video of the food category that matches the current kitchen scene, and determine that the first video is a video that can be recommended to the user.

[0420] In another possible implementation, the request for obtaining the video content in step 911 may also include user identity information.

[0421] For example, in step 912 and step 913, the content service module may further request the distribution module to query the tag information of each video, that is, according to Figure 5In the process of step 516 in , the tags of the multiple video segments included in each video are queried, and the target segments that can be played in each video are determined for the user according to the user identity. Specifically, the content service module can first determine that the first video is a video that can be recommended to the user, and further query the tags of each video segment included in the first video, and determine the target segments in the first video that can be played for the user. If the current user is a child user, the content service module also needs to determine the video segments that need to be filtered out or shielded for the child user among the multiple video segments included in the first video, or in other words, the content service module also needs to determine the target segments that can be played for the child user among the multiple video segments included in the first video, and then determine the response message, which is not limited to this embodiment of the present application.

[0422] Optionally, after the distribution module of the server 200 determines the information of one or more videos and the target segment of the multiple video segments included in each video, the response message may be returned to the content service module, and the response message may include the information of the one or more videos, the target segment information of each video, and the tag information of each video or video segment, etc. The content service module may also generate metadata of the one or more videos, or assemble metadata of the target segment of each video, and send it to the electronic device 100.

[0423] Exemplarily, Table 12 lists a possible parameter content that the response message may include. Exemplarily, as shown in Table 12, the response message sent by the server 200 to the electronic device 100 may be identified by the parameter "mvInfos", which means that the response message may include information of the one or more videos, target segment information of each video; and the parameter "tagInfos" may indicate tag information for identifying each video or video segment, etc.

[0424] Table 12

[0425]

[0426] Among them, the parameter "mvInfos" in Table 12 can further include more core parameters, and Table 13 lists a possible core parameter content of mvInfos. It should be understood that if the current scene is identified as a kitchen scene, for the kitchen scene, the information of the electronic device 100 controlling the playback progress of the first video can be identified by the parameter "tagTimeInfos".

[0427] Table 13

[0428]

[0429] The parameter "tagTimeInfos" corresponding to the information for controlling the playback progress of the first video in the kitchen scene in Table 13 may further include more core parameters. Table 14 lists a possible core parameter content of tagTimeInfos. It can be seen from Table 14 that the parameter "tagTimeInfos" corresponding to the information for controlling the playback progress of the first video may further include the playback start time and playback end time corresponding to the filtered target segment that can match the kitchen scene. Exemplarily, combining the parameter content in Table 14 and Fig.11 In Figure (c), the start time (starttime) corresponding to the playback progress and the end time (end time) corresponding to the playback progress can be used to determine the video segment with a duration t indicated by the gray area of ​​the progress bar, which will not be described in detail this time.

[0430] Table 14

[0431] Parameter name Parameter Type M / O Parameter length Parameter Description start time string M / The start time corresponding to the playback progress end time string M / End time corresponding to the playback progress tagName string M / Tag Name

[0432] Through the above process, the distribution module of the server 200 sends the response messages listed in Tables 12 to 14 above to the video application of the electronic device 100. The video application of the electronic device 100 can determine the information of the one or more videos and the target segment information of each video based on the response message, accurately display the recommended video list to the user, and when the user plays the first video, it can play the target segment of the first video in combination with the user identity.

[0433] 915 , the electronic device 100 displays a list of one or more recommended videos according to the information of the one or more videos and the target segment information of each video.

[0434] 916, the user performs an operation of playing the first video.

[0435] Fig.11 This is another example of an interface diagram for recommending video clips to users on a smart screen provided in an embodiment of the present application.

[0436] For example, through the above-described process, the server 200 recommends relevant videos matching the kitchen scene to the user according to the current kitchen scene, and the smart screen can display the following video clips: Fig.11 As shown in (a) in the figure, recommended food video 1, food video 2 and food video 3 are displayed on the interface 1101.

[0437] When the user wants to play Food Video 2, he can select and click the icon of Food Video 2 (first video). In response to the user's play operation, Food Video 2 starts to play on the smart screen.

[0438] 917, the smart screen may display the filtered partial segment of the first video. Optionally, the smart screen may also display the target segment of the first video to be played for the user.

[0439] 918. The user performs an operation to allow the smart screen to filter a portion of the first video according to the current scene.

[0440] 919. The smart screen starts playing the target segment of the first video based on the user's selection.

[0441] For example, Fig.11 As shown in Figure (b), before the smart screen starts playing the food video 2, a prompt window 60 can be automatically displayed. The prompt window 60 can show the user the multiple segments included in the food video 2, and display the segments that will be filtered during the playing of the food video 2 and / or the target segments that will be played for the user.

[0442] For example, Fig.11 As shown in FIG. 1 (b), the prompt window 60 displayed on the interface 1102 includes segment 1, segment 2 and segment 3. Among them, segment 2 is in a selected state, and segment 1 and segment 3 are in an unselected state, that is, segment 2 is the segment to be filtered in the food video 2, and segment 1 and segment 3 are the target segments to be played for the user.

[0443] If the user clicks the "OK" option in the prompt window 60, in response to the user operation, the smart screen can display the following Fig.11 The interface 1103 shown in Figure (c) is the playback interface of the food video 2, and the smart screen starts to play the food video 2 for the user. In addition, the playback progress bar can also be displayed on the interface 1103. The black area on the progress bar is the part that has been played, the white area is the part that has not been played, and the gray area is the part that cannot be played. Fig.11 In (b) of FIG. 1 , segment 2 is filtered out, and the corresponding playing time of segment 2 is t.

[0444] As the playback process of the food video 2 continues, when the playback progress of the food video 2 continues to reach a critical point, the Fig.11 In the interface 1104 shown in FIG. (d) of FIG. 1 , a prompt window 70 is automatically displayed on the smart screen, and the prompt window 70 may display: Dear user, this video segment is automatically skipped for you in the current scene. In addition, the smart screen may automatically skip the segment 2 with a duration t corresponding to the progress bar in the gray area, and continue to play the content of segment 3.

[0445] Optionally, the prompt window 70 automatically disappears after being displayed for a preset time (e.g., 5 seconds); or, if the user can click on any area of ​​the prompt window 70, the prompt window 70 disappears in response to the user's click operation. This embodiment of the present application is not limited to this.

[0446] In another possible scenario, users may want to watch video clips related to their favorite food, e.g. Fig.11 In the prompt window 60 shown in FIG. (b), the filter segment automatically selected by the smart screen is segment 2, leaving only segments 1 and 3 related to food. At this time, the user can modify and adjust the content of the filtered segment according to his or her needs.

[0447] For example, if the user does not want to watch segment 1 related to making pizza, the user can select the option corresponding to segment 1 and then click the "OK" option. In response to the user's click operation, during the subsequent playback of food video 2, segments 1 and 2 can be automatically filtered for the user, and only segment 3 can be played for the user.

[0448] Optionally, the user can also select the option corresponding to segment 2 to cancel filtering of segment 2. During the subsequent playback of food video 2, segment 1 can be automatically filtered for the user, and segments 2 and 3 can be played for the user. This will not be repeated here.

[0449] The above method identifies the life scene in which the electronic device is located, and recommends different video content to the user from the video content library according to the scene identification result. For example, for the kitchen scene, it can recommend videos related to food and food making to the user; for the study scene, it can recommend videos related to teaching to the user; for the living room scene, it can recommend movies and TV series suitable for family members to watch together to the user; for the balcony scene, it can recommend videos related to home and cleaning to the user. This method can recommend videos that match the current scene for the user to choose from, thereby improving the user experience.

[0450] Furthermore, in addition to combining the scene recognition results, the method can also combine the account of the user logged into the electronic device, the historical browsing records corresponding to the user account and other information to draw a behavioral characteristic portrait of the user. After completing the behavioral characteristic portrait of the user, the tag information of each video is combined from the video content library to match one or more videos for the user. This method can accurately recommend video content that matches the current scene, user habits and hobbies to the user, thereby improving the user's viewing experience.

[0451] It should be understood that in the various embodiments of the present application, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0452] It should also be understood that in the embodiments of the present application, "pre-setting", "pre-definition", etc. can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in an electronic device (such as a mobile phone or smart screen, etc.). This application does not limit its specific implementation method, such as the preset duration in the embodiments of the present application.

[0453] It should also be understood that the processes of the above-mentioned method 500 and method 900 can be executed simultaneously, or only method 500 can be executed in a certain scenario, or only method 900 can be executed in a certain scenario, and the embodiments of the present application are not limited to this.

[0454] It should also be understood that the method provided in the embodiment of the present application can be used to recommend personalized video scenarios to users, and can also be used to recommend personalized music scenarios to users, recommend personalized theme pictures scenarios, etc., and the embodiment of the present application is not limited to this.

[0455] Optionally, when method 500 and method 900 exist at the same time, the server can identify the life scene in which the electronic device is located, as well as the identity information of the current user, and then combine the tag information of each video clip and the user's preferences, behavioral habits, etc. to provide the current user with refined personalized videos, or match the target clip in a video for the current user.

[0456] If the user is a child or a minor, etc., a video suitable for a child or a minor can be found from multiple videos matched according to life scenarios, and the video can be recommended to the user. Specifically, if the current user is identified as a child or a minor, the method can also control the electronic device to switch to child mode, and control the scope of permissions for the video that can be played by the electronic device in child mode. If the child or minor plays a selected video, the video provided to the electronic device by the server can also be a filtered video, that is, the video clips in the video that are not suitable for child users to watch are blocked. This process does not require manual setting of the electronic device to switch to child mode, and can achieve the effect of finely controlling video playback in child mode, thereby improving the user experience.

[0457] In summary, the method for recommending video clips provided in the embodiment of the present application can, first of all, enable the server to process massive videos. Specifically, the server can analyze each video frame by frame, and perform one or more dimensions of processing such as target detection, content recognition, content understanding, and natural language understanding on each frame of each video, so as to determine the label of each frame. Then, based on a deep learning algorithm, etc., the multiple frames of the video are aggregated into multiple clips in combination with the label information of each frame, and the label information of each video and the label information of the aggregated multiple clips are automatically extracted and stored.

[0458] Secondly, in this method, the electronic device can collect and identify the identity of the user, or report the user identity to the server. For example, the server can receive the user feature data reported by the electronic device, and combine the stored massive user feature data to create a user profile for the current user, quickly determine the identity of the current user, increase the rate of user identity recognition, and improve the accuracy of user identity recognition. After determining the user's identity, the server can combine the tag information of the video in the video content library of the server to recommend videos that match the user's identity to the user.

[0459] Furthermore, when a user selects and plays a certain video, the method can also query the tag information of each video clip included in the video, and match the user with clips that meet the user's viewing needs based on the current user identity. For example, when it is identified that the current user is a child user or a minor user, the method can also control the electronic device to switch to child mode, and control the range of movie permissions that can be played by the electronic device in child mode. If the child user or minor user plays a selected video, the video provided to the electronic device by the server can also be a filtered video, that is, the video clip content in the video that is not suitable for child users to watch is blocked. This process does not require manual setting of the electronic device to switch to child mode, and can achieve refined control of the video playback effect in child mode, thereby improving the user experience.

[0460] Finally, the method can also be combined with the life scene in which the electronic device is located, and according to the recognition results of the scene, different video content can be recommended to the user from the video content library. The above method can be used for scenarios where multiple people in the family use multiple large-screen devices such as smart screens, or when multiple family members use the same smart screen or other device, or when different users use large-screen devices such as smart screens in split screens. The process can intelligently identify the current life scene of the electronic device, the identity of the current user, the account of the user logged in to the electronic device, the historical browsing records corresponding to the user account, and other information, and recommend one or more videos to the user from the video content library in combination with the tag information of each video. This method can accurately recommend video content that suits the current scene, user habits and hobbies to users, thereby improving the user's viewing experience.

[0461] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0462] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0463] In the case of dividing each functional module according to each function, the electronic device 100 involved in the above embodiment may also include: a display unit, a detection unit and a processing unit. The display unit, the detection unit and the processing unit cooperate with each other to support the electronic device to perform the above steps and / or other processes of the technology described herein.

[0464] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0465] The electronic device provided in this embodiment is used to execute the above-mentioned video playback method, and thus can achieve the same effect as the above-mentioned implementation method.

[0466] In the case of using integrated units, the electronic device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the electronic device, for example, it can be used to support the electronic device to execute the steps performed by the above-mentioned display unit, detection unit and processing unit. The storage module can be used to support the electronic device to execute stored program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0467] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0468] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device 100 involved in this embodiment can be a Figure 2 Device of the structure shown.

[0469] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method of personalized recommendation of video clips in the above-mentioned embodiment.

[0470] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the method for personalized recommendation of video clips in the above-mentioned embodiment.

[0471] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the method of personalized recommendation of video clips in the above-mentioned method embodiments.

[0472] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0473] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0474] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0475] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0476] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0477] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0478] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for recommending video clips, characterized in that: The method comprises: The electronic device detects an operation of a user running a video application, and in response to the operation, collects user features; determines the identity of the user according to the user features; or the electronic device collects user features, sends the user features to a server, and receives a recognition result of the user features sent by the server to determine the identity of the user; wherein the user features include one or more of facial features, age features, height features, clothing features, and occupation features; The electronic device collects current scene features and determines current scene information according to the scene features; or The electronic device collects the current scene features, sends the current scene features to the server, receives the recognition result of the scene features sent by the server, and determines the current scene information; Wherein, when the scene feature is a scene picture, the scene information corresponding to the scene feature is determined according to a picture detection and recognition algorithm; According to the user's identity, the electronic device switches the working mode; The electronic device displays a video list, wherein the video list includes one or more videos; The electronic device receives an operation of playing a first video by a user, and in response to the operation, the electronic device sends a request for playing the first video to a server, wherein the request for playing the first video includes first identification information, and the first video is any one of the one or more videos; the first identification information includes scene information where the electronic device is currently located and identity information of the user; The electronic device receives a response message sent by the server, the response message is used to indicate a target segment and a filtered segment of the first video, the target segment is associated with the first identification information, the target segment is a segment used to be played to the user, and the filtered segment is a segment not used to be played to the user; wherein the server determines information of one or more videos and a target segment of each video according to the scene information and the identity information of the user; The electronic device plays the target segment of the first video according to the response message.

2. The method according to claim 1, characterized in that The first identification information includes the identity information of the user and / or the scene information where the electronic device is currently located.

3. The method according to claim 1, characterized in that The response message further includes information of a video list, the information of the video list includes information of at least one second video, the second video matches the first identification information, and the method further includes: The electronic device updates the video list according to the information of the video list and displays the at least one second video in the video list.

4. The method according to claim 1, characterized in that: The electronic device collecting user characteristics further includes: And / or the electronic device periodically collects user characteristics.

5. The method according to claim 1, characterized in that: After the electronic device determines the identity of the user, the method further includes: The electronic device displays a first window, wherein the first window displays first prompt information, wherein the first prompt information is used to prompt a user that the electronic device can recommend a video to the user according to the identity of the user.

6. The method according to any one of claims 1 to 5, characterized in that The electronic device plays the target segment of the first video according to the response message, including: On the playback interface of the first video, a playback progress bar is displayed, the playback progress bar including a first area corresponding to the target segment and a second area corresponding to the filtered segment, and the second area of ​​the playback progress bar is displayed in grayscale; or A playback progress bar is displayed on the playback interface of the first video, and the playback progress bar only includes a first area corresponding to the target segment, and does not include a second area corresponding to the filtered segment.

7. The method according to claim 6, characterized in that When the playback progress bar includes the first area and the second area, during the process of playing the target segment of the first video, when the current playback time displayed on the playback progress bar is located at the starting position of the second area, the electronic device displays a second window, and the second window displays second prompt information, and the second prompt information is used to prompt the user that the electronic device will skip the filtered segment and continue to play the target segment of the first video.

8. The method according to any one of claims 1 to 5, characterized in that: The playback request of the first video also includes the account information of the electronic device logged in, and the target segment is a segment in the first video that matches the historical playback record corresponding to the first identification information and the account of the electronic device logged in.

9. The method according to any one of claims 1 to 5, characterized in that The response message also includes metadata of the target segment, and the metadata of the target segment includes one or more of a playback address, image data, audio data, and text data of the target segment.

10. A method for recommending video clips, characterized in that: Applied to a server, the server stores one or more videos and tag information of each of the one or more videos, the method comprising: The server receives user features sent by the electronic device, identifies the identity of the user according to the user features, and sends the identification result of the user identity to the electronic device, wherein the user features include one or more of facial features, age features, height features, clothing features, and occupational features; the electronic device detects the operation of the user running the video application, and collects the user features in response to the operation; the electronic device sends the user features to the server, receives the identification result of the user features sent by the server, and determines the identity of the user; the electronic device switches the working mode according to the user identity; The server receives a play request of a first video sent by an electronic device, the play request of the first video includes first identification information, the first video is any one of the one or more videos, and the first video includes one or more clips; the first identification information includes identity information of the user and scene information where the electronic device is currently located; The server receives the scene features sent by the electronic device, identifies the current scene according to the scene features, and sends the scene recognition result to the electronic device; The server queries the tag information of the first video according to the playback request of the first video, where the tag information of the first video includes the tag information of each of the one or more segments; The server determines a target segment and a filter segment of the first video according to the first identification information and tag information of each of the one or more segments, wherein the target segment is associated with the first identification information, the target segment is a segment used to be played to the user, and the filter segment is a segment not used to be played to the user; wherein the server determines information of the one or more videos and the target segment of each video according to scene information and identity information of the user; The server sends a response message to the electronic device, where the response message is used to indicate the target segment and the filtered segment.

11. The method according to claim 10, characterized in that The method further comprises: The server determines, from the one or more videos, a video list matching the first identification information according to the first identification information and the tag information of each video in the one or more videos, wherein the video list includes at least one second video; The server sends the information of the video list to the electronic device, where the information of the video list includes one or more of a playback address, image data, audio data, and text data of the at least one second video.

12. The method according to claim 10, characterized in that The method further comprises: The server obtains the first video and determines a plurality of frames included in the first video; The server detects each frame of the multiple frames included in the first video, and identifies the content included in each frame; The server determines the tag information of each frame of the multiple frames according to the content included in each frame of the frames; The server divides the first video into one or more segments according to the tag information of each frame; The server determines the tag information of each of the one or more segments and the tag information of the first video according to the tag information of each frame.

13. The method according to claim 12, characterized in that The server divides the first video into one or more segments according to the tag information of each frame, including: The server divides the first video into the one or more segments according to similarities between tag information of adjacent frames.

14. The method according to claim 12, characterized in that The server determines the tag information of each of the one or more segments and the tag information of the first video according to the tag information of each frame, including: The server determines, according to the tag information of each frame, the tag information having the largest number of repetitions in the multiple frames included in each segment as the tag information of the segment; and The server determines, according to the tag information of each segment, tag information with the largest number of repetitions among the tag information of the one or more segments as tag information of the first video.

15. The method according to any one of claims 10 to 14, characterized in that The playback request of the first video also includes account information logged in by the electronic device, and the server determines the target segment and the filter segment of the first video according to the first identification information and the tag information of each segment of the one or more segments, including: The server obtains the historical playback record of the electronic device according to the account information logged in by the electronic device; The server determines the target segment according to the first identification information, the historical playback record of the electronic device and the tag information of each of the one or more segments.

16. The method according to any one of claims 10 to 14, characterized in that The response message also includes metadata of the target segment, and the metadata of the target segment includes one or more of a playback address, image data, audio data, and text data of the target segment.

17. An electronic device, characterized in that: include: Display screen; one or more processors; one or more memories; Modules with multiple applications installed; The memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes the method according to any one of claims 1 to 9.

18. A server, characterized in that: include: one or more processors; one or more memories; The memory stores one or more programs, and when the one or more programs are executed by the processor, the server executes the method according to any one of claims 10 to 16.

19. A system for recommending video clips, characterized in that: The system comprises an electronic device and a server, wherein the electronic device can execute the method according to any one of claims 1 to 9, and the server can execute the method according to any one of claims 10 to 16.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 9 or the method according to any one of claims 10 to 16.

21. A computer program product, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 9 or the method according to any one of claims 10 to 16.

Images