Video processing method, device, electronic device and readable storage medium

Abstract

The present application provides a video processing method, device, electronic device and readable storage medium, which belongs to the field of video processing technology. The method includes: obtaining a first video file and a target file; calculating the similarity between each first element in the first video file and each second element in the target file; and updating the first video file according to the similarity.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and in particular to a video processing method, device, electronic device, and readable storage medium. Background Art

[0002] With the continuous development of electronic devices, various video playback software and video portal websites continue to emerge, and video playback has become an indispensable function for users to use electronic devices. At present, when playing the same video file, the played video content is fixed, which shows that the flexibility of existing video playback is low. Summary of the invention

[0003] The embodiments of the present application provide a video processing method, device, electronic device and readable storage medium, which can solve the problem of low flexibility of existing video playback.

[0004] In a first aspect, an embodiment of the present application provides a video processing method, including:

[0005] Obtain a first video file and a target file;

[0006] Calculating the similarity between each first element in the first video file and each second element in the target file;

[0007] The first video file is updated according to the similarity.

[0008] In a second aspect, an embodiment of the present application further provides a video processing device, including:

[0009] An acquisition module, used for acquiring a first video file and a target file;

[0010] A calculation module, used for calculating the similarity between each first element in the first video file and each second element in the target file;

[0011] An updating module is used to update the first video file according to the similarity.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the video processing method as described in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application further provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the video processing method as described in the first aspect is implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0015] In an embodiment of the present application, a first video file and a target file are obtained; the similarity between each first element in the first video file and each second element in the target file is calculated; based on the similarity, the first video file is updated to adjust the playback content of the first video file, thereby improving the flexibility of video playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 is a flow chart of a video processing method provided by an embodiment of the present application;

[0018] Figure 2a It is one of the schematic diagrams of the video screen provided in the embodiment of the present application;

[0019] Figure 2b This is the second schematic diagram of the video screen provided in the embodiment of the present application;

[0020] Figure 3 This is the third schematic diagram of the video screen provided in the embodiment of the present application;

[0021] Figure 4 This is the fourth schematic diagram of the video screen provided in the embodiment of the present application;

[0022] Figure 5 is a structural diagram of a video processing device provided in an embodiment of the present application;

[0023] Figure 6 It is one of the structural diagrams of the electronic device provided in the embodiment of the present application;

[0024] Figure 7 This is the second structural diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the second object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The video processing method provided in the embodiment of the present application is described in detail below through some embodiments and their application scenarios in combination with the accompanying drawings.

[0028] The video processing method of the embodiment of the present application can be performed by an electronic device. In practical applications, the electronic device can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer) or a notebook computer, a personal digital assistant (Personal Digital Assistant, PDA), a handheld computer, a netbook, an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device (Wearable Device), a vehicle-mounted device (VUE), a pedestrian terminal (PUE), a smart home (home appliances with wireless communication functions, such as refrigerators, televisions, washing machines or furniture, etc.) and other terminal-side devices.

[0029] See also Figure 1 , Figure 1 is a flow chart of the video processing method provided by the embodiment of the present application. Figure 1 As shown, the video processing method may include the following steps:

[0030] Step 101: Obtain a first video file and a target file.

[0031] The target file may be: a picture file, a video file or an audio file. When the target file is a video file, in order to distinguish the video file from the first video file, the video file is referred to as a second video file.

[0032] The embodiments of the present application do not limit the method for obtaining the first video file and the target file. For example, in some embodiments, the user can trigger the electronic device to obtain the first video file and the target file by touching the identifier of the first video file and the identifier of the target file.

[0033] In some other embodiments, obtaining the first video file and the target file may include:

[0034] Receiving a first input of a video frame of a first video file;

[0035] In response to the first input, the first video file is acquired, and a first page is displayed, wherein the first page includes N file identifiers corresponding to the N files one by one, where N is a positive integer;

[0036] receiving a second input for a target file identifier among the N file identifiers;

[0037] In response to the second input, a target file corresponding to the target file identifier is obtained.

[0038] In the embodiment of the present application, the input received by the electronic device may be touch input, voice input, gesture input, etc., which may be specifically set according to actual conditions, and the embodiment of the present application does not limit this.

[0039] In an optional implementation, if Figure 2a As shown, a control 20 for triggering the electronic device to display the first page can be set on the video screen of the first video file, so that the user can select a target file through the first page. In this embodiment, the first input can be a touch input for the control.

[0040] In an optional implementation, the electronic device may simultaneously display the video image of the first video file and the first page, for example, displaying the video image of the first video file in a first area or a first screen, and displaying the first page in a second area or a second screen.

[0041] In another optional implementation, the electronic device may switch display from the video screen of the first video file to the first page in response to the first input.

[0042] The video screen may be a video playback page or a video pause screen. It should be noted that, when the video screen is a video playback interface, the first video file may be paused or kept playing during the display of the first page, which may be set according to actual conditions and is not limited in the present application.

[0043] The first page may be a file management page of the electronic device, including at least one file identification. In specific implementation, the first page may include at least one of the following: at least one video file identification, at least one audio file identification, and at least one picture file identification.

[0044] Step 102: Calculate the similarity between each first element in the first video file and each second element in the target file.

[0045] In a specific implementation, each first element may include part or all of the first elements in the first video file, and each second element may include part or all of the elements in the target file; the calculation of the similarity may be actively performed by the electronic device, or may be triggered by user input to execute the calculation, which may be determined according to actual needs and is not limited in the embodiments of the present application.

[0046] In the case where the calculation of the similarity is triggered by a user input to execute the electronic device, in some embodiments, the calculating of the similarity between each first element in the first video file and each second element in the target file may specifically include:

[0047] Displaying the target file or an identifier corresponding to the target file on the video screen of the first video file;

[0048] Receiving a third input of the video picture;

[0049] In response to the third input, similarities between each first element in the first video file and each second element in the target file are calculated.

[0050] In a specific implementation, when the target file is a picture file, Figure 2b As shown, the electronic device can directly display the picture file 21 on the video screen of the first video file. In the case where the target file is a second video file, as shown in FIG. Figure 3 As shown, the logo 31 corresponding to the second video file is displayed on the video screen of the first video file. In the case where the target file is an audio file, Figure 4 As shown, the logo 41 corresponding to the audio file is displayed on the video screen of the first video file.

[0051] In this embodiment, the user can trigger the electronic device to perform the calculation of the similarity by performing a third input on the video screen. Further, the third input can also be used to determine each first element and each second element.

[0052] In some optional implementations, the third input may include: an input of sliding from the first position to the second position. In specific implementations, for target files in different representations, the specific representations of the first position and the second position may be different, as illustrated by the following examples:

[0053] In the case that the target file is a picture file, the first position is the display position of a second element in the picture file, and the second position may be any position in the video screen except the display position of the picture file.

[0054] For easier understanding, see Figure 2b .exist Figure 2b In the example, the picture file 21 includes a first character element 211 and a second character element 212. The third input may be an input of sliding from the display position of the first character element 211 to the progress bar in the video screen. In this case, the electronic device may determine the first character element 211 as the second element in the picture file in response to the third input, and calculate the similarity between each first element in the first video file and the first character element 211.

[0055] Further, assuming that the third input also includes: an input of circling a target area of ​​the progress bar of the first video file. In this case, the electronic device can, in response to the third input, determine the first character element 211 as the second element in the picture file, determine the elements of the video frame corresponding to the target area in the first video file as the first elements, and calculate the similarity between the first elements and the first character element 211.

[0056] When the target file is a second video file or an audio file, the first position may be a display position of a first element in the first video file, and the second position may be a display position of an identifier corresponding to the target file.

[0057] For easier understanding, see Figure 3 and Figure 4 .

[0058] exist Figure 3In the example, the video screen displays the identifier 31 corresponding to the second video file and the third character element 32. The third input may be an input of sliding from the display position of the third character element 32 to the identifier 31 corresponding to the second video file. In this case, the electronic device may determine the third character element 32 as the first element in the first video file in response to the third input, and calculate the similarity between the first element and each second element in the second video file.

[0059] exist Figure 4 In the example, the video screen displays the identifier 41 corresponding to the audio file and the fourth character element 42. The third input may be an input of sliding from the display position of the fourth character element 42 to the identifier 4 corresponding to the audio file. In this case, the electronic device may, in response to the third input, determine the fourth character element 42 as the first element in the first video file, and calculate the similarity between the first element and each second element in the second video file.

[0060] Step 103: Update the first video file according to the similarity.

[0061] In specific implementation, the second element can be added to the first video file according to the similarity, the first element can be replaced by the second element, or the first element in the first video file that has a higher similarity with the second element can be marked. The specific details can be determined according to actual conditions, and the embodiments of the present application are not limited to this.

[0062] It should be noted that the updating and playing of the first video file can be performed in parallel, that is, the content of the video file can be updated during the playing of the video file, thereby improving the flexibility of video playing. In addition, in the current playing of the first video file, the unplayed part can be played according to the updated content or the unupdated content, that is, the updated first video file can take effect immediately after the update or after the current playing ends, which can be set according to actual needs, and the embodiments of the present application do not limit this.

[0063] The video processing method of this embodiment obtains a first video file and a target file; calculates the similarity between each first element in the first video file and each second element in the target file; updates the first video file based on the similarity to adjust the playback content of the first video file, thereby improving the flexibility of video playback.

[0064] In some embodiments, updating the first video file according to the similarity includes:

[0065] Determine at least one element association pair according to the similarity, each element association pair includes a first element and a second element, and the similarity between the first element and the second element in each element association pair is greater than a preset threshold;

[0066] Adding the target second element in the at least one element association pair in the first video frame of the first video file; or replacing the target first element in the at least one element association pair in the first video file with the target second element in the at least one element association pair.

[0067] In this embodiment, the electronic device may determine at least one element association pair according to the similarity, and update the first video file according to the at least one element association pair.

[0068] In a specific implementation, the first element and the second element whose similarity is greater than a preset threshold value may be determined as an element association pair, and the preset threshold value may be preset according to actual needs.

[0069] After determining at least one associated element pair, the first video file can be updated based on each associated element pair. For ease of understanding, the following is an explanation of updating the first video file based on one associated element pair, hereinafter referred to as the first associated element pair. It can be understood that for other associated element pairs, the electronic device can update the first video file based on them in the same manner, and no further description is given.

[0070] Assuming that the first element association pair includes a target first element and a target second element, updating the first video file according to the first element association pair may be specifically performed as follows:

[0071] i) adding the target second element to a target video frame in the first video file, wherein the target video frame includes the target first element; or

[0072] ii) replacing the target first element in the first video file with the target second element.

[0073] In i), the first video file can be updated by adding an element to the first video file. The user can view the target second element with a high correlation with the target first element in the target video frame, so that the target first element can be associated with the target second element, thereby improving the user's understanding of the target video frame and the target first element. It can be understood that the first video frame includes the target video frame.

[0074] Further, updating the first video file according to the first element association pair may also include: displaying a thumbnail of the first video frame on the video screen of the first video file. Figure 2b .exist Figure 2b In the example, the target video frame includes video frame 1, video frame 2 and video frame 3, and the electronic device can display thumbnails of video frame 1, video frame 2 and video frame 3 on the video screen. In this way, the user can know the video frame to which the target second element is added.

[0075] In an optional implementation manner, after displaying the thumbnail of the first video frame on the video screen of the first video file, the method further includes:

[0076] receiving a fourth input for a first thumbnail, where the first thumbnail is a thumbnail corresponding to any one of the target video frames;

[0077] In response to the fourth input, the progress bar of the first video file is controlled to jump to a first playback progress point, where the first playback progress point is a playback progress point of the video frame corresponding to the first thumbnail.

[0078] In this way, after knowing the video frame with the target second element added, the user can control the first video file to jump to the corresponding video frame to continue playing by touching the corresponding identification information, thereby improving the control flexibility of video playback.

[0079] Further, updating the first video file according to the first element association pair may also include: displaying a thumbnail of a second video frame on the video screen, wherein the second video frame includes the target second element. Figure 3 .like Figure 3 As shown, the electronic device can display thumbnails of video frames including the third character element 32 in the second video file.

[0080] In an optional implementation manner, the electronic device may directly display a thumbnail of the second video frame on the video screen. In another optional implementation manner, displaying a thumbnail of the second video frame on the video screen includes:

[0081] receiving a fifth input of an identifier corresponding to the second video file;

[0082] In response to the fifth input, a thumbnail of the second video frame is displayed on the video screen.

[0083] In this embodiment, the user can touch the mark corresponding to the second video file to trigger the electronic device to display a thumbnail of the second video frame on the video screen. In this way, the impact on the video screen of the first video file can be reduced while ensuring that the user can view the at least one video frame, thereby improving the flexibility of video playback.

[0084] In an optional implementation, in response to the fifth input, a thumbnail of the second video frame may be displayed in each video frame of the first video file.

[0085] In another implementation, in response to the fifth input, a thumbnail of the second video frame may be displayed on a third video frame of the first video file, the third video frame being the currently playing video frame of the first video file when the fifth input is received.

[0086] In ii), the first video file can be updated by replacing the element in the first video file. Furthermore, when the element is a sound element, updating the first video file according to the first element association pair may also include: replacing the subtitle information corresponding to the target first element with the text information corresponding to the target second element, so that the sound element of the first video file matches the subtitle information, thereby improving the reliability of video file playback.

[0087] In an optional implementation, the electronic device may perform the above replacement operation.

[0088] In another optional implementation manner, replacing the target first element in the first video file with the target second element includes:

[0089] receiving a sixth input, wherein the sixth input is: sliding from the display position of the target first element to the display position of the target file or the mark corresponding to the target file;

[0090] In response to the sixth input, the target first element in the first video file is replaced with the target second element.

[0091] For easier understanding, please continue to refer to Figure 4 .exist Figure 4 In the embodiment, upon detecting a sliding input in which the sliding start position is the display position of the fourth character element 42 and the sliding end position is the display position of the identifier 41 corresponding to the audio file, the electronic device may perform the above-mentioned replacement operation.

[0092] In this embodiment, the electronic device first determines each element association pair whose similarity is greater than a preset threshold, and updates the first video file according to each element association pair, such as adding the second element of each element association pair to the corresponding video frame of the first video file, or replacing the corresponding first element in the first video file with the second element in each element association pair. Since the first element and the second element of each element association pair are highly similar, updating the first video file according to each element association pair can improve the reliability of updating the content of the first video file.

[0093] The calculation of the similarity is described in detail below.

[0094] In some embodiments, the electronic device may determine the similarity between elements by using content recognition technology.

[0095] In some other embodiments, calculating the similarity between each first element in the first video file and each second element in the target file may include:

[0096] Determine first vectors corresponding one-to-one to first elements in the first video file, and second vectors corresponding one-to-one to second elements in the target file;

[0097] The similarity between each first vector and each second vector is calculated.

[0098] In this embodiment, the similarity between each first element and the second element can be calculated by determining the first vector corresponding to each first element and the second vector corresponding to each second element, and calculating the similarity between each first vector and each second vector, thereby realizing the calculation of the similarity between each first element and the second element, and improving the reliability of the element similarity calculation.

[0099] In the embodiment of the present application, the vector corresponding to each element can be determined in the following manner:

[0100] In some embodiments, determining the first vectors corresponding to the first elements in the first video file and the second vectors corresponding to the second elements in the target file may include:

[0101] 1) Obtaining each characteristic information of the first element or the second element;

[0102] 2) Obtaining each first score corresponding to each feature information, wherein the first score corresponding to each feature information is determined based on the matching score between the feature information and the historical operation node;

[0103] 3) Generate a vector corresponding to the first element or the second element according to each first score.

[0104] In this embodiment, by obtaining each feature information of the element, based on the matching score between each feature information and the historical operation node, the vector corresponding to the element is determined. In this way, by using the vector corresponding to the element to calculate the similarity between the elements, the elements with similar matching scores between the feature information and the historical operation node can be determined as elements with high similarity, thereby improving the reliability of the element similarity calculation.

[0105] For 1)

[0106] In specific implementation, the electronic device can identify each feature information of each first element and each second element (hereinafter referred to as m feature information) through content understanding technology. It can be understood that for different types of elements, the feature information of the elements is expressed in different forms, as described below:

[0107] In the case where the element is an image element, the m pieces of feature information may include at least one of the following: shape, color, scene, entity content, style, emotional expression, and video plot expression. For example, the m pieces of feature information of an image element may be: round, yellow, playground, basketball, bright, warm, and basketball game.

[0108] When the element is a sound element, the m pieces of feature information may include at least one of the following: voiceprint, frequency, pitch, phoneme, entity content, emotional expression, scene, etc.

[0109] In the case where the element is a subtitle element, the m pieces of feature information may include at least one of the following: entity content, emotional expression, scene, etc.

[0110] In addition, the electronic device may also generate unique identification information for the element, and the element is uniquely identified by the identification information.

[0111] For 2)

[0112] In specific implementation, for each piece of feature information, its matching score with the n historical operation nodes may be calculated respectively, and then, based on the matching score with the n historical operation nodes, its corresponding first score may be calculated.

[0113] The n historical operation nodes may be all historical operation nodes of the electronic device, or may be all historical operation nodes of the electronic device within a certain period of time. The historical operation node is a node that has received an operation in history, and the node includes at least one of a video frame and an audio frame. It is worth noting that the historical operation node is not limited to the first video file, and may include all operation nodes in the historical video playback or audio playback of the electronic device.

[0114] In some embodiments, the step of obtaining each first score corresponding to each feature information, wherein the first score corresponding to each feature information is determined based on a matching score between the feature information and the historical operation node, may include:

[0115] Inputting the first feature information into an attribution model to obtain a first attribution factor, wherein the attribution model is trained based on the feature information of each element in the historical operation node, and the first feature information is any one of the feature information;

[0116] Inputting the first feature information into a first model to obtain a second attribution factor, wherein the first model is obtained by performing a time-frequency domain conversion on the attribution model;

[0117] Aggregating the first attribution factor and the second attribution factor to obtain a third attribution factor;

[0118] According to the weight values ​​respectively corresponding to the historical operation nodes, the third attribution factor is corrected to obtain a fourth attribution factor;

[0119] Determine each historical operation node corresponding to at least one attribution factor with the largest value among the fourth attribution factors as each target historical operation node matching the first feature information;

[0120] Determining the value of the at least one attribution factor as each matching score between the first feature information and each target historical operation node;

[0121] A first score corresponding to the first feature information is determined according to each matching score.

[0122] In this embodiment, the time domain attribution factor and the frequency domain attribution factor corresponding to the feature information can be obtained. After the time domain attribution factor and the frequency domain attribution factor are integrated, the integrated attribution factor can be pre-corrected and the target historical operation node can be determined using the corrected attribution factor, thereby improving the reliability of the determination of the target historical operation node, and further improving the accuracy of the first score corresponding to the feature information determined based on the matching score between the feature information and the target historical operation node.

[0123] In a specific implementation, obtaining each first score corresponding to each feature information may include:

[0124] 2-1) determining each target historical operation node matching the first feature information (hereinafter referred to as k target historical operation nodes), and k matching scores between the first feature information and the k target historical operation nodes, wherein the first feature information is any one of the m feature information, and k is a positive integer;

[0125] 2-2) Determine a first score corresponding to the first feature information based on the k matching scores.

[0126] For 2-1)

[0127] In a specific implementation, the k target historical operation nodes may be: historical operation nodes corresponding to the k highest matching scores among the matching scores between the first feature information and each historical operation node.

[0128] In an optional implementation, the determining k target historical operation nodes matching the first feature information and k matching scores between the first feature information and the k target historical operation nodes may include:

[0129] Inputting the first feature information into an attribution model to obtain a first attribution factor, wherein the first attribution factor includes n attribution factors, wherein the attribution model is trained based on the feature information of each element in the n historical operation nodes, and the n first attribution factors represent a matching score of the first feature information to the n historical operation nodes, where n is an integer greater than or equal to k;

[0130] According to the first attribution factor, k target historical operation nodes in the second object that match the first feature information and k matching scores between the first feature information and the k target historical operation nodes are determined.

[0131] In this implementation, the electronic device may determine the k target historical operation nodes with the help of an attribution model.

[0132] The input of the attribution model is the first feature information, and the output is the first attribution factor corresponding to the n historical operation nodes. The first attribution factor is: s(s 1 ,s 2 ,s 3 ,s 4 ,s 5 ,s 6 …). The arrangement order of the n first attribution factors corresponding to the n historical operation nodes is the same as the time of the n historical operation nodes. The time of the historical operation node can be understood as: the time when the historical operation node receives the operation for the last time.

[0133] The attribution model can be used to attribute the single feature information of an element to the historical operation node. In specific implementation, the attribution model can be constructed by the following information: the feature information of each element in the n historical operation nodes; the operation action and its weight; and the user portrait.

[0134] In an optional implementation, the electronic device may directly determine the k attribution factors with the largest values ​​among the first attribution factors, that is, the historical operation nodes corresponding to the top k attribution factors, as k target operation nodes, and determine the top k attribution factors as the k matching scores.

[0135] In another possible implementation, determining, according to the first attribution factor, k target historical operation nodes in the second object that match the first feature information, and k matching scores between the first feature information and the k target historical operation nodes includes:

[0136] Inputting the first feature information into a first model to obtain a second attribution factor, wherein the second attribution factor includes n attribution factors, and the first model is obtained by performing time-frequency domain conversion on the attribution model;

[0137] Aggregating the first attribution factor and the second attribution factor to obtain a third attribution factor, wherein the third attribution factor includes n attribution factors;

[0138] According to the weight values ​​respectively corresponding to the n historical operation nodes, the third attribution factor is corrected to obtain a fourth attribution factor, wherein the fourth attribution factor includes n attribution factors;

[0139] Determine the historical operation nodes corresponding to the k fourth attribution factors with the largest values ​​among the n fourth attribution factors as the k target historical operation nodes matching the first feature information;

[0140] The values ​​of the k fourth attribution factors are determined as k matching scores between the first feature information and the k target historical operation nodes.

[0141] In this implementation, the electronic device may further perform time-frequency domain conversion on the attribution model to obtain a first model, and then use the second model to obtain a second attribution factor, where the second attribution factor is: p(p 1 ,p 2 ,p 3 ,p 4 ,p 5 ,p 6 …).

[0142] The first model F(t) can be converted by the following formula:

[0143]

[0144] Among them, f(x) represents the attribution model, e represents the natural constant, t represents the time of the operation node, w represents the mean time interval of all historical operation nodes, and i represents the complex operation.

[0145] The first attribution factor and the second attribution factor can be understood as attribution factors in the time domain and the frequency domain. Therefore, the attribution factors in the time domain and the frequency domain can be aggregated to obtain a third attribution factor. The third attribution factor is recorded as: z(z 1 ,z 2 ,z 3 ,z 4 ,z 5 ,z 6 …).

[0146] In specific implementation, the third attribution factor can be calculated by the following aggregation formula:

[0147]

[0148] The value of j ranges from 1 to n. j is the jth attribution factor in the first attribution factor, p i is the jth attribution factor in the second attribution factor, z j is the jth attribution factor in the third attribution factor.

[0149] Of course, in other implementations, the attribution factors in the time domain and the frequency domain may also be aggregated in other ways, such as j =s j +p j , which can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0150] Furthermore, the electronic device may also perform feed-forward correction on the third attribution factor. In an optional implementation, the weight values ​​of the n historical operation nodes may be used to perform feed-forward correction on the third attribution factor to obtain a fourth attribution factor, which is recorded as: z_b(z 1b ,z 2b ,z 3b ,z 4b ,z 6b …).

[0151] Assume that the weight sequence of each historical operation node is identified as: w(w 1 ,w 2 ,w 3 ,w 4 ,w 5 ,w 6…). The weight of a historical operation point can be determined based on the operation actions it receives and the weight values ​​corresponding to the operation actions. For example, the weight of a historical operation node can be the sum of the weight values ​​of the operation actions it receives. For example, assuming that the operation actions include: fast forward, pause, skip, replay, like, share, comment, etc.; the fast forward weight is -1, the pause weight is 1, the replay weight is 10, the like weight is 5, the skip weight is -2, the share weight is 4, and the comment weight is 8. At the same time, assuming that a historical operation node receives fast forward and skip operations, then the weight of the historical operation node can be: -1+-2=-3.

[0152] In specific implementation, the fourth attribution factor can be calculated using the following feedforward correction formula:

[0153]

[0154] Among them, w j is the weight value of the jth historical operation node, is the sum of the weight values ​​of n historical operation nodes, z j is the jth attribution factor in the third attribution factor, z jb is the jth attribution factor in the fourth attribution factor.

[0155] In this implementation, the electronic device can directly determine the k attribution factors with the largest values ​​in the fourth attribution factor, that is, the historical operation nodes corresponding to the top k attribution factors, as k target operation nodes, and determine the top k attribution factors as the k matching scores.

[0156] In this implementation method, the time domain attribution factor and the frequency domain attribution factor corresponding to the characteristic information can be obtained. After the time domain attribution factor and the frequency domain attribution factor are integrated, the integrated attribution factor can be pre-corrected and the k target historical operation nodes can be determined using the corrected attribution factor, thereby improving the reliability of determining the k target historical operation nodes.

[0157] For 2-2)

[0158] In an optional implementation, the electronic device may determine the sum of the k matching scores as the first score corresponding to the first feature information.

[0159] In another optional implementation manner, determining the first score corresponding to the first feature information according to the k matching scores may include:

[0160] Determine the initial weight of each element in each target historical operation node according to the number of operations and the type of operation of each target historical operation node;

[0161] Determine a weight coefficient of each target historical operation node according to the difference between the current time and the operation time of each target historical operation node, wherein the weight coefficient is negatively correlated with the difference;

[0162] Determine the target weight value of each element in each historical operation node according to the initial weight of each element in each target historical operation node and the weight coefficient of each target historical operation node;

[0163] Utilizing the target weight value of each element in each historical operation node, element-level weighting is performed on the k matching scores to obtain a first score corresponding to the first feature information.

[0164] In this implementation manner, the electronic device performs element-level weighting on the k matching scores to obtain a first score corresponding to the first feature information.

[0165] In specific implementation, the initial weight of each element in each target historical operation node is first obtained. It should be noted that for the same historical operation node, the initial weights of the elements included therein are the same.

[0166] The initial weight of each element of the history operation node can be equal to the sum of the weight values ​​of the operation types of each operation number. For example, assuming that the operation actions include: fast forward, pause, skip, replay, like, share, comment, etc.; the fast forward weight is -1, the pause weight is 1, the replay weight is 10, the like weight is 5, the skip weight is -2, the share weight is 4, and the comment weight is 8. At the same time, assuming that a history operation node receives 1 fast forward and 2 skip operations, then the initial weight of each element in the history operation node can be: -1+-2*2=-5.

[0167] Assuming that a certain historical operation node receives two or more operations, the operation time of the node may be: the time when the node receives the last operation.

[0168] The weight coefficient is negatively correlated with the difference, that is, for nodes with closer operation time, the smaller the difference, the larger the weight coefficient. The weight coefficient b can be calculated by the following formula:

[0169] b=1 / (1+a*t)

[0170] Wherein, a is the attenuation coefficient, and t is the difference.

[0171] The target weight value of an element of a certain historical operation node may be: the product of the initial weight of the element and the weight coefficient of the historical operation node.

[0172] It is understandable that the number of elements included in different historical operation nodes may be equal or different. By performing element-level weighting on the k matching scores, a historical operation node with a larger number of elements is given a greater weight to the matching score corresponding to the historical operation node, thereby improving the reliability of the first score determination.

[0173] For 3)

[0174] In an optional implementation manner, each first score may be used as a vector element to generate a vector corresponding to the first element or the second element.

[0175] In another implementation manner, generating a vector corresponding to the first element or the second element according to each first score includes:

[0176] Performing distribution transformation on each first score to obtain each second score;

[0177] Use each second score as a vector element to generate a vector corresponding to the first element or the second element

[0178] In this embodiment, considering that the first scores of different feature information of the same element have large differences, the m first scores can be distributed transformed to obtain m second scores, and the m second scores are used to generate a vector corresponding to the target element, thereby reducing the differences between the feature information and improving the reliability of determining the first score.

[0179] Let the first score be y and the second score be h(y). The distribution transformation formula can be:

[0180] h(y r )=(y r -y min ) / (y max -y min )

[0181] Among them, y r represents the rth first score among m first scores, y min represents the lowest score among the m first scores, y max represents the highest score among the m first scores.

[0182] It should be noted that the various optional implementation modes introduced in the embodiments of the present application can be implemented in combination with each other or separately if they do not conflict with each other, and the embodiments of the present application are not limited to this.

[0183] For ease of understanding, the following examples are provided:

[0184] The video processing method of the embodiment of the present application can add customized elements to the played video, that is, can dynamically insert elements into the played video, which can specifically include the following steps:

[0185] Step 1: When the user is watching a video on the terminal device, there is a button for "adding elements" on the video screen. After clicking the button, you can select pictures, videos, audio subtitles and other types of files from the local file management or local system. Figure 2a shown.

[0186] Step 2: If you choose to add information elements of a picture to the playing video, you can manually drag each element in the picture to the video screen.

[0187] If an element is dragged to the progress bar position of a video, the electronic device can automatically retrieve all the pictures associated with the element in the played video, and then display these pictures in the form of thumbnails, such as Figure 2b shown.

[0188] If you simply drag an element to another area in the video screen, it means that the element is added to the currently playing video screen. The currently playing video screen may be the video screen that is being played when the element is dragged.

[0189] Of course, you can also drag the complete picture to a certain position of the video progress bar, and insert the picture into part or all of the video frames before or after the target frame. The target frame can be the video playback frame corresponding to the position.

[0190] This implementation helps users to quickly and conveniently associate elements by automatically associating elements to be selected (also called elements to be associated, elements to be operated, or elements to be inserted) with elements of the video being played. Specifically, the following operations are included:

[0191] 1. Generate a source ID for the element to be associated through content understanding technology. The source ID is a non-repeating self-incrementing ID. Identify the source tag of the element to be associated (i.e. the aforementioned feature information) 1 ,c 2 ,c 3 ,c 4 ,c 5 …}, which can be composed of shape, color, scene, entity content, style, emotional expression, video plot expression, etc. For example, an element source is marked as {circle, yellow, playground, basketball, bright, warm, basketball game}.

[0192] 2. Construct a causal attribution model (i.e., the aforementioned attribution model). The causal attribution model attributes a single source tag of the element to be operated to some previous operations. Based on the attribution model, determine the K historical operation nodes with the highest matching degree with the single source tag, and determine the score of the source tag based on the scores of the K historical operation nodes.

[0193] a) Get the source tags of all elements of each historical operation node (the node when the user operates the video), the user's operation actions such as fast forward, pause, skip, replay, like, share, comment, etc. when playing the video, and their weights are {a m :w m ,a st :w st ,a sk :w sk ,a r :w r ,a l :w l ,a s :w s ,a c :w c}, user basic attribute portrait, and build a user causal attribution model f(x) based on the above information, where x is the input expression of all the above information.

[0194] b) Using the causal attribution model, based on a single source tag, the matching probability of all elements to be inserted for all historical operation nodes is calculated, which is recorded as the temporal attribution factor (i.e., the first attribution factor mentioned above) s(s 1 ,s 2 ,s 3 ,s 4 ,s 5 ,s 6 …), the arrangement order of the time domain attribution factors corresponding to each historical operation node is the same as the time order of the historical operation node, and the time of the historical operation node may be the time when the historical operation node last received an operation.

[0195] c) The causal attribution model f(x) is converted into a frequency domain attribution factor (i.e., the aforementioned second attribution factor) p(p 1 ,p 2 ,p 3 ,p 4 ,p 5 ,p 6 …). The time domain-frequency domain module conversion formula is as follows Among them, t represents the time of the operation node, w represents the mean time interval of all historical operation nodes, and i represents complex operation.

[0196] d) Aggregate the time domain attribution factor and the frequency domain attribution factor to form the attribution factor (i.e. the third attribution factor mentioned above) z(z 1 ,z 2 ,z 3 ,z 4 ,z 5 ,z 6 …), the aggregation formula for a single operation node is: Similarly, we can obtain z 2 ,z 3 ,z 4 …, where n represents the number of historical operation nodes.

[0197] e) Perform feed-forward correction on the attribution factor z. Each operation node has a corresponding weight, and each node weight is in the weight set {w m ,w st ,w sk ,w r ,w l ,w s ,w c}, so the weight sequence composed of all operation nodes is expressed as w(w 1 ,w 2 ,w 3 ,w 4 ,w 5 ,w 6 …). The formula for single operation node feedforward correction is:

[0198]

[0199] Thus, the unbiased attribution factor (i.e., the fourth attribution factor mentioned above) z_b(z 1b ,z 2b ,z 3b ,z 4b ,z 6b …).

[0200] f) Select the top K unbiased attribution factors, corresponding to the top K historical operation nodes, and the scores of the K operation nodes can be represented as the scores of a single source tag of the element to be inserted.

[0201] 3. Calculate the score of a single source tag through the TopK historical nodes obtained in step 2. The steps for calculating the source tag score are as follows:

[0202] a) Initial weight: Different operation nodes are assigned different weights according to the source tags in the elements of different operation nodes. The inherited weight set {w m ,w st ,w sk ,w r ,wl ,w s ,w c}, such as the pause weight is 1, the replay weight is 10, the like weight is 5, the skip weight is -2, the share weight is 4, and the comment weight is 8.

[0203] b) Time decay: Calculate the time difference t between the current time and the operation node, and use the constant decay formula 1 / (1+a*t), where a is the decay coefficient. Give appropriate decay to different weights. The core idea is that users will give greater weight to operations that are closer in time.

[0204] c) Through steps a and b, the scores of all historical operation nodes are element-wise weighted to obtain the score of the source tag (i.e., the first score mentioned above). For example, the final score of the source tag (circle) is 92.3.

[0205] 4. Calculate steps 2 and 3 in parallel to get the scores of all source tags of the element to be inserted, such as an element {circle: 92.3, yellow: 10.5, playground: 4.8, basketball: 78.0, bright: 23.7, warm: 78.6, basketball game: 24.9}. It can be seen that the variance of the scores of the elements is quite different, so a distribution transformation is required. The transformation formula is h(y) = (y-min) / (max-min), where y represents the score to be transformed, min represents the maximum value of the source tag score, max represents the minimum value of the source tag score, and h(y) represents the transformed value (i.e., the second score mentioned above). The transformed source tag can be converted into the source vector E[e 1 ,e 2 ,e 3 ,e 4 ,e 5 ,e 6 …], such as E = [0.13, 0.45, 0, 89, 0.02, 0.11, 0.25…].

[0206] 5. By circling the video progress bar, confirm the associated range of the video, automatically identify all elements of the circled video, and generate source IDs (region binding) and source tags for all elements. Use steps 2, 3, and 4 to calculate the source tag scores and source vectors of all elements in the circled video.

[0207] 6. Through vector calculation, calculate the similarity score between the source vector of the element to be inserted and the source vectors of all elements in the circled video. By setting the threshold, get the q most similar elements, and complete the automatic association of the elements.

[0208] Step 3. If a video file is inserted, the added video file will be displayed above the current video playback screen. If part of the video is extracted from the added video for addition, for example, a video clip containing all the pictures of a certain person A in the current playback screen is extracted from the added video, the person A in the current playback screen can be dragged to the video file above by gestures, and the video will automatically parse out the clip containing only all the pictures of the person A (the automatic parsing and association function is similar to step 102. The subsequent video adding process: you can long press the extracted video again, and multiple frame thumbnails will appear below the added video, such as Figure 3 shown.

[0209] Step 4: If it is a dynamic insertion of a sound information element, the sound file is displayed above the video screen. If the insertion is related to a sound feature in the current video playback, the character in the current playback screen can be dragged into the sound file, and the system automatically associates the sound of the character in the currently playing video with the sound in the sound file (the automatic association process is similar to step 102). If the text of subtitles is used as key information to extract the sound, the key information in the subtitles can be selected, and the subtitle information can be moved to the sound file to extract the sound part containing the keyword from the sound file.

[0210] Step 5: Based on step 4, the extracted sound file is dragged to the subtitle part or character part of the video screen again, and the sound part involved in the current screen can be replaced, such as Figure 4 shown.

[0211] Step 6. After the information elements are added, the next time you play the video, the elements added last time will be automatically recorded and played accordingly.

[0212] The embodiment of the present application can solve the problem of low flexibility of existing video playback. It can support the insertion or replacement of information such as pictures, videos, and audio files. Specifically, it can update the content of video files by calculating the similarity between elements, thereby improving the flexibility of video playback.

[0213] It should be noted that the video processing method provided in the embodiment of the present application can be executed by a video processing device, or a control module in the video processing device for executing the video processing method. In the embodiment of the present application, the video processing device provided in the embodiment of the present application is described by taking the video processing device executing the video processing method as an example.

[0214] See also Figure 5 , Figure 5 It is a structural diagram of a video processing device provided in an embodiment of the present application.

[0215] like Figure 5As shown, the video processing device 500 includes:

[0216] An acquisition module 501 is used to acquire a first video file and a target file;

[0217] A calculation module 502, configured to calculate the similarity between each first element in the first video file and each second element in the target file;

[0218] An updating module is used to update the first video file according to the similarity.

[0219] In some embodiments, the update module includes:

[0220] A first determining submodule, configured to determine at least one element association pair according to the similarity, each element association pair comprising a first element and a second element, and the similarity between the first element and the second element in each element association pair is greater than a preset threshold;

[0221] An updating submodule is used to add a target second element in the at least one element association pair in the first video frame of the first video file; or replace the target first element in the at least one element association pair in the first video file with the target second element in the at least one element association pair.

[0222] In some embodiments, the computing module comprises:

[0223] A second determination submodule, used to determine first vectors corresponding one-to-one to first elements in the first video file, and second vectors corresponding one-to-one to second elements in the target file;

[0224] The calculation submodule is used to calculate the similarity between each first vector and each second vector.

[0225] In some embodiments, the second determining submodule includes:

[0226] A first acquisition unit, used to acquire each feature information of the first element or the second element;

[0227] A second acquisition unit is used to acquire each first score corresponding to each feature information, wherein the first score corresponding to each feature information is determined based on a matching score between the feature information and a historical operation node;

[0228] A generating unit is used to generate a vector corresponding to the first element or the second element according to each first score.

[0229] In some embodiments, the generating unit is specifically used for:

[0230] Performing distribution transformation on each first score to obtain each second score;

[0231] Each second score is used as a vector element to generate a vector corresponding to the first element or the second element.

[0232] In some embodiments, the second acquiring unit includes:

[0233] A first acquisition subunit is used to input the first feature information into an attribution model to obtain a first attribution factor, wherein the attribution model is trained based on the feature information of each element in the historical operation node, and the first feature information is any one of the feature information;

[0234] A second acquisition subunit is used to input the first feature information into a first model to obtain a second attribution factor, wherein the first model is obtained by performing time-frequency domain conversion on the attribution model;

[0235] an aggregation subunit, configured to aggregate the first attribution factor and the second attribution factor to obtain a third attribution factor;

[0236] A deviation correction processing subunit, configured to perform deviation correction processing on the third attribution factor according to the weight values ​​respectively corresponding to the historical operation nodes, so as to obtain a fourth attribution factor;

[0237] A first determining subunit is used to determine each historical operation node corresponding to at least one attribution factor with the largest value among the fourth attribution factors as each target historical operation node matching the first feature information;

[0238] a second determining subunit, configured to determine the value of the at least one attribution factor as each matching score between the first feature information and each target historical operation node;

[0239] The third determining subunit is configured to determine a first score corresponding to the first feature information according to each matching score.

[0240] In some embodiments, the third determining subunit is specifically used for:

[0241] Determine the initial weight of each element in each target historical operation node according to the number of operations and the type of operation of each target historical operation node;

[0242] Determine the attenuation value of each target historical operation node according to the difference between the current time and the operation time of each target historical operation node, wherein the attenuation value is negatively correlated with the difference;

[0243] Determine the target weight value of each element in each target historical operation node according to the initial weight of each element in each target historical operation node and the attenuation value of each target historical operation node;

[0244] Utilizing the target weight value of each element in each target historical operation node, element-level weighting is performed on each matching score to obtain a first score corresponding to the first feature information.

[0245] The video processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0246] The video processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0247] The video processing device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the video processing device in the method embodiment are not described here.

[0248] Optional, such as Figure 6 As shown, the embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, each process of the above-mentioned video processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0249] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0250] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0251] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 7010 and other components.

[0252] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 170 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0253] The processor 7010 is used for:

[0254] Obtain a first video file and a target file;

[0255] Calculating the similarity between each first element in the first video file and each second element in the target file;

[0256] The first video file is updated according to the similarity.

[0257] The electronic device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the video processing device in the method embodiment are not described here.

[0258] It should be understood that in the embodiment of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 709 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 7010 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 7010.

[0259] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned video processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0260] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0261] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated.

[0262] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0263] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0264] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0265] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video processing method, characterized in that: include: Obtain a first video file and a target file; Calculating the similarity between each first element in the first video file and each second element in the target file; According to the similarity, updating the first video file; The method further comprises: Obtaining each characteristic information of the first element or the second element; Inputting the first feature information into an attribution model to obtain a first attribution factor, wherein the attribution model is trained based on feature information of each element in the historical operation node, and the first feature information is any one of the feature information; Inputting the first feature information into a first model to obtain a second attribution factor, wherein the first model is obtained by performing a time-frequency domain conversion on the attribution model; Aggregating the first attribution factor and the second attribution factor to obtain a third attribution factor; According to the weight values ​​respectively corresponding to the historical operation nodes, the third attribution factor is corrected to obtain a fourth attribution factor; Determine each historical operation node corresponding to at least one attribution factor with the largest value among the fourth attribution factors as each target historical operation node matching the first feature information; Determining the value of the at least one attribution factor as each matching score between the first feature information and each target historical operation node; Determining a first score corresponding to the first feature information according to each matching score; Generate a vector corresponding to the first element or the second element according to each first score; The similarity between each first element in the first video file and each second element in the target file is calculated based on each first vector corresponding to each first element and each second vector corresponding to each second element.

2. The method according to claim 1, characterized in that The updating of the first video file according to the similarity includes: Determine at least one element association pair according to the similarity, each element association pair includes a first element and a second element, and the similarity between the first element and the second element in each element association pair is greater than a preset threshold; Adding the target second element in the at least one element association pair in the first video frame of the first video file; or replacing the target first element in the at least one element association pair in the first video file with the target second element in the at least one element association pair.

3. The method according to claim 1, characterized in that Generating a vector corresponding to the first element or the second element according to each first score includes: Performing distribution transformation on each first score to obtain each second score; Each second score is used as a vector element to generate a vector corresponding to the first element or the second element.

4. The method according to claim 1, characterized in that The determining, according to each matching score, a first score corresponding to the first feature information includes: Determine the initial weight of each element in each target historical operation node according to the number of operations and the type of operation of each target historical operation node; Determine the attenuation value of each target historical operation node according to the difference between the current time and the operation time of each target historical operation node, wherein the attenuation value is negatively correlated with the difference; Determine the target weight value of each element in each target historical operation node according to the initial weight of each element in each target historical operation node and the attenuation value of each target historical operation node; Utilizing the target weight value of each element in each target historical operation node, element-level weighting is performed on each matching score to obtain a first score corresponding to the first feature information.

5. A video processing device, characterized in that: include: An acquisition module, used for acquiring a first video file and a target file; A calculation module, used for calculating the similarity between each first element in the first video file and each second element in the target file; An updating module, used for updating the first video file according to the similarity; The calculation module comprises: A first acquisition unit, used to acquire each feature information of the first element or the second element; A first acquisition subunit is used to input the first feature information into an attribution model to obtain a first attribution factor, wherein the attribution model is trained based on the feature information of each element in the historical operation node, and the first feature information is any one of the feature information; A second acquisition subunit is used to input the first feature information into a first model to obtain a second attribution factor, wherein the first model is obtained by performing time-frequency domain conversion on the attribution model; an aggregation subunit, configured to aggregate the first attribution factor and the second attribution factor to obtain a third attribution factor; A deviation correction processing subunit, configured to perform deviation correction processing on the third attribution factor according to the weight values ​​respectively corresponding to the historical operation nodes, so as to obtain a fourth attribution factor; A first determining subunit is used to determine each historical operation node corresponding to at least one attribution factor with the largest value among the fourth attribution factors as each target historical operation node matching the first feature information; a second determining subunit, configured to determine the value of the at least one attribution factor as each matching score between the first feature information and each target historical operation node; A third determining subunit, configured to determine a first score corresponding to the first feature information according to each matching score; a generating unit, configured to generate a vector corresponding to the first element or the second element according to each first score; The similarity between each first element in the first video file and each second element in the target file is calculated based on each first vector corresponding to each first element and each second vector corresponding to each second element.

6. The device according to claim 5, characterized in that The update module comprises: A first determining submodule, configured to determine at least one element association pair according to the similarity, each element association pair comprising a first element and a second element, and the similarity between the first element and the second element in each element association pair is greater than a preset threshold; An updating submodule is used to add a target second element in the at least one element association pair in the first video frame of the first video file; or replace the target first element in the at least one element association pair in the first video file with the target second element in the at least one element association pair.

7. The device according to claim 5, characterized in that The generating unit is specifically used for: Performing distribution transformation on each first score to obtain each second score; Each second score is used as a vector element to generate a vector corresponding to the first element or the second element.

8. The device according to claim 5, characterized in that The third determining subunit is specifically used for: Determine the initial weight of each element in each target historical operation node according to the number of operations and the type of operation of each target historical operation node; Determine the attenuation value of each target historical operation node according to the difference between the current time and the operation time of each target historical operation node, wherein the attenuation value is negatively correlated with the difference; Determine the target weight value of each element in each target historical operation node according to the initial weight of each element in each target historical operation node and the attenuation value of each target historical operation node; Utilizing the target weight value of each element in each target historical operation node, element-level weighting is performed on each matching score to obtain a first score corresponding to the first feature information.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the video processing method according to any one of claims 1 to 4.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the video processing method according to any one of claims 1 to 4 are implemented.

Images