Video positioning method and device

Abstract

The invention discloses a video positioning method and device, and belongs to the technical field of artificial intelligence, and the method comprises the steps: receiving a first input which is used for inputting scene key information, and the scene key information comprises at least one of the following items: scene description information; scene stick figure images; and in response to the first input, displaying a target image frame in the at least one first video, the picture scene of the target image frame in the at least one first video being matched with the picture scene indicated by the scene key information.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a video positioning method and device thereof. Background Art

[0002] With the popularization of smart electronic devices, users have an increasing need to use electronic devices to watch videos. During the process of watching a video, if the user needs to locate the position of an image frame in the video, the user can manually slide the slider on the progress bar so that the electronic device determines the position of the image frame according to the end position of the user's sliding.

[0003] However, in the above method, the user usually needs to slide the slider multiple times to find the approximate position of the video clip corresponding to the image frame, that is, the user's operation steps are cumbersome and time-consuming, thus resulting in low efficiency and poor accuracy in locating the image frame by the electronic device. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video positioning method and a device thereof, which can improve the efficiency and accuracy of electronic equipment in positioning image frames in a video.

[0005] In a first aspect, an embodiment of the present application provides a video positioning method, which includes: receiving a first input, the first input being used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; a scene stick figure image; in response to the first input, displaying a target image frame in at least one first video, wherein the picture scene of the target image frame in at least one first video matches the picture scene indicated by the scene key information.

[0006] In a second aspect, an embodiment of the present application provides a video positioning device, which includes: a receiving module and a display module; the receiving module is used to receive a first input, the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; scene stick figure image; the display module is used to display a target image frame in at least one first video in response to the first input received by the receiving module, wherein the picture scene of the target image frame in at least one first video matches the picture scene indicated by the scene key information.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program / program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In an embodiment of the present application, a first input is received, and the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; scene stick figure image; in response to the first input, a target image frame in at least one first video is displayed, wherein the picture scene of the target image frame in at least one first video matches the picture scene indicated by the scene key information. In this solution, since the electronic device can directly display the target image frame matching the picture scene indicated by the scene key information according to the scene key information input by the user, without the user having to operate the progress bar multiple times, the efficiency of the electronic device in locating a specific image frame in the video can be improved, and the accuracy of the image frame display can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of an example of manually sliding a slider on a progress bar to adjust an image frame displayed on an interface in the related art;

[0013] Figure 2 This is one of the flow charts of a video positioning method provided in an embodiment of the present application;

[0014] Figure 3 This is one of the example schematic diagrams of inputting scene key information provided in the embodiment of the present application;

[0015] Figure 4 This is a second example schematic diagram of inputting scene key information provided by an embodiment of the present application;

[0016] Figure 5 It is one of the flow charts for generating a scene prediction image according to scene key information provided in an embodiment of the present application;

[0017] Figure 6 This is the second flowchart of generating a scene prediction image according to scene key information provided by an embodiment of the present application;

[0018] Figure 7 This is a flow chart of determining a target image frame according to scene key information provided by an embodiment of the present application;

[0019] Figure 8 This is the second flowchart of a video positioning method provided in an embodiment of the present application;

[0020] Fig. 9A This is one of the schematic diagrams of an example of editing a scene stick figure image provided by an embodiment of the present application;

[0021] Fig. 9B This is the second schematic diagram of an example of editing a scene stick figure image provided by an embodiment of the present application;

[0022] Fig. 9C This is a third schematic diagram of an example of editing a scene stick figure image provided by an embodiment of the present application;

[0023] Fig.10 This is the third flowchart of a video positioning method provided in an embodiment of the present application;

[0024] Fig.11 This is a fourth flowchart of a video positioning method provided in an embodiment of the present application;

[0025] Fig.12 is a structural schematic diagram of a video positioning device provided in an embodiment of the present application;

[0026] Fig.13 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0027] Fig.14 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The terms "at least one (item)", "at least one of" and the like in the specification of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0031] The video positioning method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0032] The video positioning method provided in the embodiment of the present application can be applied to the scenario of viewing a specific image frame or video clip during the process of watching a video.

[0033] In the related art, when a user is watching a video, if the user needs to locate a position of an image frame in the video, Figure 1 As shown, the user can manually slide the slider on the progress bar so that the electronic device determines the position of the image frame according to the end position of the user's sliding.

[0034] However, in the existing manual sliding slider method, the user usually needs to slide the slider multiple times to search repeatedly before finding the approximate position of the video clip corresponding to the image frame; and sometimes the user may manually slide for a long time but still cannot find the position of the image frame to be located; thus, the user's need to quickly and accurately locate the image frame cannot be met, resulting in low efficiency and poor accuracy in locating the image frame by the electronic device.

[0035] Alternatively, in the prior art, a method of locating an image frame based on the dialogue text can be used to completely match the dialogue text remembered by the user with the dialogue text corresponding to the video. However, in the implementation of this method, if the user's memory of the dialogue text is incomplete or inaccurate, the electronic device will have poor accuracy in locating the image frame, or may even be unable to locate the image frame. Furthermore, if the user does not remember the dialogue text corresponding to the image frame to be located, or the image frame originally has no dialogue, it is even more impossible to locate the image frame using the dialogue text matching method.

[0036] In the video positioning method and device provided in the embodiment of the present application, a first input is received, and the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; scene stick figure image; in response to the first input, a target image frame in at least one first video is displayed, wherein the picture scene of the target image frame in at least one first video matches the picture scene indicated by the scene key information. In this solution, since the electronic device can directly display the target image frame matching the picture scene indicated by the scene key information according to the scene key information input by the user, without the user having to operate the progress bar multiple times, the efficiency of the electronic device in positioning a specific image frame in the video can be improved, and the accuracy of the image frame display can be improved.

[0037] The execution subject of the video positioning method provided in the embodiment of the present application is a video positioning device, which can be an independent device, or an electronic device, or a functional module or entity in an electronic device, and the embodiment of the present application does not limit this. The following will take an electronic device as an example to exemplify the video playback method provided in the embodiment of the present application.

[0038] The present application embodiment provides a video positioning method. Figure 2 FIG. 1 is a flow chart of a video positioning method provided by an embodiment of the present application. Figure 2 As shown, the video positioning method provided in the embodiment of the present application may include the following steps 201 and 202.

[0039] Step 201: An electronic device receives a first input.

[0040] In an embodiment of the present application, the above-mentioned first input can be used to input scene key information.

[0041] In the embodiment of the present application, the above-mentioned scene key information may include at least one of the following: scene description information; scene stick figure image.

[0042] In some embodiments of the present application, the first input may be an input to an information input box, or may be an input of a specified video.

[0043] In some embodiments of the present application, the above-mentioned information input box may be a specific information input box displayed by the electronic device according to the user's input.

[0044] In some embodiments of the present application, the above step 201 can be specifically implemented through the following step 201a.

[0045] Step 201a: The electronic device receives a first input for a second video.

[0046] In some embodiments of the present application, the second video may be any one of the following: a video locally stored in the electronic device, or an online video displayed in the electronic device.

[0047] In some embodiments of the present application, the scenario in which the electronic device receives the first input for the second video may include but is not limited to any of the following:

[0048] In a case where the interface displayed by the electronic device includes a video identifier of the second video, the electronic device may receive a first input of the second video;

[0049] In a case where the electronic device displays an image frame of the second video (for example, the electronic device is playing the second video, or the electronic device pauses playing the second video), the electronic device may receive a first input for the second video.

[0050] It should be noted that the image frame of the second video may be any image frame in the second video.

[0051] In some embodiments of the present application, when the electronic device displays the video identifier of the second video, the electronic device can receive a double-click input, a long press input, a specific gesture input, or other feasible input from the user on the video identifier of the second video to display an information input box, so that the user can input scene key information through the information input box.

[0052] In some embodiments of the present application, when the electronic device displays an image frame of a second video, the electronic device can receive a click input from the user on a specific icon in the playback interface of the second video, or the electronic device can receive a specific gesture input from the user to display an information input box so that the user can input scene key information through the information input box.

[0053] Exemplarily, the specific gesture input may be: two-finger long press input, three-finger sliding input, etc.

[0054] In some embodiments of the present application, the electronic device may display the information input box in any of the following ways:

[0055] (1) displaying a floating window on the playback interface of the second video, so as to display the information input box through the floating window;

[0056] (2) Jumping to display a new interface, so as to display the information input box in the new interface;

[0057] (3) reducing the display size of the playback interface of the second video, and displaying the playback interface of the second video and an interface including an information input box in a split-screen display;

[0058] (4) In the interface displaying the video identifier of the second video, a floating window is superimposed and displayed, so as to display the information input box through the floating window.

[0059] It should be noted that in the above-mentioned method (1) and method (3), the electronic device can continue playing the second video or stop playing the second video, and this application does not limit this.

[0060] In the above method (1), when the electronic device continues to play the second video, the floating window can be displayed in the edge area to reduce the interference of the floating window on the user's viewing of the second video; when the electronic device stops playing the second video, the floating window can be displayed in any position, such as the central display area of ​​the electronic device, to facilitate the user to input scene key information.

[0061] In the above method (2), the electronic device can display the playback interface of the second video on the new interface in the form of a floating window.

[0062] In the above-mentioned methods (2) and (4), the electronic device may not be involved in playing the second video.

[0063] In an embodiment of the present application, the electronic device can directly receive the user's first input for the second video, so that subsequently only the image frames contained in the second video need to be searched to locate the specified video. In this way, the efficiency of the electronic device in locating specific image frames in the specified video can be improved.

[0064] In some embodiments of the present application, after the electronic device displays the information input box, the electronic device may display prompt information at the position corresponding to the information input box to prompt the user to enter key information of the scene, for example, "Please enter the content description of the scene to be located", "Please simply draw a stick figure of the scene to be located".

[0065] In some implementations of the present application, the above-mentioned scene description information can be used to describe relevant information of the object contained in the image frame, picture background and other information.

[0066] The object-related information may include at least one of the following information about the object:

[0067] Quantity, posture, identity, hairstyle, action gender, clothing, age, clothing characteristics, hair accessories characteristics, object type, color, variety, etc.

[0068] In some embodiments of the present application, the above-mentioned object may include but is not limited to at least one of the following: a person, an animal, a plant, an object, etc.

[0069] Exemplarily, when the scene description information includes description information of characters, the scene description information may include but is not limited to at least one of the following: the number of characters, the gender of the characters, the age of the characters, the identity of the characters, the clothing of the characters, the hairstyle of the characters, the actions of the characters, etc.

[0070] For example, "a young lady wearing a white shirt and black suit pants."

[0071] Exemplarily, when the scene description information includes description information of animals, the scene description information may include but is not limited to at least one of the following: the number of animals, the type of animals, the color of animals, the posture of animals, etc.

[0072] For example, "two orange cats sleeping."

[0073] Exemplarily, when the scene description information includes description information of plants, the scene description information may include but is not limited to at least one of the following: the number of plants, the variety of plants, the color of plants, etc.

[0074] For example, "a bouquet of red roses".

[0075] Exemplarily, when the scene description information includes description information of an object, the scene description information may include but is not limited to at least one of the following: object type, object quantity, object color, object material, object shape, etc.

[0076] For example, "A blue transparent cup is placed on a wooden round table."

[0077] In some embodiments of the present application, the above-mentioned picture background may include but is not limited to at least one of the following: architectural background, natural environment background, etc.

[0078] Exemplarily, the above-mentioned architectural background may include but is not limited to: shopping malls, churches, temples, hospitals, castles, etc.

[0079] Exemplarily, the above-mentioned natural environment background may include, but is not limited to: grassland, hills, lakes, and sky.

[0080] In some embodiments of the present application, the above-mentioned scene description information may include but is not limited to at least one of the following: a phrase, a sentence, and a label.

[0081] In some embodiments of the present application, after the electronic device displays the information input box, the user triggers the electronic device to display scene description information in the information input box through text editing or voice input, etc. For example, the scene description information in the form of phrases "two children", "female teacher" and "bowing"; or the scene description information in the form of sentences "a scene of two children bowing to a female teacher".

[0082] For example, Figure 3 As shown, an example is given in which an electronic device receives a first input to a second video during the playback of the second video. If the user wants to find a specific image frame in the second video during the playback of the second video, the user performs a two-finger long press input on the playback interface 10 of the second video to trigger the electronic device to superimpose a floating window 11 on the playback interface 10 of the second video, and display a prompt message "Please enter the content description of the scene to be located" and an information input box 12 in the floating window. Among them, the information input box 12 can display the scene description information in text form "a scene of two children bowing to a female teacher" according to the user's first input.

[0083] It should be noted that, when the scene description information describes a more detailed scene, the accuracy of the target image frame subsequently displayed according to the scene description information will be higher.

[0084] In some embodiments of the present application, before the above-mentioned voice input, the user can click the voice input logo displayed by the electronic device to trigger the electronic device to start receiving voice information. Then, the electronic device can convert the received voice information into text information through voice recognition and other technologies, and display it in the information input box for the user to view or modify. Among them, the text information converted from the voice information is the scene description information.

[0085] In some embodiments of the application, after the electronic device displays the information input box, the electronic device may display a drawing mark, which can be used to switch the input form of scene key information so that the electronic device can receive the user's input of the scene stick figure image.

[0086] For example, Figure 4As shown, an example is given in which an electronic device receives a first input of a second video during the playback of the second video. If the user wants to find a specific image frame in the second video during the playback of the second video, the user performs a two-finger long press input on the playback interface 10 of the second video to trigger the electronic device to display a floating window 13 superimposed on the playback interface 10 of the second video, and a prompt message "Please simply draw a stick figure of the scene to be located" and an information input box 14 are displayed in the floating window. Among them, the information input box 14 can display a stick figure according to the user's first input, in which two children are bowing to a female teacher.

[0087] It should be noted that, when the scene content drawn by the scene stick figure is more detailed, the accuracy of the target image frame subsequently displayed according to the scene stick figure is higher.

[0088] In some embodiments of the present application, when an electronic device receives a scene stick figure image, the electronic device may display at least one brush identifier in an interface, each brush identifier indicating a type of brush, to help the user draw stick figures by using different types of brushes.

[0089] In some embodiments of the present application, when the electronic device receives a scene stick figure image, the electronic device may display a control for adjusting the line color in the interface, such as a color selector or a color extractor.

[0090] For example, taking a color selector as an example, the electronic device displays a variety of colors in the form of a color card or a color ring for the user to select. Taking a color extractor as an example, the electronic device can receive a user's selection input of any position in the interface to obtain the color parameter corresponding to the arbitrary position as the color parameter of the line displayed subsequently.

[0091] In some embodiments of the present application, when a user inputs a scene stick figure image, the electronic device may simultaneously display at least one label for the user to select, so that the user can supplement the description of the scene stick figure image by selecting a note. Each label is used to indicate the relevant information of an object. For the description of the relevant information of the object, refer to the description of the relevant information of the object in the above embodiment.

[0092] For example, in combination Figure 4, the electronic device may display at least one label in other areas of the playback interface 10 of the first video except the display area of ​​the floating window 13, such as "male teacher", "female teacher", "male student", "female student", etc. The user may select "female teacher", "male student", "female student" to supplement the "two children bowing to a female teacher" represented by the stick figure, so the target key information actually received by the electronic device is "a male student and a female student bowing to a female teacher".

[0093] In this way, on the basis of the scene sketch image, by adding tags and supplementing the information of the image drawn by the scene sketch image, the accuracy of the electronic device in locating a specific image frame in the video can be improved.

[0094] Step 202: The electronic device displays a target image frame in at least one first video in response to a first input.

[0095] In an embodiment of the present application, the picture scene of the target image frame in the at least one first video matches the picture scene indicated by the scene key information.

[0096] It can be understood that the electronic device responds to the first input and matches all image frames contained in the first video according to the scene key information input by the user, thereby determining an image frame that matches the picture scene indicated by the scene key information as the target image frame, and displays the target image frame for the user to view.

[0097] In some embodiments of the present application, when the above-mentioned first input is an input to an information input box, the above-mentioned first video may include at least one of the following: a video locally stored in the electronic device, an online video displayed by the electronic device.

[0098] In some embodiments of the present application, in combination with the above step 201a, the "displaying the target image frame in the first video" in the above step 202 can be specifically implemented by the following step 202a.

[0099] Step 202a: The electronic device displays the target image frame in the second video.

[0100] Among them, the picture scene of the target image frame in the second video matches the picture scene indicated by the scene key information.

[0101] In this way, the electronic device may only search for the image frames included in the second video to locate the designated video, thus improving the efficiency of the electronic device in locating specific image frames in the designated video.

[0102] In some embodiments of the present application, if before the first input, the interface displayed by the electronic device includes the video identifier of the first video, then in response to the first input, the electronic device may display the target image frame in any of the following ways:

[0103] The playback interface of the first video is displayed, and the target image frame is displayed through the playback interface, and the progress bar of the first video is adjusted to the progress display position corresponding to the target image frame, for example, directly displaying Figure 1 The image frame shown;

[0104] In the interface displayed by the electronic device, a floating window is superimposed and displayed to display the target image frame in the first video.

[0105] In some embodiments of the present application, if the electronic device has displayed an image frame of the first video before the first input, then in response to the first input, the electronic device may display the target image frame in any of the following ways:

[0106] The electronic device directly adjusts the displayed content in the playback interface of the first video to the target image frame, and adjusts the progress bar of the first video to the progress display position corresponding to the target image frame, for example, directly displaying Figure 1 The image frame shown;

[0107] The electronic device may continue to play the image frame played before receiving the first input, and superimpose a floating window on the playback interface of the first video to display the target image frame for the user to determine whether to jump to display.

[0108] In some implementations of the present application, if the user confirms that the target image frame is the image frame that needs to be located, the user can trigger the electronic device to display the target image frame in full screen by inputting the target image frame, for example, a two-finger zoom input.

[0109] In the video positioning method provided in the embodiment of the present application, since the electronic device can directly display the target image frame matching the picture scene indicated by the scene key information based on the scene key information input by the user, without the user having to perform multiple operations on the progress bar, the efficiency of the electronic device in locating specific image frames in the video can be improved, and the accuracy of the image frame display can be improved.

[0110] In some embodiments of the present application, before "displaying the target image frame in the first video" in the above step 202, the video positioning method provided in the embodiment of the present application also includes the following steps 301 and 302.

[0111] Step 301: The electronic device generates a scene prediction image based on scene key information.

[0112] In some embodiments of the present application, the picture resolution of the scene prediction image is the same as the picture resolution of the first video.

[0113] In some embodiments of the present application, after the electronic device generates a scene prediction image, the scene prediction image can be displayed so that the user can confirm whether the predicted scene image meets the requirements.

[0114] In some embodiments of the present application, if the predicted scene image generated by the electronic device does not meet user requirements, the user may supplement the input of scene key information so that the electronic device can update the predicted scene image generated before the supplementation according to the supplemented scene key information; or the electronic device can regenerate a new corresponding predicted scene image according to the supplemented scene key information.

[0115] In some embodiments of the present application, in one manner, the above step 301 can be specifically implemented through the following steps 301a and 301b.

[0116] Step 301a: The electronic device generates a Gaussian noise image having the same resolution as the picture resolution of the first video based on the picture resolution of the first video.

[0117] In some embodiments of the present application, the Gaussian noise image is randomly generated, and the picture resolution of the Gaussian noise image is the same as the picture resolution of the first video.

[0118] It can be understood that the picture resolution of any image frame in the first video is the same.

[0119] In some embodiments of the present application, after receiving the first input, the electronic device may first obtain the picture resolution of the video corresponding to the input, and then randomly generate a Gaussian noise image with the same picture resolution as the video.

[0120] Step 301b: Based on the scene key information, perform diffusion denoising on the Gaussian noise image to obtain a scene prediction image.

[0121] It should be noted that, since the diffusion denoising process of the Gaussian noise image is guided by the scene key information, it can be ensured that the picture scene of the Gaussian noise image after the diffusion denoising process matches the picture scene indicated by the scene key information.

[0122] In some embodiments of the present application, the above-mentioned diffusion denoising process can be: a process of removing noise points in a Gaussian noise image by pixel iteration under the guidance of scene key information. It should be noted that the more times of iteration, the more the subsequently generated scene prediction image conforms to the picture scene indicated by the scene key information.

[0123] In some embodiments of the present application, the electronic device can input scene key information and Gaussian noise image into an image generation model for processing. Based on the scene key information, through the diffusion denoising module in the image generation model, the Gaussian noise image is subjected to diffusion denoising processing based on the scene key information to obtain and output a scene prediction image.

[0124] The image generation model may be a Stable Diffsuion model. Of course, the image generation model may also be any other possible model. For other descriptions of the Stable Diffsuion model, see the related art.

[0125] In some embodiments of the present application, the electronic device may perform targeted training on the initial Stable Diffusion model using a training data set related to the image frame to obtain a Stable Diffusion model.

[0126] In some embodiments of the present application, the electronic device may obtain at least one training data set according to the video type to perform model training through the at least one training data set, wherein different training data sets in the at least one training data set correspond to different video types.

[0127] Exemplarily, the above-mentioned video types may include but are not limited to at least one of the following: game videos, animation videos, sports competition commentary videos, animal science documentaries, character introduction documentaries, etc.

[0128] In some embodiments of the present application, after the electronic device obtains the scene key information, it can first preprocess the scene key information, and then generate a scene prediction image through the preprocessed scene key information and the Gaussian noise image.

[0129] The following describes the preprocessing process of the electronic device for the scene key information.

[0130] In some embodiments of the present application, when the scene key information is a scene stick figure image, the electronic device may first preprocess the picture resolution of the received scene stick figure image so that the picture resolution of the scene stick figure image is the same as the picture resolution of the first video.

[0131] For example, adjust the image size, image clarity, etc. of the scene stick figure image.

[0132] In some embodiments of the present application, when the scene key information is scene description information, the electronic device can encode the scene description information in text form into input information that can be recognized by the image generation model through the text encoder (Text Encoder) of the Contrastive Language-Image Pre-Training, CLIP), and the input information can be represented as: T_token.

[0133] In some embodiments of the present application, when the scene key information is a scene stick figure image, the electronic device can encode the scene stick figure image in image form into a latent variable feature Z that can be recognized by a subsequent image generation model through an image encoder.

[0134] It should be noted that the electronic device performs diffusion denoising on the Gaussian noise image based on the scene key information to obtain an encoded scene prediction image, and then the electronic device can decode the encoded scene prediction image through an image decoder (Image Decoder) to obtain the final scene prediction image.

[0135] The following describes a process for an electronic device to generate a scene prediction image with reference to a specific example.

[0136] For example, Figure 5 As shown, when the key information of the scene is the scene description information "a scene in which two children are bowing to a female teacher", the electronic device encodes the scene description information "a scene in which two children are bowing to a female teacher" through the CLIP Text Encoder in the Stable Diffsuion model into input information T_token that can be recognized by the aligned subsequent image generation model. Then, the electronic device generates a Gaussian noise image N1 with the same resolution as the picture resolution based on the picture resolution of the first video. Therefore, the electronic device inputs the encoded T_token and the Gaussian noise N1 into the denoising diffusion module in the Stable Diffusion model for multiple iterations to obtain the coded information after diffusion denoising. Furthermore, the electronic device generates a scene prediction image corresponding to the scene description information "a scene in which two children are bowing to a female teacher" through the image decoder Image Decoder based on the coded information after diffusion denoising.

[0137] For example, Figure 6 As shown, the key information in the scene is as follows Figure 4In the case of the stick figure shown in , the electronic device encodes the scene stick figure image in image form into a latent variable feature Z that can be recognized by the subsequent image generation model through the Image Ecoder in the Stable Diffsuion model. Then, the electronic device can adjust the picture resolution of the stick figure image according to the picture resolution of the first video, and generate a Gaussian noise image N2 with the same resolution as the picture resolution. Therefore, the electronic device inputs the latent variable feature Z and the Gaussian noise N2 into the denoising diffusion module in the Stable Diffusion model for multiple iterations to obtain the latent variable adjustment Z_o after diffusion denoising. Further, the electronic device generates a scene prediction image corresponding to the scene stick figure image based on the latent variable adjustment Z_o after diffusion denoising through the image decoder Image Decoder, wherein the scene prediction image shows two children bowing to a female teacher.

[0138] In the embodiment of the present application, since the electronic device performs diffusion denoising on the Gaussian noise image based on the scene key information to obtain the scene prediction image, the picture scene of the Gaussian noise image obtained after the diffusion denoising matches the picture scene indicated by the scene key information. This facilitates the subsequent positioning of the target image frame according to the scene prediction image, and improves the accuracy of the electronic device in positioning the image frame.

[0139] In some embodiments of the present application, in another manner, the electronic device may obtain at least two image blocks based on keywords in the scene key information, and generate a scene prediction image by splicing the at least two image blocks.

[0140] In some embodiments of the present application, the above-mentioned at least two image blocks may include but are not limited to at least one of the following: an image block obtained by the electronic device through image segmentation based on a stored image; an image block obtained by the electronic device through network retrieval of keywords and downloading.

[0141] In some embodiments of the present application, the above-mentioned stored images may include at least one of the following: images stored in the storage space of the electronic device, images stored in the cloud space or cloud disk of the electronic device, and images stored in the server of a third-party application used by the electronic device.

[0142] In some embodiments of the present application, each of the at least two image blocks may include an object.

[0143] In some embodiments of the present application, when the scene key information is scene description information, the electronic device may first perform semantic analysis based on the scene description information, obtain keywords, and match corresponding image blocks for each keyword. Then, the electronic device may perform splicing based on the matched image blocks to generate a scene prediction image.

[0144] For example, when the scene description information is "a scene of two children bowing to a female teacher", the electronic device can first obtain the keywords "two children", "a female teacher" and "bow". Then, the electronic device can splice the image blocks corresponding to each keyword to generate a scene prediction image.

[0145] In some embodiments of the present application, when the scene key information is a scene sketch image, the electronic device may first perform image analysis based on the scene sketch image, obtain keywords, and match corresponding image blocks for each keyword. Then, the electronic device may perform splicing based on the matched image blocks to generate a scene prediction image.

[0146] For example, when the scene stick figure image is an image of two children bowing to a female teacher, the electronic device can first obtain the keywords "two children", "a female teacher" and "bow". Then, the electronic device can splice the image blocks corresponding to each keyword to generate a scene prediction image.

[0147] In some embodiments of the present application, when the scene key information is a scene stick figure image, the electronic device can also directly segment the scene stick figure image to obtain at least two stick figure image blocks; then, the electronic device can match similar image blocks for each stick figure image block, and generate a scene prediction image by splicing the at least two image blocks obtained by the matching.

[0148] Step 302: The electronic device determines an image frame corresponding to a first picture scene in at least one first video as a target image frame.

[0149] Among them, the scene matching degree between the first picture scene and the scene prediction image is greater than a first threshold.

[0150] In some embodiments of the present application, after the electronic device obtains the scene prediction image, it can first obtain the first scene feature information of the scene prediction image. Thus, the electronic device can determine the scene matching degree between the first scene feature information and the second scene feature information of any image frame in the first video, and obtain multiple scene matching degrees, and each scene matching degree corresponds to an image frame in the first video. Then, the electronic device can determine the image frame corresponding to the scene matching degree greater than the first threshold among the multiple scene matching degrees as the target image frame.

[0151] In some embodiments of the present application, the target image frame may be at least one image frame.

[0152] In some embodiments of the present application, the first threshold may be set to any value between 75% and 100%. For example, the first threshold may be 80%. Thus, the electronic device may display all image frames in the first video whose scene matching degree with the scene prediction image is greater than 80%.

[0153] It should be noted that the specific numerical setting can be determined according to the actual needs of the user, and this application does not limit it here.

[0154] In some embodiments of the present application, the first picture scene may also have the highest scene matching degree with the scene prediction image. Therefore, the target image frame is a video frame in the at least one first video that has the highest scene matching degree with the scene prediction image.

[0155] In some embodiments of the present application, the electronic device may first obtain the second scene feature information of any image frame in the first video after playing the first video. In this way, compared with obtaining the second scene feature information after receiving the first input, the solution of obtaining the second scene feature information of the image frame in the first video in advance can improve the search efficiency of the electronic device in searching for the image frame in the first video that matches the scene key information.

[0156] In some embodiments of the present application, both the first scene feature information and the second scene feature information may be formally represented by feature vectors.

[0157] Specifically, the electronic device may calculate the cosine similarity value between the feature vector of the first scene feature information and the feature vector of any second scene feature information through a cosine similarity algorithm, and use the cosine similarity value as the scene matching degree between the first scene feature information and the any second scene feature information.

[0158] It should be noted that the cosine similarity algorithm evaluates the similarity between two feature vectors by measuring the cosine value of the angle between the two feature vectors.

[0159] The following describes a method for calculating the cosine similarity between a feature vector of a first scene feature information and a feature vector of a second scene feature information.

[0160] In some embodiments of the present application, assuming a feature vector A of first scene feature information and a feature vector B corresponding to second scene feature information, the electronic device can calculate the cosine similarity between feature vector A and feature vector B using the following formula (1).

[0161]

[0162] Among them, ||A|| and ||B|| represent the Euclidean norm corresponding to eigenvector A and eigenvector B, that is, the length of the eigenvector; A i represents the i-th vector component in the eigenvector A, B i Represents the i-th eigenvector component in eigenvector B.

[0163] For example, assuming that the eigenvector A is (4,0,5) and the eigenvector B is (5,1,4), then the formula 1 is used to calculate That is, the similarity between the feature vector A and the feature vector B is approximately 0.989.

[0164] In some embodiments of the present application, the electronic device may obtain a feature vector of scene feature information of an image through a deep learning feature extractor network FeatureNet.

[0165] For example, Figure 7 As shown, the electronic device can obtain the feature vector corresponding to each image frame in the first video through FeatureNet in advance when playing the first video, and obtain the query combination key [Img_i, F_i] corresponding to each image frame, where 1≤i≤Num, and Num is the total number of image frames contained in the first video. In addition, after obtaining the scene prediction image based on the scene key information, the electronic device obtains the feature vector F_A corresponding to the scene prediction image Img_A through FeatureNet, and obtains the query combination key [Img_A, F_A] corresponding to the scene prediction image Img_A.

[0166] Then, if Figure 7 As shown, the electronic device can calculate the cosine similarity between F_A and F_i based on the query combination key [Img_A, F_A] corresponding to the scene prediction image Img_A and the query combination key [Img_i, F_i] corresponding to each image frame in the first video to obtain an order of cosine similarity, such as an order from large to small. Thus, the electronic device can determine the feature vector F_m corresponding to the target image frame whose similarity with F_A is greater than the first threshold, where 1≤m≤Num. Then, the electronic device can quickly locate the corresponding image frame Img_m based on F_m, that is, the Img_m is an image frame among the image frames contained in the first video whose similarity with the scene prediction image is greater than the first threshold, that is, the Img_m is the target image frame.

[0167] In the embodiment of the present application, since the scene prediction image is generated based on the scene key information, the picture scene of the scene prediction image can represent the picture scene indicated by the scene key information. Therefore, the higher the scene matching degree between the image frame and the scene prediction image, the higher the similarity between the image frame and the picture scene indicated by the scene key information. This can ensure that the electronic device can more accurately locate the specific image frame in the video.

[0168] In some embodiments of the present application, the scene key information is a scene stick figure image. Figure 2 ,like Figure 8 As shown, after the above step 202, the video positioning method provided in the embodiment of the present application further includes the following steps 401 and 402.

[0169] Step 401: The electronic device receives a second input of a target area in a scene stick figure image.

[0170] In some embodiments of the present application, after the electronic device acquires the scene stick figure image, it can keep playing the first video and display the scene stick figure image. Moreover, when the electronic device displays the target video frame, the electronic device can keep displaying the scene stick figure image, so that the user can input the target area in the stick figure image.

[0171] In some embodiments of the present application, the electronic device may display the scene stick figure image in any of the following forms:

[0172] Displaying the target image frame in full screen, and displaying a floating window on the target image frame, so as to display the stick figure image of the scene through the floating window;

[0173] Displaying the scene stick figure image in full screen, and displaying a floating window on the scene stick figure image, so as to display the target image frame through the floating window;

[0174] The display size of the target image frame is reduced, and the scene stick figure image and the target image frame are displayed in a split-screen display.

[0175] In some embodiments of the present application, the target area may be any image area in a scene stick figure image.

[0176] For example, the target area may be a display area of ​​at least one stick figure object in the scene stick figure image.

[0177] For example, the target area may be a blank area in the scene stick figure image.

[0178] In some embodiments of the present application, the second input is an operation of adding an object or deleting an object in the target area.

[0179] In some embodiments of the present application, the second input may be a touch input of the user in the target area, or may be a selection input of the target area and a target editing control in at least one editing control. Any editing control may include but is not limited to at least one of the following: drawing a control, adding a control, deleting a control, etc.

[0180] The above-mentioned drawing control can be used to additionally draw new stick figure content in the target area of ​​the scene stick figure image, for example, drawing a hat on a character in the scene stick figure image;

[0181] The above-mentioned adding controls can add filters, display effects, stickers, etc. to the target area of ​​the scene stick figure image;

[0182] The above-mentioned delete control can be used to delete the image content in the target area.

[0183] For example, Fig. 9A As shown, the electronic device displays a stick figure image 15 and a target image frame 16. At the same time, three editing controls, such as a "draw" editing control, an "add" editing control, and a "delete" convenient control, can be displayed in the bottom area of ​​the stick figure image 15 to help the user trigger the electronic device to edit the stick figure image 15 by selecting the editing control.

[0184] Step 402: The electronic device updates the picture content corresponding to the target area in the target image frame in response to the second input based on the input parameters of the second input.

[0185] It should be noted that, since the target image frame is determined based on the scene sketch image, there is a corresponding relationship between the scene sketch and the target image frame, and the relative position relationship between the sketch objects in the scene sketch and the objects in the target image frame also has a mapping relationship. Therefore, after the electronic device receives the second input of the target area in the scene sketch image, it can find the target image frame based on the corresponding relationship, and determine the mapping area of ​​the target area in the target image frame based on the mapping relationship. In this way, the electronic device can update the picture corresponding to the target area in the target image frame based on the input parameters of the second input.

[0186] It can be understood that the electronic device can perform the same operation on the target area and the mapping area of ​​the target area in the target image frame, and the operation is determined by the input parameter of the second input.

[0187] In some embodiments of the present application, the input parameters of the second input may include but are not limited to at least one of the following: input position, input direction, input trajectory, etc.

[0188] The following is an exemplary description of step 401 and step 402 with reference to the accompanying drawings.

[0189] For example, Fig. 9A As shown, the electronic device displays a stick figure image 15 and a target image frame 16. The stick figure image 15 includes images of three characters, namely, teacher A1, student B1 and student C1; the target image frame 16 includes images of three characters, namely, teacher A2, student B2 and student C2; the teacher A1 corresponds to the teacher A2, the student B1 corresponds to the student B2, and the student C1 corresponds to the student C2. Fig. 9B As shown, if the user adds a hat to the student B1 in the stick figure image 15, the electronic device can add a hat to the student B2 corresponding to the student B1 in the target image frame 16. The color, shape, material and other parameters of the hat displayed by the electronic device in the stick figure image 15 and the target image frame 16 can be the same or different. In the case where the displayed hats are different, the hat image displayed in the target image frame 16 can be a hat image downloaded by the electronic device through the Internet.

[0190] For example, in combination Fig. 9A , the student C1 in the stick figure image 15 corresponds to the student C2 in the target image frame 16. Fig. 9C As shown, if the user deletes the student C1 in the stick figure image 15 , the electronic device may delete the student C2 corresponding to the student C1 in the target image frame 16 based on the deleted image content.

[0191] In some embodiments of the present application, after the electronic device updates the picture content corresponding to the target area in the target image frame, the electronic device may also perform corresponding content adjustment on the picture content of at least one image frame adjacent to the target image frame in the first video.

[0192] For example, combined with Fig. 9C , assuming that the electronic device adds a hat to the student B2 in the target image frame, the electronic device can add an identical hat to the student B2 displayed in the first two image frames and the last two image frames of the target image frame in the first video.

[0193] In some embodiments of the present application, the at least one image frame adjacent to the target image frame may include at least one of the following: an image frame before the target image frame, and an image frame after the target image frame.

[0194] In some embodiments of the present application, the at least one image frame may be an image frame selected by a user, or may be an image frame automatically determined by the electronic device based on the content of the picture.

[0195] In an embodiment of the present application, the electronic device can edit the content of the target image frame based on the correspondence between the scene stick figure image and the target image frame, thereby improving the flexibility of the electronic device in editing videos.

[0196] The present application embodiment provides a video positioning method. Fig.10 A flowchart of a video positioning method provided by an embodiment of the present application is shown. The following is an example of an exemplary description in which a user triggers an electronic device to display a target image frame through a first input during the playback of video 1, and the first threshold is 80%.

[0197] like Fig.10 As shown, the video positioning method provided in the embodiment of the present application may include the following steps 1 to 12.

[0198] Step 1. The electronic device plays video 1.

[0199] Step 2: The electronic device obtains the picture resolution of video 1 and the feature vectors of all image frames contained in video 1.

[0200] Step 3: During the playback of video 1, the electronic device receives an input of video 1 by the user.

[0201] Step 4: In response to the input, the electronic device displays an interface including an information input box and a playback interface of video 1 in a split-screen format.

[0202] Step 5: The electronic device receives a first input from a user in an information input box for inputting scene description information.

[0203] Step 6: In response to the first input, the electronic device encodes the received scene description information into encoding information T_token recognizable by the electronic device through CLIP Text Encoder in the Stable Diffsuion model.

[0204] Step 7: Based on the picture resolution of video 1, the electronic device generates a Gaussian noise image with the same resolution as the picture resolution.

[0205] Step 8: The electronic device performs diffusion denoising on the Gaussian noise image based on the scene key information through the diffusion denoising module in the Stable Diffsuion model to obtain a scene prediction image.

[0206] Step 9: The electronic device obtains a feature vector of the scene prediction image.

[0207] Step 10: The electronic device calculates cosine similarity based on the feature vector of the scene prediction image and the feature vectors of all image frames included in video 1.

[0208] Step 11: The electronic device sorts the at least one cosine similarity calculated, and determines a value greater than 80% of the at least one cosine similarity.

[0209] Step 12: The electronic device determines a corresponding image frame as a target image frame based on the cosine similarity that meets the conditions.

[0210] In this way, the user's need for quickly locating the video can be met even if the user does not remember the video lines or the video clip scene has no lines.

[0211] The present application embodiment provides a video positioning method. Fig.11 A flow chart of a video positioning method provided by an embodiment of the present application is shown. The following is an example of a user triggering an electronic device to display a target image frame through a first input during the playback of video 1, for exemplary description.

[0212] like Fig.11 As shown, the video positioning method provided in the embodiment of the present application may include the following steps 13 to 27.

[0213] Step 13: The electronic device plays video 1.

[0214] Step 14: The electronic device obtains the picture resolution of the video 1 and the feature vectors of all image frames contained in the video 1.

[0215] Step 15: During the playback of video 1, the electronic device receives an input of video 1 by the user.

[0216] Step 16: In response to the input, the electronic device displays an interface including an information input box and a playback interface of video 1 in a split-screen format.

[0217] Step 17: The electronic device receives a first input from a user in an information input box for inputting a scene stick figure image.

[0218] Step 18: In response to the first input, the electronic device encodes the received scene stick figure image into a latent variable feature Z that can be recognized by the electronic device through the ImageEcoder in the Stable Diffsuion model.

[0219] Step 19: The electronic device generates a Gaussian noise image with the same resolution as the picture resolution of video 1 based on the picture resolution of video 1.

[0220] Step 20: The electronic device performs diffusion denoising on the Gaussian noise image based on the scene key information through the diffusion denoising module in the Stable Diffsuion model to obtain a scene prediction image.

[0221] Step 21: The electronic device obtains a feature vector of a scene prediction image.

[0222] Step 22: The electronic device calculates cosine similarity based on the feature vector of the scene prediction image and the feature vectors of all image frames included in video 1.

[0223] Step 23: The electronic device performs sorting based on the at least one cosine similarity obtained by calculation, and determines a value greater than 80% of the at least one cosine similarity.

[0224] Step 24: The electronic device determines a corresponding image frame as a target image frame based on the cosine similarity that meets the conditions.

[0225] Step 25: The electronic device displays the scene stick figure image and the target image frame in a split-screen format.

[0226] Step 26: The electronic device receives a second input from the user regarding a target area in the scene stick figure image.

[0227] Step 27: The electronic device updates the picture content corresponding to the target area in the target image frame in response to the second input based on the input parameters of the second input.

[0228] In this way, the user's need for quickly locating the video can be met even if the user does not remember the video lines or the video clip scene has no lines.

[0229] It should be noted that the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0230] It should be noted that the video positioning method provided in the embodiment of the present application can be executed by a video positioning device. In the embodiment of the present application, the video positioning device executing the video positioning method is taken as an example to illustrate the video positioning device provided in the embodiment of the present application.

[0231] Fig.12 A possible structural diagram of a video positioning device involved in an embodiment of the present application is shown. Fig.12 As shown, the video positioning device 70 may include: a receiving module 71 and a display module 72 .

[0232] Among them, the receiving module 71 is used to receive a first input, and the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; scene stick figure image; the display module 72 is used to display the target image frame in at least one first video in response to the first input received by the receiving module 71, wherein the picture scene of the target image frame in at least one first video matches the picture scene indicated by the scene key information.

[0233] In the video positioning device provided in the embodiment of the present application, since the video positioning device can directly display the target image frame matching the picture scene indicated by the scene key information according to the scene key information input by the user, without the user having to perform multiple operations on the progress bar, the efficiency of the video positioning device in displaying specific image frames in the video can be improved, and the accuracy of the image frame display can be improved.

[0234] In a possible implementation, the video positioning device 70 provided in the embodiment of the present application also includes: a generation module and a determination module; the above-mentioned generation module is used to generate a scene prediction image based on scene key information before displaying the target image frame in at least one first video; the above-mentioned determination module is used to determine the image frame corresponding to the first picture scene in at least one first video as the target image frame, and the scene matching degree between the first picture scene and the scene prediction image generated by the generation module is greater than the first threshold.

[0235] In a possible implementation, the above-mentioned generation module is specifically used to: generate a Gaussian noise image with the same resolution as the picture resolution based on the picture resolution of the first video; and perform diffusion denoising on the Gaussian noise image based on scene key information to obtain a scene prediction image.

[0236] In one possible implementation, the above-mentioned scene key information is a scene stick figure image; the video positioning device 70 provided in the embodiment of the present application also includes: an update module; the above-mentioned receiving module 71 is also used to receive a second input of a target area in the scene stick figure image after the display module 71 displays the target image frame in at least one first video; the above-mentioned update module is used to update the screen content corresponding to the target area in the target image frame in response to the second input received by the receiving module 71, based on the input parameters of the second input.

[0237] In a possible implementation, the above-mentioned receiving module 71 is specifically used to receive a first input for a second video; the above-mentioned display module 72 is specifically used to display a target image frame in the second video, wherein the picture scene of the target image frame in the second video matches the picture scene indicated by the scene key information.

[0238] The video positioning device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0239] The video positioning device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0240] The video positioning device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0241] Alternatively, if Fig.13 As shown, an embodiment of the present application further provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned video positioning method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0242] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0243] Fig.14 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0244] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0245] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.14 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0246] Here, the electronic device 100 is taken as an example:

[0247] The user input unit 107 is used to receive a first input, where the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; scene stick figure image;

[0248] The display unit 106 is configured to display a target image frame in at least one first video in response to a first input, wherein a picture scene of the target image frame in at least one first video matches the picture scene indicated by the scene key information.

[0249] In the electronic device provided in the embodiment of the present application, since the electronic device can directly display the target image frame matching the picture scene indicated by the scene key information based on the scene key information input by the user, without the user having to perform multiple operations on the progress bar, the efficiency of the electronic device in locating specific image frames in the video can be improved, and the accuracy of the image frame display can be improved.

[0250] Optionally, the processor 110 is used to generate a scene prediction image based on scene key information before displaying a target image frame in at least one first video; and determine an image frame corresponding to a first picture scene in the first video as a target image frame, and a scene matching degree between the first picture scene and the scene prediction image is greater than a first threshold.

[0251] Optionally, the processor 110 is specifically used to generate a Gaussian noise image with a resolution the same as the picture resolution of the first video based on the picture resolution of the first video; and perform diffusion denoising on the Gaussian noise image based on scene key information to obtain a scene prediction image.

[0252] Optionally, the scene key information is a scene stick figure image; the user input unit 107 is further used to receive a second input of a target area in the scene stick figure image after displaying the target image frame in at least one first video; the processor 110 is further used to update the screen content corresponding to the target area in the target image frame in response to the second input based on the input parameters of the second input.

[0253] Optionally, the user input unit 107 is specifically used to receive a first input for a second video; the display unit 106 is specifically used to display a target video frame in at least one second video, wherein the picture scene of the target image frame in the second video matches the picture scene indicated by the scene key information.

[0254] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0255] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0256] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0257] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0258] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0259] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0260] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0261] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0262] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0263] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0264] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0265] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0266] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video positioning method, characterized in that: The method comprises: Receive a first input, where the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; a scene stick figure image; In response to the first input, a target image frame in at least one first video is displayed, wherein a picture scene of the target image frame in the at least one first video matches a picture scene indicated by the scene key information.

2. The method according to claim 1, characterized in that Before displaying the target image frame in at least one first video, the method further includes: Based on the scene key information, generating a scene prediction image; An image frame corresponding to a first picture scene in the at least one first video is determined as the target image frame, and a scene matching degree between the first picture scene and the scene prediction image is greater than a first threshold.

3. The method according to claim 2, characterized in that The generating a scene prediction image based on the scene key information includes: Based on the picture resolution of the first video, generating a Gaussian noise image with a resolution the same as the picture resolution; Based on the scene key information, diffusion denoising is performed on the Gaussian noise image to obtain the scene prediction image.

4. The method according to claim 1, characterized in that The scene key information is a scene stick figure image; After displaying the target image frame in at least one first video, the method further includes: receiving a second input of a target area in the scene stick figure image; In response to the second input, based on input parameters of the second input, the picture content corresponding to the target area in the target image frame is updated.

5. The method according to any one of claims 1 to 4, characterized in that: The receiving a first input comprises: receiving a first input for a second video; The displaying of the target image frame in at least one first video comprises: The target image frame in the second video is displayed, wherein the picture scene of the target image frame in the second video matches the picture scene indicated by the scene key information.

6. A video positioning device, characterized in that: The video positioning device comprises: a receiving module and a display module; The receiving module is used to receive a first input, wherein the first input is used to input scene key information, wherein the scene key information includes at least one of the following: scene description information; scene stick figure image; The display module is used to display a target image frame in at least one first video in response to the first input received by the receiving module, wherein the picture scene of the target image frame in the at least one first video matches the picture scene indicated by the scene key information.

7. The device according to claim 6 or 7, characterized in that The video positioning device also includes: a generation module and a determination module; The generating module is configured to generate a scene prediction image based on the scene key information before the display module displays the target image frame in at least one first video; The determination module is used to determine an image frame corresponding to a first picture scene in the at least one first video as the target image frame, and the scene matching degree between the first picture scene and the scene prediction image generated by the generation module is greater than a first threshold.

8. The device according to claim 7, characterized in that The generating module is specifically used for: Based on the picture resolution of the first video, generating a Gaussian noise image with a resolution the same as the picture resolution; Based on the scene key information, diffusion denoising is performed on the Gaussian noise image to obtain the scene prediction image.

9. The device according to claim 6, characterized in that The scene key information is a scene stick figure image; The video positioning device further includes: an updating module; The receiving module is further configured to receive a second input of a target area in the scene stick figure image after the display module displays the target image frame in at least one first video; The updating module is used to update the picture content corresponding to the target area in the target image frame based on the input parameters of the second input in response to the second input received by the receiving module.

10. The method according to any one of claims 6 to 9, characterized in that: The receiving module is specifically configured to receive a first input for a second video; The display module is specifically used to display the target image frame in the second video, wherein the picture scene of the target image frame in the second video matches the picture scene indicated by the scene key information.

Images