Method and device for identifying close-up space shots, storage medium and electronic device

Abstract

This application relates to a method, apparatus, storage medium, and electronic device for recognizing close-up establishing shots. The method includes: acquiring all shot segments of a target video and extracting the shot image of each frame of each shot segment; inputting all shot images into a trained pre-model to obtain a first recognition result for each shot segment; inputting the target shot images of all first shot segments into a trained close-up establishing shot model to obtain a second recognition result for each first shot segment, wherein the first shot segment is the shot segment whose first recognition result is a close-up establishing shot, and the target shot image is multiple shot images extracted from all shot images of the corresponding shot segment; if the second recognition result of a second shot segment is a close-up establishing shot, then the second shot segment is determined to be a close-up establishing shot. This application solves the technical problem that traditional algorithms cannot effectively recognize close-up establishing shots.

Description

Technical Field

[0001] This application relates to the field of empty shot algorithm recognition technology, and in particular to a method, device, storage medium and electronic device for recognizing close-up empty shots. Background Technology

[0002] In the field of establishing shot recognition, there is a type of close-up shot, primarily used to highlight detailed descriptions of animals, objects, environments, buildings, etc. Effective recognition of these close-up establishing shots is crucial to reduce the workload and cost of manual screening. However, because close-up establishing shots represent a kind of cinematic language and are relatively abstract, traditional algorithms cannot directly use convolutional networks to extract features from them, and therefore cannot cover all categories, making effective recognition of close-up establishing shots impossible. Summary of the Invention

[0003] This application provides a method, apparatus, storage medium, and electronic device for identifying close-up empty shots, in order to solve the technical problem that traditional algorithms cannot effectively identify close-up empty shots.

[0004] In a first aspect, this application provides a method for identifying close-up aerial shots, comprising: acquiring all shot segments of a target video and extracting the shot image of each frame of each shot segment; inputting all shot images into a trained pre-model to obtain a first identification result for each of the aforementioned shot segments, wherein the first identification result includes close-up aerial shots and non-close-up aerial shots; inputting the target shot images of all the first shot segments into a trained close-up aerial model to obtain a second identification result for each of the first shot segments, wherein the first shot segment is a shot segment for which the first identification result is a close-up aerial shot, and the target shot image is multiple shot images extracted from all shot images of the corresponding shot segment, and the second identification result includes close-up aerial shots and non-close-up aerial shots; and determining that the second shot segment is a close-up aerial shot when the second identification result of the second shot segment is a close-up aerial shot, wherein the second shot segment is any shot segment among all the first shot segments.

[0005] Secondly, this application provides a device for recognizing close-up aerial shots, comprising: a first acquisition module, configured to acquire all shot segments of a target video and extract the shot image of each frame of each shot segment; a first recognition module, configured to input all shot images into a trained pre-model to obtain a first recognition result for each of the aforementioned shot segments, wherein the first recognition result includes close-up aerial shots and non-close-up aerial shots; a second recognition module, configured to input the target shot images of all first shot segments into a trained close-up aerial model to obtain a second recognition result for each first shot segment, wherein the aforementioned first shot segment is a shot segment for which the first recognition result is a close-up aerial shot, and the target shot image is multiple shot images extracted from all shot images of the corresponding shot segment, and the aforementioned second recognition result includes close-up aerial shots and non-close-up aerial shots; and a determination module, configured to determine that the aforementioned second shot segment is a close-up aerial shot when the aforementioned second recognition result of the second shot segment is a close-up aerial shot, wherein the aforementioned second shot segment is any shot segment among all the first shot segments.

[0006] As an optional example, the first acquisition module includes: an acquisition unit for acquiring the target video; a reading unit for reading the target video frame by frame, detecting and marking the shot positions of the target video; and a segmentation unit for segmenting the target video into multiple shot segments based on the shot positions, wherein each shot segment consists of video frames between two consecutive shot positions.

[0007] As an optional example, the above apparatus further includes: a second acquisition module, configured to acquire the trained close-up aerial scene model and a first training dataset before inputting all the shot images into the trained pre-model to obtain the first recognition result for each of the above shot segments; a first processing module, configured to input the first training dataset into the trained close-up aerial scene model, so that the trained close-up aerial scene model performs data cleaning and classification on the first training dataset to obtain a close-up aerial scene training dataset and a non-close-up aerial scene training dataset; and a training module, configured to train the pre-model using the close-up aerial scene training dataset and the non-close-up aerial scene training dataset to obtain the trained pre-model.

[0008] As an optional example, the above apparatus further includes: an adding module for adding a low-rank matrix factorization layer to the large-scale close-up spatial model before acquiring the trained large-scale close-up spatial model; a third acquiring module for acquiring a second training dataset and input prompts, and inputting the second training dataset and the input prompts into the large-scale close-up spatial model to obtain a prediction result dataset; a calculation module for calculating a loss value based on the second training dataset and the prediction result dataset; and an updating module for updating the parameters of the low-rank matrix factorization layer based on the loss value to obtain the trained large-scale close-up spatial model.

[0009] As an optional example, the first identification module includes: a first processing unit, configured to determine each segment of all shot clips as the current shot clip, and perform the following operations on the current shot clip: perform close-up aerial image prediction on each shot image of the current shot clip to determine whether each shot image is a close-up aerial image; count the first number of all shot images in the current shot clip and the second number of close-up aerial images in the current shot clip; divide the second number by the first number to obtain the first proportion of close-up aerial images in the current shot clip; if the first proportion of close-up aerial images is greater than or equal to a first target threshold, determine the first identification result of the current shot clip as a close-up aerial shot; if the first proportion of close-up aerial images is less than the first target threshold, determine the first identification result of the current shot clip as a non-close-up aerial shot.

[0010] As an optional example, the above-described apparatus further includes: a second processing module, configured to, before inputting the target shot images of all first shot segments into a trained close-up spatial model to obtain a second recognition result for each first shot segment, determine each segment of the first shot segments as a current shot segment, and perform the following operations on the current shot segment: divide the current shot segment into multiple partial shot segments on an average basis; extract a shot image from each partial shot segment to obtain the target shot image of the current shot segment.

[0011] As an optional example, the second recognition module includes: a second processing unit, configured to determine each segment of the first shot fragment as the current shot fragment, and perform the following operations on the current shot fragment: perform close-up aerial image prediction on each target shot image of the current shot fragment, and determine whether each target shot image is a close-up aerial image; count the third number of all target shot images in the current shot fragment, and the fourth number of all target shot images that are close-up aerial images; divide the fourth number by the third number to obtain the second close-up aerial image proportion of the current shot fragment; if the second close-up aerial image proportion is greater than or equal to a second target threshold, determine that the second recognition result of the current shot fragment is a close-up aerial shot; if the second close-up aerial image proportion is less than the second target threshold, determine that the second recognition result of the current shot fragment is a non-close-up aerial shot.

[0012] Thirdly, this application provides a storage medium storing a computer program, wherein the computer program is executed by a processor to perform the above-described method for recognizing close-up aerial shots.

[0013] Fourthly, this application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the aforementioned method for recognizing close-up aerial shots through the computer program.

[0014] In this embodiment, all shot segments of the target video are acquired, and each frame of each shot segment is extracted. All shot images are input into a pre-trained model to obtain a first recognition result for each shot segment, wherein the first recognition result includes close-up shots of empty space and non-close-up shots of empty space. The target shot images of all first shot segments are input into a pre-trained close-up shot model to obtain a second recognition result for each first shot segment, wherein the first shot segment is the shot segment for which the first recognition result is a close-up shot of empty space, and the target shot image is multiple shot images extracted from all shot images of the corresponding shot segment. The second recognition result includes close-up shots of empty space and non-close-up shots of empty space. The second recognition result of the second shot segment... In the case of a close-up empty shot, the method of determining the aforementioned second shot segment as a close-up empty shot, wherein the aforementioned second shot segment is any segment of all the first shot segments, is as follows: In the above method, a pre-trained pre-model is first used to preliminarily filter the close-up empty shot segments in the input target video; then, based on the pre-trained pre-model, the shot segments identified as close-up empty shots are input into the pre-trained close-up empty shot large model to determine whether they are close-up empty shots; finally, the recognition results of the pre-trained pre-model and the pre-trained close-up empty shot large model are fused to obtain the final recognition result. This achieves the purpose of effectively reducing the algorithm inference time, reducing memory usage, reducing costs, and improving processing efficiency, thereby solving the technical problem that traditional algorithms cannot effectively recognize close-up empty shots. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0018] Figure 1 This is a flowchart of an optional method for identifying close-up aerial shots according to an embodiment of this application;

[0019] Figure 2This is a flowchart of the pre-model training process for an optional close-up aerial shot recognition method according to an embodiment of this application;

[0020] Figure 3 This is a flowchart of a large-scale model training method for an optional close-up aerial shot recognition method according to an embodiment of this application.

[0021] Figure 4 This is an overall implementation flowchart of an optional close-up aerial shot recognition method according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an optional close-up aerial shot recognition device according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0026] According to a first aspect of the embodiments of this application, a method for identifying close-up aerial shots is provided, optionally, as follows: Figure 1 As shown, the above method includes:

[0027] S102, acquire all shot segments of the target video, and extract the shot image of each frame of each shot segment;

[0028] S104, input all the shot images into the trained pre-model to obtain the first recognition result of each shot segment, wherein the first recognition result includes close-up shots of the empty space and non-close-up shots of the empty space.

[0029] S106, input all target shot images of the first shot segments into the trained close-up air scene model to obtain the second recognition result of each first shot segment. The first shot segment is the shot segment whose first recognition result is a close-up air scene shot. The target shot image is multiple shot images extracted from all shot images of the corresponding shot segment. The second recognition result includes close-up air scene shots and non-close-up air scene shots.

[0030] S108, if the second identification result of the second shot segment is a close-up empty shot, the second shot segment is determined to be a close-up empty shot, wherein the second shot segment is any shot segment among all the first shot segments.

[0031] Optionally, in this embodiment, the pre-trained model can be a Convolutional Neural Network (CNN), primarily used in image and video recognition, image classification, and image analysis. A Large Language Model (LLM) refers to a deep learning model with a large number of parameters capable of processing and generating natural language text. It is trained on massive amounts of text data to understand and generate syntactically and semantically consistent natural language text. The trained close-up spatial model is obtained by fine-tuning some parameters of the large-scale language model using the LoRA (Low-Rank Adaptation) method. LoRA is a method for efficiently fine-tuning large-scale pre-trained models, reducing the number of parameters that need to be updated through low-rank matrix factorization, thereby reducing computational resources and storage requirements.

[0032] Optionally, in this embodiment, existing shot segmentation algorithms or tools are used to identify different shot transition points in the target video, segmenting the target video into multiple shot segments. Each frame within a shot segment is traversed to extract shot images from each frame of that segment. These extracted shot images are then input into a pre-trained pre-model, which performs initial classification of these frames to obtain a first identification result for each shot segment. This first identification result classifies the shot segment as either a "close-up empty shot" or a "non-close-up empty shot." For shot segments initially identified as "close-up empty shots" by the pre-model, multiple target shot images are extracted from each of these shot segments. These target shot images are keyframes selected from the corresponding shot segment or several frames uniformly extracted. The extracted target shot images are then input into a pre-trained close-up empty shot model, which further classifies and identifies these target shot images to obtain a second identification result for each shot segment, i.e., whether the shot segment corresponding to these target shot images is a "close-up empty shot." If the second identification result indicates that a certain shot segment is identified as a "close-up empty shot", then the shot segment is finally determined to be a "close-up empty shot".

[0033] Optionally, in this embodiment, a two-stage recognition process (preliminary model and close-up aerial scene model) can improve recognition accuracy. The preliminary model is responsible for initial screening, while the close-up aerial scene model performs fine classification. The preliminary model can quickly filter out a large number of irrelevant shot segments, only performing detailed analysis on potential close-up aerial scene shot segments, thereby improving overall processing efficiency. Furthermore, the above method fully utilizes all frame image information in the target video to ensure comprehensive and accurate recognition results.

[0034] Optionally, in this embodiment, the pre-trained pre-model is first used to preliminarily filter close-up empty shot segments in the input target video. Then, the close-up empty shot segments identified as close-up empty shots by the pre-trained pre-model are input into the trained close-up empty shot large model to determine whether they are close-up empty shots. Finally, the recognition results of the pre-trained pre-model and the trained close-up empty shot large model are fused to obtain the final recognition result. This effectively reduces the algorithm inference time, reduces memory usage, reduces costs, and improves processing efficiency, thereby solving the technical problem that traditional algorithms cannot effectively recognize close-up empty shots.

[0035] As an optional example, obtaining all shot clips of the target video includes:

[0036] Acquire the target video;

[0037] Read the target video frame by frame, and detect and mark the camera positions in the target video;

[0038] Based on the camera positions, the target video is divided into multiple shot segments, where each shot segment consists of video frames between two consecutive camera positions.

[0039] Optionally, in this embodiment, to obtain all shot segments of the target video, it is necessary to detect and mark the shot locations in the target video, and then segment the target video into multiple shot segments based on the shot locations. Shot locations, also known as scene transition points, are the transition points in the video from one shot to another. The method for detecting shot locations is to calculate the difference in color histograms between adjacent frames; when the difference exceeds a certain threshold, it is marked as a shot location. Specifically, each frame of the target video is read, and each frame can be analyzed by reading it frame by frame. Once a shot location is detected, it is marked, and its frame number or timestamp is recorded, indicating that the point is a scene transition point. Based on the marked shot locations, the target video is segmented into multiple shot segments, each shot segment consisting of video frames between two consecutive shot locations. For example, if the shot locations are frame 1, frame 50, and frame 100, the video can be segmented into two segments: frame 1 to frame 50 and frame 51 to frame 100.

[0040] Optionally, in this embodiment, by analyzing and detecting frame by frame, the shot switching points can be accurately identified, achieving high-precision video segmentation, which greatly facilitates subsequent video analysis and processing.

[0041] As an alternative example, before inputting all the shot images into the trained pre-model to obtain the first recognition result for each shot segment, the above method also includes:

[0042] Obtain the trained close-up spatial model and the first training dataset;

[0043] The first training dataset is input into the trained close-up aerial scene model so that the trained close-up aerial scene model can perform data cleaning and classification on the first training dataset to obtain the close-up aerial scene training dataset and the non-close-up aerial scene training dataset.

[0044] The pre-trained model is obtained by training the training dataset of close-up shots and non-close-up shots.

[0045] Optionally, in this embodiment, before inputting all shot images into the trained pre-model to obtain the first recognition result for each shot segment, the pre-model needs to be trained based on the trained close-up spatial model and a massive first training dataset to obtain the trained pre-model. Specifically, as shown... Figure 2The flowchart shown above illustrates the pre-training process of the pre-trained close-up aerial scene model. The first training dataset, which includes a massive amount of online data, is cleaned to achieve the purpose of classifying and labeling aerial scene images. The pre-training model is then trained using the classified close-up aerial scene training dataset. After feature extraction, the pre-training model adds a cross-attention mechanism to enhance the representation ability of the features, thus completing the pre-training process.

[0046] As an alternative example, the above method also includes the following steps before obtaining the trained close-up spatial large model:

[0047] Add a low-rank matrix factorization layer to the close-up spatial large model;

[0048] Obtain the second training dataset and input prompts, and input the second training dataset and input prompts into the close-up spatial large model to obtain the prediction result dataset;

[0049] Calculate the loss value based on the second training dataset and the prediction result dataset;

[0050] The parameters of the low-rank matrix factorization layer are updated based on the loss value to obtain the trained close-up spatial large model.

[0051] Optionally, in this embodiment, before training the pre-trained close-up spatial model using the trained large-scale close-up model, the large-scale close-up model needs to be trained to obtain the trained large-scale close-up model. Fine-tuning the large-scale language model, i.e., the large-scale close-up model, is done using LoRA. Using LoRA for model fine-tuning reduces the amount of data labeling and the cost of data labeling; only a small batch of data is needed for the second training dataset to achieve good recognition results. LoRA (Low-Rank Adaptation) is a method for efficiently fine-tuning large-scale pre-trained models. It reduces the number of parameters that need to be updated through low-rank matrix factorization, thereby reducing computational resources and storage requirements. Specifically, as... Figure 3 The flowchart shown illustrates the training process for a large-scale close-up empty scene model. The training process uses the LoRA method to fine-tune only some parameters of the large-scale close-up empty scene model, for example, fine-tuning about 30% of the parameters. The input prompt format during fine-tuning is as follows: {"instruction":"The above is an image I input. Please judge whether it is a building. You only need to answer 'it is a building' or 'it is not a building'","output":"it is a building"}. Through fine-tuning, a large-scale language model specific to close-up empty scenes can be obtained.

[0052] Optionally, in this embodiment, a low-rank matrix factorization (LMF) layer is inserted at an appropriate position in the close-up spatial context large model. This decomposes the original weight matrix into the product of two low-rank matrices, reducing the number of parameters. The second training dataset and input cues are fed into the close-up spatial context large model, which processes these inputs and generates a prediction result dataset. The difference between the model's prediction result dataset and the actual labels is calculated using a loss function. Based on the loss value, the gradient is calculated using the backpropagation algorithm, and the parameters of the LMF layer are updated. After multiple iterations of training, the parameters of the LMF layer are continuously optimized, ultimately resulting in a trained close-up spatial context large model.

[0053] As an optional example, all shot images are input into a pre-trained model to obtain the first recognition result for each shot segment, including:

[0054] Define each segment of all shot clips as the current shot clip, and perform the following operations on the current shot clip:

[0055] For each shot image in the current shot segment, perform close-up empty scene image prediction to determine whether each shot image is a close-up empty scene image;

[0056] Count the first number of all shot images in the current shot segment, and the second number of close-up empty shots in the current shot segment;

[0057] Divide the second quantity by the first quantity to obtain the percentage of the first close-up empty image in the current shot segment;

[0058] If the proportion of the first close-up aerial image is greater than or equal to the first target threshold, the first recognition result of the current shot segment is determined to be a close-up aerial shot;

[0059] If the proportion of the first close-up aerial image is less than the first target threshold, the first identification result of the current shot segment is determined to be a non-close-up aerial shot.

[0060] Optionally, in this embodiment, all shot images are input into a trained pre-model to obtain the first recognition result for each shot segment. Taking a specific shot segment as an example, the process is as follows: each frame of the shot segment is input into the trained pre-model. For each frame, the model outputs a prediction result, determining whether the shot image is a close-up aerial image. The number of all shot images in this shot segment is counted and recorded as the first number, which represents the total number of frames in the shot segment. The number of shots predicted to be close-up aerial images in this shot segment is counted and recorded as the second number. The ratio of the second number to the first number is calculated, i.e., the first proportion of close-up aerial images. This ratio represents the proportion of close-up aerial images in this shot segment. The calculated first proportion of close-up aerial images is compared with a pre-set first target threshold. If the proportion is greater than or equal to the first target threshold, the first recognition result of this shot segment is determined to be a close-up aerial shot. If the proportion is less than the first target threshold, the first recognition result of this shot segment is determined to be a non-close-up aerial shot. The first recognition results for all shot segments are obtained through the above method.

[0061] Optionally, in this embodiment, predicting close-up aerial images frame by frame and determining the category of the entire shot segment by statistical proportion can improve the accuracy and reliability of recognition. By calculating the proportion instead of judging a single frame, noise and random errors can be effectively resisted, improving the robustness of the model in practical applications.

[0062] As an optional example, before inputting the target shot images of all first shot segments into a trained close-up spatial model to obtain the second recognition result for each first shot segment, the above method further includes:

[0063] Define each segment of the first-shot sequence as the current shot segment, and perform the following operations on the current shot segment:

[0064] Divide the current shot segment into multiple equal parts;

[0065] Extract one frame from each segment of the shot to obtain the target shot image of the current shot segment.

[0066] Optionally, in this embodiment, multiple target shot images are extracted from each of the first shot segments as recognition data input to the trained close-up spatial model. Taking a certain shot segment as an example, the entire shot segment is divided into multiple partial shot segments. Assuming the shot segment contains N frames, it is divided into K segments on average, and the frame count of each segment is calculated as A = N / K (rounded down). For the last partial shot segment, if the number of frames is not divisible by K, it may contain slightly more or slightly fewer frames to ensure that all frames are included. In each partial shot segment, one shot image is selected as a representative. This can be done using various strategies, such as: extracting the middle frame of the partial shot segment; extracting the first or last frame of the partial shot segment; or randomly selecting any frame from the partial shot segment. The set of frame images extracted from each partial shot segment constitutes the target shot image for this shot segment. Assuming this shot segment is divided into K segments, the set of target shot images will contain K images.

[0067] Optionally, in this embodiment, by dividing the entire shot segment into multiple partial shot segments and extracting frame images from them, the diversity and variation within the shot segment can be better captured, ensuring that the selected images are more representative. Extracting only a small number of frame images for subsequent processing significantly reduces the computational burden and improves processing efficiency compared to processing all frames of the entire shot segment.

[0068] As an optional example, the target shot images of all first shot segments are input into a trained close-up spatial model to obtain the second recognition result for each first shot segment, including:

[0069] Define each segment of the first-shot sequence as the current shot segment, and perform the following operations on the current shot segment:

[0070] For each target shot image in the current shot segment, perform close-up aerial image prediction to determine whether each target shot image is a close-up aerial image;

[0071] Count the third number of all target shot images in the current shot segment, and the fourth number of close-up empty shots among all target shot images;

[0072] Divide the fourth quantity by the third quantity to obtain the percentage of the second close-up empty image in the current shot segment;

[0073] If the proportion of the second close-up aerial image is greater than or equal to the second target threshold, the second recognition result of the current shot segment is determined to be a close-up aerial shot;

[0074] If the proportion of the second close-up aerial image is less than the second target threshold, the second recognition result of the current shot segment is determined to be a non-close-up aerial shot.

[0075] Optionally, in this embodiment, all target shot images of the first shot segment are input into a trained close-up aerial scene model to obtain a second recognition result for each first shot segment. Taking a certain first shot segment as an example, the specific process is as follows: each target shot image in this shot segment is input into the trained close-up aerial scene model. The model classifies each target shot image and determines whether it is a close-up aerial scene image. For each target shot image, the model's prediction result (whether it is a close-up aerial scene image or not) is recorded. The total number of all target shot images in this first shot segment is counted and recorded as the third number, which is the total number of target images extracted in this first shot segment. The number of target shot images predicted as close-up aerial scenes in this first shot segment is counted and recorded as the fourth number. The ratio of the fourth number to the third number is calculated, which is the second proportion of close-up aerial scenes. This ratio represents the proportion of close-up aerial scenes in this first shot segment. The calculated proportion of the second close-up aerial image is compared with a pre-set second target threshold. If the proportion is greater than or equal to the second target threshold, the second recognition result of this first shot segment is determined to be a close-up aerial shot. If the proportion is less than the second target threshold, the second recognition result of this first shot segment is determined to be a non-close-up aerial shot.

[0076] To illustrate this with an example, this application proposes a method for recognizing close-up empty shots to address the problem that traditional algorithms cannot effectively identify abstract categories like close-up empty shots. This method leverages the powerful generalization ability of large-scale language models to identify close-up empty shots in videos, significantly improving the algorithm's recall, reducing false positives, and thus improving the overall recognition accuracy. By combining the full dataset with a pre-trained model and a large-scale model of close-up empty shots with partial data, the method ensures that inference speed and memory usage meet business requirements while integrating the recognition results of small-scale pre-trained models and large-scale models, thereby improving the algorithm's recognition accuracy and effectively reducing false positives.

[0077] The specific implementation steps are as follows: Figure 4 The overall implementation flowchart shown below:

[0078] Step 1: Input the entire video, detect and mark the camera positions in the video, and then divide the video into multiple camera segments based on the camera positions.

[0079] Step 2: Input each frame of all the shot images into the pre-trained model to obtain the first recognition result of whether each shot is a close-up empty shot, including: determining whether each shot image in each shot is a close-up empty image, calculating the proportion of the first close-up empty image in each shot, setting an appropriate first target threshold, and if the proportion of the first close-up empty image exceeds the first target threshold, then the shot is considered to be a close-up empty shot.

[0080] Step 3: Based on the selected close-up empty shot segments, randomly select several frames, generally one frame each from the beginning, middle and end of the time distribution of the shot segment, and input them into the trained close-up empty shot model to obtain the second recognition result of whether each shot segment is a close-up empty shot.

[0081] Step 4: Merge the first and second recognition results. Only when the first and second recognition results are consistent is the shot clip a true close-up establishing shot. Output all close-up establishing shots.

[0082] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0083] According to another aspect of the embodiments of this application, a device for recognizing close-up aerial shots is also provided, such as... Figure 5 As shown, it includes:

[0084] The first acquisition module 502 is used to acquire all the shot segments of the target video and extract the shot image of each frame of each shot segment;

[0085] The first recognition module 504 is used to input all the shot images into the trained pre-model to obtain the first recognition result of each shot segment, wherein the first recognition result includes close-up shots of the empty space and non-close-up shots of the empty space.

[0086] The second recognition module 506 is used to input the target shot images of all first shot segments into the trained close-up air scene model to obtain the second recognition result of each first shot segment. The first shot segment is the shot segment whose first recognition result is a close-up air scene shot. The target shot image is multiple shot images extracted from all shot images of the corresponding shot segment. The second recognition result includes close-up air scene shots and non-close-up air scene shots.

[0087] The determining module 508 is used to determine that the second shot segment is a close-up empty shot when the second identification result of the second shot segment is a close-up empty shot, wherein the second shot segment is any shot segment among all the first shot segments.

[0088] Optionally, in this embodiment, the pre-trained model can be a Convolutional Neural Network (CNN), primarily used in image and video recognition, image classification, and image analysis. A Large Language Model (LLM) refers to a deep learning model with a large number of parameters capable of processing and generating natural language text. It is trained on massive amounts of text data to understand and generate syntactically and semantically consistent natural language text. The trained close-up spatial model is obtained by fine-tuning some parameters of the large-scale language model using the LoRA (Low-Rank Adaptation) method. LoRA is a method for efficiently fine-tuning large-scale pre-trained models, reducing the number of parameters that need to be updated through low-rank matrix factorization, thereby reducing computational resources and storage requirements.

[0089] Optionally, in this embodiment, existing shot segmentation algorithms or tools are used to identify different shot transition points in the target video, segmenting the target video into multiple shot segments. Each frame within a shot segment is traversed to extract shot images from each frame of that segment. These extracted shot images are then input into a pre-trained pre-model, which performs initial classification of these frames to obtain a first identification result for each shot segment. This first identification result classifies the shot segment as either a "close-up empty shot" or a "non-close-up empty shot." For shot segments initially identified as "close-up empty shots" by the pre-model, multiple target shot images are extracted from each of these shot segments. These target shot images are keyframes selected from the corresponding shot segment or several frames uniformly extracted. The extracted target shot images are then input into a pre-trained close-up empty shot model, which further classifies and identifies these target shot images to obtain a second identification result for each shot segment, i.e., whether the shot segment corresponding to these target shot images is a "close-up empty shot." If the second identification result indicates that a certain shot segment is identified as a "close-up empty shot", then the shot segment is finally determined to be a "close-up empty shot".

[0090] Optionally, in this embodiment, a two-stage recognition process (preliminary model and close-up aerial scene model) can improve recognition accuracy. The preliminary model is responsible for initial screening, while the close-up aerial scene model performs fine classification. The preliminary model can quickly filter out a large number of irrelevant shot segments, only performing detailed analysis on potential close-up aerial scene shot segments, thereby improving overall processing efficiency. Furthermore, the above method fully utilizes all frame image information in the target video to ensure comprehensive and accurate recognition results.

[0091] Optionally, in this embodiment, the pre-trained pre-model is first used to preliminarily filter close-up empty shot segments in the input target video. Then, the close-up empty shot segments identified as close-up empty shots by the pre-trained pre-model are input into the trained close-up empty shot large model to determine whether they are close-up empty shots. Finally, the recognition results of the pre-trained pre-model and the trained close-up empty shot large model are fused to obtain the final recognition result. This effectively reduces the algorithm inference time, reduces memory usage, reduces costs, and improves processing efficiency, thereby solving the technical problem that traditional algorithms cannot effectively recognize close-up empty shots.

[0092] As an optional example, the first acquisition module includes:

[0093] The acquisition unit is used to acquire the target video;

[0094] The reading unit is used to read the target video frame by frame, detect and mark the camera positions in the target video;

[0095] The segmentation unit is used to divide the target video into multiple shot segments based on the shot position. Each shot segment consists of video frames between two consecutive shot positions.

[0096] Optionally, in this embodiment, to obtain all shot segments of the target video, it is necessary to detect and mark the shot locations in the target video, and then segment the target video into multiple shot segments based on the shot locations. Shot locations, also known as scene transition points, are the transition points in the video from one shot to another. The method for detecting shot locations is to calculate the difference in color histograms between adjacent frames; when the difference exceeds a certain threshold, it is marked as a shot location. Specifically, each frame of the target video is read, and each frame can be analyzed by reading it frame by frame. Once a shot location is detected, it is marked, and its frame number or timestamp is recorded, indicating that the point is a scene transition point. Based on the marked shot locations, the target video is segmented into multiple shot segments, each shot segment consisting of video frames between two consecutive shot locations. For example, if the shot locations are frame 1, frame 50, and frame 100, the video can be segmented into two segments: frame 1 to frame 50 and frame 51 to frame 100.

[0097] Optionally, in this embodiment, by analyzing and detecting frame by frame, the shot switching points can be accurately identified, achieving high-precision video segmentation, which greatly facilitates subsequent video analysis and processing.

[0098] As an optional example, the device also includes:

[0099] The second acquisition module is used to acquire the trained close-up spatial model and the first training dataset before inputting all the shot images into the trained pre-model and obtaining the first recognition result of each shot segment.

[0100] The first processing module is used to input the first training dataset into the trained close-up aerial scene model so that the trained close-up aerial scene model can perform data cleaning and classification on the first training dataset to obtain the close-up aerial scene training dataset and the non-close-up aerial scene training dataset.

[0101] The training module is used to train the pre-model using a training dataset of close-up and non-close-up shots of empty space, resulting in a trained pre-model.

[0102] Optionally, in this embodiment, before inputting all shot images into the trained pre-model to obtain the first recognition result for each shot segment, the pre-model needs to be trained based on the trained close-up spatial model and a massive first training dataset to obtain the trained pre-model. Specifically, as shown... Figure 2 The flowchart shown above illustrates the pre-training process of the pre-trained close-up aerial scene model. The first training dataset, which includes a massive amount of online data, is cleaned to achieve the purpose of classifying and labeling aerial scene images. The pre-training model is then trained using the classified close-up aerial scene training dataset. After feature extraction, the pre-training model adds a cross-attention mechanism to enhance the representation ability of the features, thus completing the pre-training process.

[0103] As an optional example, the device also includes:

[0104] Add a module to add a low-rank matrix factorization layer to the large close-up spatial model before acquiring the trained large close-up spatial model;

[0105] The third acquisition module is used to acquire the second training dataset and input prompts, and input the second training dataset and input prompts into the close-up spatial large model to obtain the prediction result dataset.

[0106] The calculation module is used to calculate the loss value based on the second training dataset and the prediction result dataset;

[0107] The update module is used to update the parameters of the low-rank matrix factorization layer based on the loss value, so as to obtain the trained close-up spatial large model.

[0108] Optionally, in this embodiment, before training the pre-trained close-up spatial model using the trained large-scale close-up model, the large-scale close-up model needs to be trained to obtain the trained large-scale close-up model. Fine-tuning the large-scale language model, i.e., the large-scale close-up model, is done using LoRA. Using LoRA for model fine-tuning reduces the amount of data labeling and the cost of data labeling; only a small batch of data is needed for the second training dataset to achieve good recognition results. LoRA (Low-Rank Adaptation) is a method for efficiently fine-tuning large-scale pre-trained models. It reduces the number of parameters that need to be updated through low-rank matrix factorization, thereby reducing computational resources and storage requirements. Specifically, as... Figure 3 The flowchart shown illustrates the training process for a large-scale close-up empty scene model. The training process uses the LoRA method to fine-tune only some parameters of the large-scale close-up empty scene model, for example, fine-tuning about 30% of the parameters. The input prompt format during fine-tuning is as follows: {"instruction":"The above is an image I input. Please judge whether it is a building. You only need to answer 'it is a building' or 'it is not a building'","output":"it is a building"}. Through fine-tuning, a large-scale language model specific to close-up empty scenes can be obtained.

[0109] Optionally, in this embodiment, a low-rank matrix factorization (LMF) layer is inserted at an appropriate position in the close-up spatial context large model. This decomposes the original weight matrix into the product of two low-rank matrices, reducing the number of parameters. The second training dataset and input cues are fed into the close-up spatial context large model, which processes these inputs and generates a prediction result dataset. The difference between the model's prediction result dataset and the actual labels is calculated using a loss function. Based on the loss value, the gradient is calculated using the backpropagation algorithm, and the parameters of the LMF layer are updated. After multiple iterations of training, the parameters of the LMF layer are continuously optimized, ultimately resulting in a trained close-up spatial context large model.

[0110] As an optional example, the first recognition module includes:

[0111] The first processing unit is used to determine each segment of all shot clips as the current shot clip, and to perform the following operations on the current shot clip:

[0112] For each shot image in the current shot segment, perform close-up empty scene image prediction to determine whether each shot image is a close-up empty scene image;

[0113] Count the first number of all shot images in the current shot segment, and the second number of close-up empty shots in the current shot segment;

[0114] Divide the second quantity by the first quantity to obtain the percentage of the first close-up empty image in the current shot segment;

[0115] If the proportion of the first close-up aerial image is greater than or equal to the first target threshold, the first recognition result of the current shot segment is determined to be a close-up aerial shot;

[0116] If the proportion of the first close-up aerial image is less than the first target threshold, the first identification result of the current shot segment is determined to be a non-close-up aerial shot.

[0117] Optionally, in this embodiment, all shot images are input into a trained pre-model to obtain the first recognition result for each shot segment. Taking a specific shot segment as an example, the process is as follows: each frame of the shot segment is input into the trained pre-model. For each frame, the model outputs a prediction result, determining whether the shot image is a close-up aerial image. The number of all shot images in this shot segment is counted and recorded as the first number, which represents the total number of frames in the shot segment. The number of shots predicted to be close-up aerial images in this shot segment is counted and recorded as the second number. The ratio of the second number to the first number is calculated, i.e., the first proportion of close-up aerial images. This ratio represents the proportion of close-up aerial images in this shot segment. The calculated first proportion of close-up aerial images is compared with a pre-set first target threshold. If the proportion is greater than or equal to the first target threshold, the first recognition result of this shot segment is determined to be a close-up aerial shot. If the proportion is less than the first target threshold, the first recognition result of this shot segment is determined to be a non-close-up aerial shot. The first recognition results for all shot segments are obtained through the above method.

[0118] Optionally, in this embodiment, predicting close-up aerial images frame by frame and determining the category of the entire shot segment by statistical proportion can improve the accuracy and reliability of recognition. By calculating the proportion instead of judging a single frame, noise and random errors can be effectively resisted, improving the robustness of the model in practical applications.

[0119] As an optional example, the device also includes:

[0120] The second processing module is used to determine each segment of the first shot as the current shot before inputting the target shot images of all the first shot segments into the trained close-up spatial model to obtain the second recognition result of each segment of the first shot. The module then performs the following operations on the current shot segment:

[0121] Divide the current shot segment into multiple equal parts;

[0122] Extract one frame from each segment of the shot to obtain the target shot image of the current shot segment.

[0123] Optionally, in this embodiment, multiple target shot images are extracted from each of the first shot segments as recognition data input to the trained close-up spatial model. Taking a certain shot segment as an example, the entire shot segment is divided into multiple partial shot segments. Assuming the shot segment contains N frames, it is divided into K segments on average, and the frame count of each segment is calculated as A = N / K (rounded down). For the last partial shot segment, if the number of frames is not divisible by K, it may contain slightly more or slightly fewer frames to ensure that all frames are included. In each partial shot segment, one shot image is selected as a representative. This can be done using various strategies, such as: extracting the middle frame of the partial shot segment; extracting the first or last frame of the partial shot segment; or randomly selecting any frame from the partial shot segment. The set of frame images extracted from each partial shot segment constitutes the target shot image for this shot segment. Assuming this shot segment is divided into K segments, the set of target shot images will contain K images.

[0124] Optionally, in this embodiment, by dividing the entire shot segment into multiple partial shot segments and extracting frame images from them, the diversity and variation within the shot segment can be better captured, ensuring that the selected images are more representative. Extracting only a small number of frame images for subsequent processing significantly reduces the computational burden and improves processing efficiency compared to processing all frames of the entire shot segment.

[0125] As an optional example, the second recognition module includes:

[0126] The second processing unit is used to determine each segment of the first shot as the current shot segment, and to perform the following operations on the current shot segment:

[0127] For each target shot image in the current shot segment, perform close-up aerial image prediction to determine whether each target shot image is a close-up aerial image;

[0128] Count the third number of all target shot images in the current shot segment, and the fourth number of close-up empty shots among all target shot images;

[0129] Divide the fourth quantity by the third quantity to obtain the percentage of the second close-up empty image in the current shot segment;

[0130] If the proportion of the second close-up aerial image is greater than or equal to the second target threshold, the second recognition result of the current shot segment is determined to be a close-up aerial shot;

[0131] If the proportion of the second close-up aerial image is less than the second target threshold, the second recognition result of the current shot segment is determined to be a non-close-up aerial shot.

[0132] Optionally, in this embodiment, all target shot images of the first shot segment are input into a trained close-up aerial scene model to obtain a second recognition result for each first shot segment. Taking a certain first shot segment as an example, the specific process is as follows: each target shot image in this shot segment is input into the trained close-up aerial scene model. The model classifies each target shot image and determines whether it is a close-up aerial scene image. For each target shot image, the model's prediction result (whether it is a close-up aerial scene image or not) is recorded. The total number of all target shot images in this first shot segment is counted and recorded as the third number, which is the total number of target images extracted in this first shot segment. The number of target shot images predicted as close-up aerial scenes in this first shot segment is counted and recorded as the fourth number. The ratio of the fourth number to the third number is calculated, which is the second proportion of close-up aerial scenes. This ratio represents the proportion of close-up aerial scenes in this first shot segment. The calculated proportion of the second close-up aerial image is compared with a pre-set second target threshold. If the proportion is greater than or equal to the second target threshold, the second recognition result of this first shot segment is determined to be a close-up aerial shot. If the proportion is less than the second target threshold, the second recognition result of this first shot segment is determined to be a non-close-up aerial shot.

[0133] For other examples of this embodiment, please refer to the examples above, which will not be repeated here.

[0134] Figure 6 This is a schematic diagram of an optional electronic device according to an embodiment of this application, such as... Figure 6 As shown, it includes a processor 602, a communication interface 604, a memory 606, and a communication bus 608. The processor 602, communication interface 604, and memory 606 communicate with each other via the communication bus 608.

[0135] Memory 606 is used to store computer programs;

[0136] When processor 602 executes a computer program stored in memory 606, it performs the following steps:

[0137] Acquire all shot segments of the target video and extract the shot image of each frame of each shot segment;

[0138] All shot images are input into the pre-trained pre-model to obtain the first recognition result for each shot segment, where the first recognition result includes close-up shots of the empty space and non-close-up shots of the empty space.

[0139] Input the target shot images of all first shot segments into the trained close-up air scene model to obtain the second recognition result of each first shot segment. The first shot segment is the shot segment whose first recognition result is a close-up air scene shot. The target shot image is multiple shot images extracted from all shot images of the corresponding shot segment. The second recognition result includes close-up air scene shots and non-close-up air scene shots.

[0140] If the second identification result of the second shot segment is a close-up empty shot, then the second shot segment is determined to be a close-up empty shot, wherein the second shot segment is any shot segment among all the first shot segments.

[0141] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0142] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0143] As an example, the memory 606 described above may include, but is not limited to, the first acquisition module 502, the first identification module 504, the second identification module 506, and the determination module 508 of the aforementioned close-up aerial shot recognition device. Furthermore, it may include, but is not limited to, other module units of the aforementioned close-up aerial shot recognition device, which will not be elaborated upon in this example.

[0144] The aforementioned processor can be a general-purpose processor, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0145] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0146] Those skilled in the art will understand that Figure 6 The structure shown is for illustrative purposes only. The device that implements the above-described method for recognizing close-up aerial shots can be a terminal device, such as a smartphone (e.g., an Android phone, an iOS phone), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 6 The different configurations shown.

[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0148] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which, when executed by a processor, performs the steps in the above-described method for recognizing close-up aerial shots.

[0149] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0150] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0151] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0152] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for identifying close-up aerial shots, characterized in that, include: Acquire all shot segments of the target video and extract the shot image of each frame of each shot segment; All shot images are input into a pre-trained model to obtain a first recognition result for each shot segment, wherein the first recognition result includes close-up shots of the empty space and non-close-up shots of the empty space. Input all target shot images of the first shot segments into the trained close-up air scene model to obtain the second recognition result of each first shot segment. The first shot segment is the shot segment whose first recognition result is a close-up air scene shot. The target shot image is multiple shot images extracted from all shot images of the corresponding shot segment. The second recognition result includes close-up air scene shots and non-close-up air scene shots. If the second identification result of the second shot segment is a close-up empty shot, then the second shot segment is determined to be a close-up empty shot, wherein the second shot segment is any shot segment among all the first shot segments.

2. The method according to claim 1, characterized in that, The acquisition of all shot segments of the target video includes: Acquire the target video; The target video is read frame by frame, and the camera positions in the target video are detected and marked. Based on the camera positions, the target video is divided into multiple camera segments, where each camera segment consists of video frames between two consecutive camera positions.

3. The method according to claim 1, characterized in that, Before inputting all the shot images into the trained pre-model to obtain the first recognition result for each of the shot segments, the method further includes: Obtain the trained close-up spatial model and the first training dataset; The first training dataset is input into the trained close-up aerial scene model so that the trained close-up aerial scene model can perform data cleaning and classification on the first training dataset to obtain a close-up aerial scene training dataset and a non-close-up aerial scene training dataset. The pre-trained model is obtained by training the pre-trained model using the close-up aerial shot training dataset and the non-close-up aerial shot training dataset.

4. The method according to claim 3, characterized in that, Before acquiring the trained close-up spatial model, the method further includes: Add a low-rank matrix factorization layer to the close-up spatial large model; Obtain a second training dataset and input prompts, and input the second training dataset and the input prompts into the close-up spatial large model to obtain the prediction result dataset; Calculate the loss value based on the second training dataset and the prediction result dataset; The parameters of the low-rank matrix factorization layer are updated based on the loss value to obtain the trained close-up spatial large model.

5. The method according to claim 1, characterized in that, The step of inputting all the shot images into the trained pre-model to obtain the first recognition result for each shot segment includes: Each segment of all shot clips is designated as the current shot clip, and the following operations are performed on the current shot clip: For each shot image of the current shot segment, perform close-up empty scene image prediction to determine whether each shot image is a close-up empty scene image; Count the first number of all shot images in the current shot segment, and the second number of close-up empty shots in the current shot segment; Divide the second quantity by the first quantity to obtain the first close-up empty image ratio of the current shot segment; If the proportion of the first close-up aerial image is greater than or equal to the first target threshold, the first recognition result of the current shot segment is determined to be a close-up aerial shot; If the proportion of the first close-up aerial image is less than the first target threshold, the first identification result of the current shot segment is determined to be a non-close-up aerial shot.

6. The method according to claim 1, characterized in that, Before inputting the target shot images of all first shot segments into the trained close-up spatial model to obtain the second recognition result for each first shot segment, the method further includes: Each segment of the first-shot footage is designated as the current shot segment, and the following operations are performed on the current shot segment: Divide the current shot segment into multiple partial shot segments on an equal basis; Extract one frame from each segment of the shot to obtain the target shot image of the current shot segment.

7. The method according to claim 1, characterized in that, The step of inputting the target shot images of all first shot segments into the trained close-up spatial model to obtain the second recognition result for each first shot segment includes: Each segment of the first-shot footage is designated as the current shot segment, and the following operations are performed on the current shot segment: For each target shot image in the current shot segment, perform close-up aerial image prediction to determine whether each target shot image is a close-up aerial image; The third count is the total number of all target shot images in the current shot segment, and the fourth count is the total number of close-up empty shots among all target shot images; Divide the fourth quantity by the third quantity to obtain the proportion of the second close-up empty image in the current shot segment; If the proportion of the second close-up aerial image is greater than or equal to the second target threshold, the second recognition result of the current shot segment is determined to be a close-up aerial shot; If the proportion of the second close-up aerial image is less than the second target threshold, the second identification result of the current shot segment is determined to be a non-close-up aerial shot.

8. A device for recognizing close-up aerial shots, characterized in that, include: The first acquisition module is used to acquire all shot segments of the target video and extract the shot image of each frame of each shot segment; The first recognition module is used to input all the shot images into the trained pre-model to obtain the first recognition result for each of the shot segments, wherein the first recognition result includes close-up shots of the empty space and non-close-up shots of the empty space. The second recognition module is used to input the target shot images of all first shot segments into the trained close-up air scene model to obtain the second recognition result of each first shot segment. The first shot segment is the shot segment whose first recognition result is a close-up air scene shot. The target shot image is multiple shot images extracted from all shot images of the corresponding shot segment. The second recognition result includes close-up air scene shots and non-close-up air scene shots. The determining module is used to determine that the second shot segment is a close-up empty shot when the second identification result of the second shot segment is a close-up empty shot, wherein the second shot segment is any shot segment among all the first shot segments.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to perform the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.

Images