Adaptive picture cutting method and system based on target position
Through object detection algorithm and adaptive tracking processing, intelligent cutting and dynamic adjustment of video images are realized, solving the problems of target deviation and inaccurate cutting in traditional methods, and improving the user experience and viewing of video content.
Patent Information
- Application Number
- CN202411051872.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional video processing methods cannot be dynamically adjusted according to the screen direction or target position, resulting in the important targets that may deviate from the center of the picture when watching videos on different devices, affecting the user experience, especially in real-time video transmission and monitoring systems, which can handle multiple targets or target positions quickly. The effect is not good when changing, and it is prone to problems such as missing targets or inaccurate screen cutting.
Adaptive picture cutting method based on target position is adopted, target recognition is performed on the first frame of the video stream through the object detection algorithm, adaptive tracking is performed, and the initial tracking frame is obtained according to the specified screen direction, and in the subsequent frame screen Update the target position, make sure the target is located in the center of the crop window, update the tracking box in real time, and get landscape and portrait images that are suitable for landscape and vertical screens.
It realizes intelligent cutting and dynamic adjustment of video images, improves the viewing and practicality of video content, enhances user experience and satisfaction, and ensures that the target is always stable in different screen directions.
Smart Images

Figure CN119090898A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual processing technology, and in particular to an adaptive image cutting method and a cutting system based on a target position. Background Art
[0002] With the rapid development of multimedia technology, the display methods of video content are becoming increasingly diverse, from traditional TV screens to portable devices such as smartphones and tablets, the screen orientation and size vary significantly. Traditional video processing methods often use a fixed screen cutting method, which cannot be dynamically adjusted according to the screen orientation or target position. As a result, when watching videos on different devices, important targets may deviate from the center of the screen, affecting the user experience. Especially in real-time video transmission and monitoring systems, it is particularly important to accurately capture and display the target object.
[0003] Traditional video cutting methods mostly rely on preset fixed sizes and positions, and cannot be adaptively adjusted according to the target position in the video content. In addition, these methods are particularly ineffective when dealing with multiple targets or when the target position changes rapidly, and are prone to target loss or inaccurate cutting.
[0004] Therefore, it is necessary to provide an adaptive screen cutting method and cutting system based on target position to solve the above technical problems. Summary of the invention
[0005] To solve the above technical problems, the present invention provides an adaptive screen cutting method and cutting system based on target position, which realizes intelligent cutting and dynamic adjustment of video screen. The method not only improves the viewing and practicality of video content, but also enhances user experience and satisfaction.
[0006] The present invention provides an adaptive image cutting method based on target position, and the cutting method comprises the following steps:
[0007] S1: Use the target detection algorithm to identify the first frame in the video stream and obtain the initial position of the target;
[0008] S2: performing adaptive tracking processing on the initial position of the target, and obtaining an initial tracking frame according to a specified screen direction, wherein the screen direction includes a horizontal screen and a vertical screen;
[0009] S3: in a subsequent frame, updating the position of the target and cutting the frame based on the updated position of the target to obtain a cropping window with the target located at the center;
[0010] S4: Under the constraint of the cropping window, the initial tracking frame is updated according to the position of the target in the subsequent frame to obtain an updated tracking frame;
[0011] S5: Using the updated tracking frame, the image containing the target is processed in real time to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen, respectively.
[0012] Preferably, step S1 specifically includes the following steps:
[0013] S101: Preprocessing the first frame in the video stream using the YOLO target detection algorithm to obtain preprocessed image data;
[0014] S102: Detecting the target based on the preprocessed image data to obtain a bounding box containing the target;
[0015] S103: extracting regional features from the bounding box containing the target to obtain target features;
[0016] S104: Positioning the target based on the extracted target features to obtain an initial position of the target in the first frame, wherein the initial position is represented by coordinates.
[0017] Preferably, step S2 specifically includes the following steps:
[0018] S201: Apply Fourier transform to convert the target feature into the frequency domain to obtain feature representation in the frequency domain;
[0019] S202: In the frequency domain, the filter is trained using the obtained feature representation to obtain a filter for target positioning in subsequent frames;
[0020] S203: Calculate a response map of the target area of the first frame using the trained filter, and determine an initial tracking frame adapted to the screen direction based on the peak position of the response map.
[0021] Preferably, step S3 specifically includes the following steps:
[0022] S301: Use the KCF tracking algorithm to match the target features in the subsequent frame to obtain the position of the target in the current frame;
[0023] S302: Based on the updated position of the target, the current frame is centrally processed using a screen cutting algorithm to ensure that the target is located at the center of the cropping window, thereby obtaining a cropping window adapted to the target position;
[0024] S303: Using a cropping window to crop the current frame to obtain a cropping window, wherein the target is located in a central area of the cropping window.
[0025] Preferably, step S4 specifically includes the following steps:
[0026] S401: using the KCF tracking algorithm to update the position of the target in the subsequent frame to obtain the updated position of the target;
[0027] S402: dynamically adjusting the initial tracking frame based on the updated position of the target to obtain a tracking frame adapted to the updated position of the target;
[0028] S403: Optimizing the updated tracking frame within the constraint range of the cropping window to obtain an updated tracking frame that fits the target.
[0029] Preferably, step S5 specifically includes the following steps:
[0030] S501: using the updated tracking frame to locate the image in the cropping window containing the target, so that the target is at the center of the image;
[0031] S502: scaling and format conversion processing are performed on the located image to obtain images suitable for horizontal and vertical screen displays respectively;
[0032] S503: Output the processed image in real time to obtain horizontal and vertical images adapted to different screen directions, so as to achieve stable presentation of the target in the video stream.
[0033] The present invention also provides an adaptive screen cutting system based on target position, which is used to execute the adaptive screen cutting method based on target position, and the cutting system comprises:
[0034] An initialization module is used to identify the target in the first frame of the video stream through a target detection algorithm to obtain the initial position of the target;
[0035] A tracking configuration module, used to perform adaptive tracking processing on the initial position of the target and obtain an initial tracking frame according to a specified screen orientation, wherein the screen orientation includes a horizontal screen and a vertical screen;
[0036] A target centering module, used for updating the position of the target in subsequent frames and cutting the frame based on the updated position of the target to obtain a cropping window with the target at the center;
[0037] A tracking frame adjustment module, used for updating the initial tracking frame according to the position of the target in the subsequent frame under the constraint of the cropping window to obtain an updated tracking frame;
[0038] The output adaptation module is used to perform real-time output processing on the image containing the target using the updated tracking frame to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen respectively.
[0039] Compared with the related art, the adaptive screen cutting method and cutting system based on target position provided by the present invention have the following beneficial effects:
[0040] The present invention uses a target detection algorithm to accurately identify the first frame of a video stream, ensuring that the initial position of the target can be accurately obtained; then, adaptive tracking processing is performed according to the initial position of the target, and an initial tracking frame is obtained according to a specified screen direction (horizontal or vertical); then, in subsequent frames, the target position is updated in real time, and the screen is cut based on the updated position to ensure that the target is always located at the center of the cropping window; under the constraint of the cropping window, the tracking frame is updated in real time according to the position of the target in subsequent frames; finally, the image containing the target is output in real time using the updated tracking frame to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen, respectively, thereby realizing intelligent cutting and dynamic adjustment of the video screen. The method not only improves the viewing and practicality of the video content, but also enhances the user experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 The process intention of the adaptive screen cutting method based on target position provided by the present invention;
[0042] Figure 2 This is a module structure diagram of the target position-based adaptive screen cutting system provided by the present invention. DETAILED DESCRIPTION
[0043] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.
[0044] It should also be noted that, for ease of description, only the part relevant to the present invention but not all the content is shown in the accompanying drawings. It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as the processing or method described as a flow chart. Although the flow chart describes each operation (or step) as being processed sequentially, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. When its operation is completed, the processing can be terminated, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.
[0045] Embodiment 1
[0046] The adaptive image cutting method based on target position provided by the present invention refers to Figure 1 As shown, the cutting method comprises the following steps:
[0047] S1: Use the target detection algorithm to identify the target in the first frame of the video stream and obtain the initial position of the target.
[0048] In this embodiment, the first frame in the video stream is processed using a deep learning YOLO target detection model. These models are trained with a large amount of data and can accurately identify multiple target categories. The model outputs the target's bounding box coordinates and category label as initialization data. In order to improve detection speed and accuracy, the model can be pre-trained to focus on specific types of targets, such as face recognition or vehicle detection.
[0049] Ultimately, it ensures that the target can be quickly locked from the beginning of the video stream, providing a precise starting position for subsequent tracking, reducing false detections and missed detections, and improving the robustness of the entire system.
[0050] S2: performing adaptive tracking processing on the initial position of the target, and obtaining an initial tracking frame according to a specified screen direction, wherein the screen direction includes a horizontal screen and a vertical screen.
[0051] In this embodiment, the tracker is initialized according to the target position in the first frame, and the target position is continuously estimated in subsequent frames by the tracker according to the appearance features and motion prediction of the target.
[0052] At the same time, the screen orientation information is obtained through device sensors (such as the accelerometer and gyroscope) to adjust the aspect ratio of the tracking frame to match the current screen ratio.
[0053] S3: In a subsequent frame, the position of the target is updated and the frame is cut based on the updated position of the target to obtain a cropping window with the target located at the center.
[0054] In this embodiment, in each frame, after the target position is updated, a cropping window centered on the target is calculated. The size of the cropping window can be set to a fixed ratio, or dynamically adjusted according to the target size to ensure that the target occupies a certain proportion of the screen, and the video stream processing library is used to perform image cropping to obtain a sub-image containing the target.
[0055] Dynamic cropping ensures that the target always remains in visual focus in the picture. Even if the target moves quickly, the user can still see the target clearly, enhancing the viewing experience.
[0056] S4: Under the constraint of the cropping window, the initial tracking frame is updated according to the position of the target in the subsequent frame to obtain an updated tracking frame.
[0057] In this embodiment, after the target position is updated, it is checked whether the tracking frame needs to be resized or reshaped to adapt to the size change of the target.
[0058] If the target is close to the edge of the picture, the boundaries of the cropping window may be restricted. In this case, you need to adjust the relative position of the tracking box to prevent part of the target from being cropped off.
[0059] By calculating the relative position between the tracking frame and the cropping window, a smooth transition can be made to avoid sudden jumps and improve the smoothness of viewing experience.
[0060] Update the tracking frame in real time to reduce the target loss rate, especially when the target size changes greatly, and maintain good tracking performance.
[0061] S5: Using the updated tracking frame, the image containing the target is processed in real time to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen, respectively.
[0062] In this embodiment, according to the screen direction, a suitable output resolution and aspect ratio are selected, and the cropped image is scaled and filled to adapt to different display devices.
[0063] At the same time, image quality optimization, such as sharpening and color correction, can be performed before output to improve image quality. The output image takes into account both the center positioning of the target and the screen direction, providing users with a consistent and high-quality visual experience.
[0064] Specifically, step S1 specifically includes the following steps:
[0065] S101: Preprocess the first frame in the video stream using the YOLO target detection algorithm to obtain preprocessed image data.
[0066] In this embodiment, the first frame of the video stream is captured, which is usually image data in RGB format.
[0067] The image is preprocessed, including resizing, normalization, and color space conversion. For example, the image size is adjusted to the input size required by the YOLO model, such as 416x416 pixels, to fit the convolution layer of the model.
[0068] Normalization usually scales pixel values from the range of 0-255 to between 0-1 to optimize the learning process of the neural network.
[0069] Preprocessing also includes noise removal and contrast enhancement operations to improve the accuracy of detection.
[0070] Preprocessing ensures that the image data meets the input specifications of the YOLO model, thereby improving detection speed and accuracy; resizing and normalization enable the model to run stably under different size and brightness conditions, enhancing the generalization ability of the algorithm.
[0071] S102: Detecting the target based on the preprocessed image data to obtain a bounding box containing the target.
[0072] In this embodiment, the preprocessed image is sent to the YOLO model, and the model outputs a grid. Each grid unit predicts multiple bounding boxes and their corresponding category probabilities. The YOLO model directly predicts the coordinates and size of the bounding box by regression, and gives the confidence score of the target category. The non-maximum suppression (NMS) algorithm is used to filter out overlapping bounding boxes and retain the most likely detection results.
[0073] S103: Extract regional features from the bounding box containing the target to obtain target features.
[0074] In this embodiment, the bounding box coordinates and category information of the target are extracted from the output of the YOLO model, and the target area is cropped from the original image using these coordinates, and feature extraction is performed on the cropped target area. This can be done through the output of the last convolutional layer of the YOLO model, or using an additional feature extraction network such as ResNet or VGG.
[0075] The feature vector is used in subsequent tracking algorithms as the basis for the target appearance characteristics.
[0076] Feature extraction provides a visual description of the target, which is crucial for the subsequent tracking algorithm, helping the tracker to recognize and locate the same target in subsequent frames.
[0077] S104: Positioning the target based on the extracted target features to obtain an initial position of the target in the first frame, wherein the initial position is represented by coordinates.
[0078] In this embodiment, after the feature extraction of the target area is completed, these features are used to accurately locate the position of the target in the image.
[0079] The coordinates of the upper left and lower right corners of the bounding box are used to represent the position of the target. These coordinates are usually relative to the origin of the entire image (usually the upper left corner), and the coordinates are expressed in absolute pixel values.
[0080] The initial position coordinates of the target provide the starting point required by the tracking algorithm, ensuring the continuity and accuracy of target tracking in subsequent frames. At the same time, using the coordinate form of the bounding box, the tracking algorithm can know the space occupied by the target more accurately, which is especially important when the target size changes.
[0081] Specifically, step S2 specifically includes the following steps:
[0082] S201: Apply Fourier transform to convert the target feature into the frequency domain to obtain feature representation in the frequency domain.
[0083] In this embodiment, the features extracted from the target area (for example, color histogram, gradient orientation histogram HOG, etc.) are converted into numerical vectors, and these feature vectors are then converted to the frequency domain through discrete Fourier transform (DFT). The Fourier transform converts the feature information in the spatial domain into information in the frequency domain, so that the local and overall patterns of the features can be reflected in the spectrum. The frequency domain representation can usually better handle the translation invariance problem and can simplify the calculation process in some cases.
[0084] S202: In the frequency domain, the obtained feature representation is used to train the filter to obtain a filter for target positioning in subsequent frames.
[0085] In this embodiment, a filter is designed and trained using feature representation in the frequency domain. This filter is capable of identifying characteristic patterns of a target in the frequency domain. The training process involves minimizing a loss function of a least squares error to ensure that the filter can correctly match the frequency domain features of the target.
[0086] S203: Calculate a response map of the target area of the first frame using the trained filter, and determine an initial tracking frame adapted to the screen direction based on the peak position of the response map.
[0087] In this embodiment, the trained filter is applied to the frequency domain representation of the first frame to generate a response map, wherein the response map reflects the degree of match between the filter and each part of the image. By finding the peak in the response map, the area most likely to contain the target can be determined. The position corresponding to the highest peak will be used to define the initial tracking frame, which will be appropriately adjusted according to the direction and scale of the screen to ensure that it matches the display characteristics of the screen, for example, maintaining the aspect ratio or avoiding exceeding the border of the picture. This initial tracking frame will serve as the basis for target tracking in subsequent frames to ensure that the tracking algorithm can accurately follow the movement of the target.
[0088] Specifically, step S3 specifically includes the following steps:
[0089] S301: Use the KCF tracking algorithm to match the target features in the subsequent frame to obtain the position of the target in the current frame.
[0090] In this embodiment, the KCF algorithm is a tracking method based on correlation filters, which uses a kernel function to enhance the performance of the filter and uses a circulant matrix structure and a fast Fourier transform (FFT) to accelerate the calculation process.
[0091] In each frame, the KCF algorithm uses the previously trained filter to perform correlation operations on the candidate region of the current frame to produce a response map, which indicates the strength of the filter output. The maximum position in the response map is determined as the estimated position of the target in the current frame. In order to improve accuracy, a multi-scale search strategy may be used to adapt to changes in target size.
[0092] S302: Based on the updated position of the target, the current frame is centralized using a screen cutting algorithm to ensure that the target is located at the center of the cropping window, thereby obtaining a cropping window adapted to the target position.
[0093] In this embodiment, a cropping window is calculated based on the target position obtained in S301, the center of which is aligned with the center of the target, and the size of the cropping window can be dynamically adjusted according to the size of the target to ensure that the target occupies most of the space in the window while taking into account the possible range of motion of the target.
[0094] If the target is close to the edge of the picture, the cropping window needs to be adjusted accordingly to prevent the cropped picture from being incomplete.
[0095] S303: Using a cropping window to crop the current frame to obtain a cropping window, wherein the target is located in a central area of the cropping window.
[0096] In this embodiment, the cropping window parameters determined in S302 are used to extract a sub-picture containing the target from the current frame. This sub-picture should be centered on the target, so as to ensure that the target is in a prominent position in the entire video sequence, which is convenient for subsequent processing and analysis.
[0097] Specifically, step S4 specifically includes the following steps:
[0098] S401: Using the KCF tracking algorithm to update the position of the target in the subsequent frame, to obtain the updated position of the target.
[0099] In this embodiment, a model is established based on the characteristics of the target (such as color, texture, etc.), and this model is used in the new frame for correlation filtering to find the best matching point. By maximizing the peak in the response map, the new position of the target can be located. In addition, the KCF algorithm usually adopts an online learning mechanism to continuously update the filter to adapt to changes in the target appearance.
[0100] S402: Based on the updated position of the target, dynamically adjust the initial tracking frame to obtain a tracking frame adapted to the updated position of the target.
[0101] In this embodiment, when the target position is updated, the size and position of the tracking frame need to be readjusted to ensure that it still accurately surrounds the target. The dynamic adjustment includes moving the center of the tracking frame to the new position of the target and adjusting the width and height of the tracking frame according to the size change of the target.
[0102] S403: Optimizing the updated tracking frame within the constraint range of the cropping window to obtain an updated tracking frame that fits the target.
[0103] In this embodiment, the cropping window limits the range of movement of the tracking frame to ensure that the target does not exceed the preset field of view. The updated tracking frame needs to be further optimized to avoid instability caused by rapid movement or size change of the target. The optimization involves smoothing the boundary changes of the tracking frame, reducing jitter, and adjusting the size of the tracking frame to keep the target in an appropriate proportion within the cropping window.
[0104] In addition, if the target is close to the edge of the cropping window, the adjustment of the tracking frame also needs to take into account the boundary conditions of the window to prevent part of the target from being cropped.
[0105] Specifically, step S5 specifically includes the following steps:
[0106] S501: Using the updated tracking frame, the image within the cropping window containing the target is positioned so that the target is at the center of the image.
[0107] In this embodiment, in this step, the updated tracking frame obtained in the previous step is used to determine the precise position of the target in the current frame.
[0108] Next, adjust the center point of the cropping window to coincide with the center of the updated tracking box, ensuring that the target is always in the center of the image.
[0109] This localization method helps keep the target visually stable in the video stream, allowing viewers to continue to focus on the target even if the target moves in the scene.
[0110] S502: Scaling and format conversion are performed on the positioned image to obtain images suitable for horizontal and vertical screen displays, respectively.
[0111] In this embodiment, the positioned image needs to be scaled according to different screen orientations to adapt to the display ratio of the horizontal screen or the vertical screen.
[0112] Format conversion involves color space conversion, encoding format changes, etc. to ensure that the output image is compatible with various display devices and platforms.
[0113] S503: Output the processed image in real time to obtain horizontal and vertical images adapted to different screen directions, so as to achieve stable presentation of the target in the video stream.
[0114] In this embodiment, real-time output means that the speed of image processing and transmission must be fast enough to ensure the smoothness of the video stream and avoid delays or freezes. The output image should automatically switch according to the screen direction of the receiving end, that is, display horizontal images on horizontal screen devices and display vertical images on vertical screen devices.
[0115] The working principle of the adaptive screen cutting method based on target position provided by the present invention is as follows: the first frame of the video stream is accurately identified by the target detection algorithm to ensure that the initial position of the target can be accurately obtained; then, adaptive tracking processing is performed according to the initial position of the target, and the initial tracking frame is obtained according to the specified screen direction (horizontal screen or vertical screen); then, in the subsequent frame, the target position is updated in real time, and the screen is cut based on the updated position to ensure that the target is always located at the center of the cropping window; under the constraint of the cropping window, the tracking frame is updated in real time according to the position of the target in the subsequent frame; finally, the image containing the target is output in real time using the updated tracking frame to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen, respectively, so as to realize intelligent cutting and dynamic adjustment of the video screen. The method not only improves the viewing and practicality of the video content, but also enhances the user experience and satisfaction.
[0116] Embodiment 2
[0117] The present invention also provides a target position-based adaptive screen cutting system for executing the target position-based adaptive screen cutting method. Figure 2 As shown, the cutting system comprises:
[0118] The initialization module 100 is used to perform target recognition on the first frame of the video stream through a target detection algorithm to obtain an initial position of the target.
[0119] The tracking configuration module 200 is used to perform adaptive tracking processing on the initial position of the target and obtain an initial tracking frame according to a specified screen direction, wherein the screen direction includes a horizontal screen and a vertical screen.
[0120] The target centering module 300 is used to update the position of the target in subsequent frames and cut the frame based on the updated position of the target to obtain a cropping window with the target located at the center.
[0121] The tracking frame adjustment module 400 is used to update the initial tracking frame according to the position of the target in the subsequent frame under the constraint of the cropping window to obtain an updated tracking frame.
[0122] The output adaptation module 500 is used to perform real-time output processing on the image containing the target using the updated tracking frame to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen, respectively.
[0123] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0124] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0125] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or includes elements that are inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.
Claims
1. Adaptive screen cutting method based on target position, It is characterized in that The cutting method comprises the following steps: S1: Use the target detection algorithm to identify the first frame in the video stream and obtain the initial position of the target; S2: performing adaptive tracking processing on the initial position of the target, and obtaining an initial tracking frame according to a specified screen direction, wherein the screen direction includes a horizontal screen and a vertical screen; S3: in a subsequent frame, updating the position of the target and cutting the frame based on the updated position of the target to obtain a cropping window with the target located at the center; S4: Under the constraint of the cropping window, the initial tracking frame is updated according to the position of the target in the subsequent frame to obtain an updated tracking frame; S5: Using the updated tracking frame, the image containing the target is processed in real time to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen, respectively.
2. The method for adaptively cutting images based on target position according to claim 1, It is characterized in that Step S1 specifically includes the following steps: S101: Preprocessing the first frame in the video stream using the YOLO target detection algorithm to obtain preprocessed image data; S102: Detecting the target based on the preprocessed image data to obtain a bounding box containing the target; S103: extracting regional features from the bounding box containing the target to obtain target features; S104: Positioning the target based on the extracted target features to obtain an initial position of the target in the first frame, wherein the initial position is represented by coordinates.
3. The method for adaptively cutting images based on target position according to claim 2, It is characterized in that Step S2 specifically includes the following steps: S201: Apply Fourier transform to convert the target feature into the frequency domain to obtain feature representation in the frequency domain; S202: In the frequency domain, the filter is trained using the obtained feature representation to obtain a filter for target positioning in subsequent frames; S203: Calculate a response map of the target area of the first frame using the trained filter, and determine an initial tracking frame adapted to the screen direction based on the peak position of the response map.
4. The method for adaptively cutting images based on target position according to claim 3, It is characterized in that Step S3 specifically includes the following steps: S301: Use the KCF tracking algorithm to match the target features in the subsequent frame to obtain the position of the target in the current frame; S302: Based on the updated position of the target, the current frame is centrally processed using a screen cutting algorithm to ensure that the target is located at the center of the cropping window, thereby obtaining a cropping window adapted to the target position; S303: Using a cropping window to crop the current frame to obtain a cropping window, wherein the target is located in a central area of the cropping window.
5. The method for adaptively cutting images based on target position according to claim 4, It is characterized in that Step S4 specifically includes the following steps: S401: using the KCF tracking algorithm to update the position of the target in the subsequent frame to obtain the updated position of the target; S402: dynamically adjusting the initial tracking frame based on the updated position of the target to obtain a tracking frame adapted to the updated position of the target; S403: Optimizing the updated tracking frame within the constraint range of the cropping window to obtain an updated tracking frame that fits the target.
6. The method for adaptively cutting images based on target position according to claim 5, It is characterized in that Step S5 specifically includes the following steps: S501: using the updated tracking frame to locate the image in the cropping window containing the target, so that the target is at the center of the image; S502: scaling and format conversion processing are performed on the located image to obtain images suitable for horizontal and vertical screen displays respectively; S503: Output the processed image in real time to obtain horizontal and vertical images adapted to different screen directions, so as to achieve stable presentation of the target in the video stream.
7. A target position-based adaptive screen cutting system, configured to execute a target position-based adaptive screen cutting method according to any one of claims 1 to 6, It is characterized in that The cutting system comprises: An initialization module is used to identify the target in the first frame of the video stream through a target detection algorithm to obtain the initial position of the target; A tracking configuration module, used to perform adaptive tracking processing on the initial position of the target and obtain an initial tracking frame according to a specified screen orientation, wherein the screen orientation includes a horizontal screen and a vertical screen; A target centering module, used for updating the position of the target in subsequent frames and cutting the frame based on the updated position of the target to obtain a cropping window with the target at the center; A tracking frame adjustment module, used for updating the initial tracking frame according to the position of the target in the subsequent frame under the constraint of the cropping window to obtain an updated tracking frame; The output adaptation module is used to perform real-time output processing on the image containing the target using the updated tracking frame to obtain a horizontal image and a vertical image adapted to the horizontal screen and the vertical screen respectively.
