A method and system for intelligently converting a horizontal screen to a vertical screen

Abstract

The application provides a kind of intelligent horizontal screen to vertical screen method and system, belong to image processing technical field;The application is by lens boundary detection and is divided into independent lens segment for horizontal screen video, target detection and scene semantic analysis are carried out to each lens, the weight of saliency factor is dynamically adjusted based on semantic label, and the optimal cutting and following main body are comprehensively screened;Face priority protection strategy is used to determine initial cutting center point, and exponential moving average hybrid smoothing processing based on lens boundary is applied, including lens boundary forced reset, motion speed adaptive attenuation factor adjustment and dynamic threshold abnormal frame detection and rejection, to obtain high stability cutting center point sequence;According to this sequence, vertical screen picture is cut and encoded and packaged output;The application realizes the full process automation and scene adaptation of horizontal and vertical conversion, effectively eliminates picture jump and jitter while ensuring the sensitivity of main body following, significantly improves the main body retention rate and picture stability in multiple scenes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for intelligently converting landscape to portrait mode. Background Technology

[0002] With the rise of mobile internet and short video platforms, the demand for converting traditional 16:9 landscape videos to 9:16 portrait videos has increased dramatically. However, simple centering or fixed-area cropping methods result in the loss of the main subject and compositional imbalance, failing to meet the requirements of high-quality content production. Therefore, video retargeting technology based on intelligent analysis has become a research hotspot in the fields of computer vision and video processing.

[0003] Currently, the automated screen-to-portrait conversion solutions proposed in the industry typically involve steps such as video content analysis, saliency detection, cropping window generation, and smoothing. However, existing solutions generally have the following shortcomings: 1. High subject loss rate and lack of intelligent content perception. Traditional fixed-region cropping algorithms lack semantic-level understanding of image content and cannot autonomously perceive and locate the core salient subject of the image. In video scenes where people are shifting, targets are moving rapidly, or there are multiple subjects in the frame, problems such as subjects being truncated, core objects being lost, or key information (such as text or logos) being cropped out of the frame are very likely to occur, seriously damaging the integrity of the video content and the effectiveness of information delivery.

[0004] 2. Significant technical shortcomings exist in image smoothing processing. To ensure a good viewing experience for vertical videos, the motion trajectory of the cropping window needs to be smoothed. Existing solutions mostly use simple moving averages or global Kalman filters, which are single smoothing algorithms. These algorithms fail to differentiate the processing based on the characteristics of the video's shot boundaries, resulting in lag and a strong "dragging" effect when following the subject's movement within the shot. Furthermore, at shot transition boundaries, the residual state of the smoothing algorithm produces an unnatural smooth transition, causing jumps or jitter in the starting frame of the new shot, severely affecting the stability of the visual experience.

[0005] 3. Saliency identification lacks scene adaptability. Mainstream saliency target selection schemes generally use a fixed-weight linear fusion of multi-dimensional features such as target category, location, and size to calculate saliency scores. This "one-size-fits-all" weighting strategy fails to differentiate between video shot types and scene semantics. For example, using the same weighting calculation rules in scenarios with vastly different compositional logics—such as close-ups of dialogue, intense sports events, expansive landscapes, or detailed product demonstrations—leads to frequent problems like misjudgment of saliency subjects, focus shifts during cropping, and unbalanced image composition, making it unsuitable for the intelligent conversion needs of diverse video content.

[0006] 4. Fragmented technical solutions and low degree of automation. Most current technologies only optimize a single step in the horizontal-to-vertical conversion process, such as focusing solely on object detection or image smoothing. They lack an end-to-end automated closed-loop solution that integrates video structured analysis, semantic understanding, dynamic saliency calculation, adaptive smoothing, intelligent cropping, and encoding encapsulation. This results in significant manual intervention in actual production applications for cropping area correction and image jitter repair, leading to high labor costs and difficulty in guaranteeing the stability and consistency of output quality during batch processing.

[0007] In summary, there is an urgent need for a method and system for intelligent landscape-to-portrait screen conversion to address the technical problems of existing technologies, such as high subject loss rate, lack of intelligent content perception, significant technical shortcomings in smooth image processing, lack of scene adaptability in salience recognition, fragmented technical solutions, and low degree of automation. Summary of the Invention

[0008] To address the aforementioned technical issues, this invention provides an intelligent landscape-to-portrait method and system based on scene-adaptive saliency tracking and boundary blending smoothing. It aims to eliminate screen jumps at the lens switching boundary while taking into account the lens's internal tracking sensitivity through a scene semantic-driven dynamic multi-weight saliency fusion and a lens boundary-aware blending smoothing architecture, and to achieve fully automated, high-quality processing from input to output.

[0009] To achieve the above objectives, this application proposes a method for intelligent landscape-to-portrait screen conversion, comprising: Perform shot boundary detection on the input landscape video, divide the landscape video into multiple shot segments, and record the shot transition boundary markers; Each shot segment is frame-by-frame extracted, and target detection and scene semantic parsing are performed on the extracted images to obtain the attribute information of each target, as well as the shooting type and scene semantic type of the shot segment. Based on the shooting type and scene semantic type, the weights of multiple preset saliency factors are dynamically adjusted, the comprehensive saliency score of each target is calculated, and the target with the highest score is selected as the cropped follow subject. Determine the initial cropping center point based on the vertical screen frame and the position of the subject being cropped. The initial crop center point sequence within the shot segment is subjected to a hybrid smoothing process based on the shot boundary to obtain a smoothed crop center point sequence. The hybrid smoothing process resets the historical state of the smoothing algorithm when the shot boundary is detected, adaptively adjusts the smoothing coefficient according to the movement speed of the crop center point within the shot, and detects and removes abnormal frames. Based on the smoothed cropping center point sequence, a vertical screen image is cropped from the video frames of the horizontal screen video. The cropped vertical screen image is video encoded and encapsulated with the original audio extracted from the horizontal screen video to output a vertical screen video.

[0010] As a further solution, the multiple saliency factors include target type score, location score, size score, and frequency of occurrence score; the dynamic adjustment of the weights of the preset multiple saliency factors includes: Based on a preset scene rule base, the dynamic weight vector of each saliency factor is determined using the shooting type and scene semantic type; The comprehensive significance score of each objective is obtained by weighting and summing the significance factors using the dynamic weight vector.

[0011] As a further solution, the shooting types include close-ups, medium shots, wide shots, and dynamic tracking shots; the scene semantic types include dialogue, sports events, landscape displays, product demonstrations, and video montages; and the determination of the dynamic weight vector for each saliency factor specifically includes at least one of the following scene adaptive adjustment rules: When the shooting type is a close-up and the scene semantic type is a dialogue, the target type weight is increased, and a facial feature-specific bonus independent of the face position is added, while the position weight is reduced. When the shooting type is panoramic and the scene semantic type is sports event, the target type weight is reduced and the size weight and occurrence frequency weight are increased. When the scene semantic type is landscape display, the position weight is increased to guide the cropping center to fit the composition area of ​​the image; When the semantic type of the scenario is product demonstration, the weight of the target type and the weight of the size are balanced. When the semantic type of the scene is a video montage, the weights of each saliency factor are balanced.

[0012] As a further solution, determining the initial cropping center point based on the position of the vertical screen frame and the cropping-following-the-subject body specifically includes: Determine whether the vertical screen frame can completely encompass the cropped and following subject; If it can completely wrap around the subject, then the x-coordinate of the center point of the cut that follows the subject is used as the initial x-coordinate of the cut center point; If the cut cannot completely cover the subject, further determine whether the subject being cut is a person. If it is a person, then the face detection model will be forcibly invoked, and the x-coordinate of the center of the detected face region will be used as the x-coordinate of the initial cropping center point. If the target is a non-human object such as a vehicle, product, or landscape, the x-coordinate of the center point of the cropping subject itself is used as the initial x-coordinate of the cropping center point.

[0013] As a further solution, the lens boundary-based blending smoothing process employs an exponential moving average algorithm, specifically including: Forced reset of lens boundary: When the lens switching boundary marker is detected, the historical cumulative state of the exponential moving average algorithm is cleared, and the initial cropping center point of the first frame of the current lens segment is directly used as the smoothing reference value. Motion speed adaptive attenuation factor adjustment steps: Within a single shot segment, the moving speed of the cropping center point of adjacent frames is calculated in real time, and the smoothing coefficient is dynamically adjusted based on a preset speed threshold; Dynamic threshold abnormal frame detection and removal steps: Calculate the standard deviation of the center point displacement of multiple recent consecutive frames in real time, and dynamically generate a judgment threshold based on the standard deviation; when the deviation between the original cropped center point and the algorithm-predicted center point of the current frame exceeds the dynamic judgment threshold, the frame is judged as an abnormal frame caused by target recognition noise or instantaneous interference, and the original cropped center point of the frame is replaced with the value of the algorithm-predicted center point to complete the frame.

[0014] As a further solution, in the motion speed adaptive attenuation factor adjustment step, the dynamic adjustment of the smoothing coefficient satisfies the following relationship: ; Where α is the dynamically adjusted smoothing coefficient. The base smoothness is defined as a value ranging from 0.7 to 0.9, where v is the moving speed. The preset speed threshold is defined as follows.

[0015] As a further solution, the dynamic determination threshold is the product of the standard deviation of the center point displacement in the most recent consecutive frames and a preset coefficient.

[0016] As a further solution, the attribute information of each target includes target type, bounding box pixel coordinates, center coordinates, occupied area size, and frequency of appearance in the shot segment.

[0017] As a further solution, the height of the vertical screen frame is consistent with the original screen height of the horizontal screen video, and the aspect ratio is 9:16; the step of cropping the vertical screen image from the video frames of the horizontal screen video includes: keeping the original screen height unchanged, using the horizontal coordinate of the center point in the smoothed cropping center point sequence as the cropping center, and cropping out a 9:16 vertical screen image.

[0018] On the other hand, the present invention also provides an intelligent landscape-to-portrait screen conversion system based on scene adaptive saliency tracking and boundary blending smoothing, comprising: The video segmentation preprocessing module is used to perform shot boundary detection on the input landscape video, divide the landscape video into multiple shot segments, and record shot transition boundary markers. The scene and target parsing module is used to extract frames from each shot segment, perform target detection and scene semantic parsing on the extracted frames, and obtain the attribute information of each target as well as the shooting type and scene semantic type of the shot segment. The scene adaptive saliency calculation module is used to dynamically adjust the weights of multiple preset saliency factors based on the shooting type and scene semantic type, calculate the comprehensive saliency score of each target, and select the target with the highest score as the cropping follow subject; The cropping center point determination module is used to determine the initial cropping center point based on the position of the vertical screen frame and the cropping follow-the-subject; wherein, when the vertical screen frame cannot completely wrap the cropping follow-the-subject and the subject is a person, the face detection model is forcibly invoked with the center of the face region as the reference. The boundary blending smoothing processing module is used to apply a blending smoothing process based on the lens boundary to the initial cropping center point sequence within the shot segment to obtain a smoothed cropping center point sequence. The blending smoothing process resets the historical state of the smoothing algorithm when the lens boundary is detected, adaptively adjusts the smoothing coefficient according to the moving speed of the cropping center point inside the shot, and detects and removes abnormal frames caused by target recognition noise. The intelligent cropping and encoding module is used to crop vertical screen images from the video frames of the horizontal video based on the smoothed cropping center point sequence, and then encode the video. The audio and video encapsulation module is used to encapsulate the encoded video stream with the original audio extracted from the landscape video and output a portrait video.

[0019] Compared with related technologies, the intelligent landscape-to-portrait screen conversion method and system provided by this invention have the following advantages: 1. This invention calculates visual impact and persuasiveness scores separately, and then weights and integrates visual indicators such as contrast, color harmony, symmetry, and alignment with content persuasiveness indicators such as visual hierarchy depth, information density, visual element proportion, and animation density to obtain a comprehensive excellence score. This dual evaluation mechanism ensures that the selected pages are not only visually appealing but also possess good information hierarchy and logical persuasiveness, better meeting the quality requirements of PPT pages in practical application scenarios.

[0020] 2. This invention automatically clusters the feature vectors of compelling pages using density-based clustering algorithms such as DBSCAN. Within each cluster, representative pages are selected based on their scores, and deduplication is performed using a cosine similarity threshold. This mechanism ensures that the final generated PPT template file covers a variety of distinct layout styles, while retaining only the highest-quality, non-redundant pages within each style. This effectively avoids the problem of template homogenization and enhances the user's choice and applicability when using templates.

[0021] 3. This invention constructs a DPO model, which is trained using paired page data with user preference labels. It learns user scenario preferences by comparing the differences between the policy network output and the reference policy score. An uncertainty head is introduced during training to evaluate the reliability of the predictions, and a weighted loss function is used to fuse user confidence levels, enabling the model to efficiently capture subjective aesthetic preferences in specific scenarios from a limited amount of human feedback. The trained model is then used to re-evaluate and filter candidate pages, resulting in template files that better suit the actual needs of users in specific scenarios (such as academic reports, business presentations, etc.).

[0022] 4. When generating template pages, this invention does not simply copy the original page elements. Instead, it converts the extracted shape object information into placeholders of corresponding types, while retaining the original text content, images, and animations as design references. The resulting template files have a standardized, fillable structure, allowing users to directly replace content, while also preserving the original, visually appealing design style for reference. This significantly reduces the time cost of layout design for users and improves template reusability and creation efficiency. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This invention provides a schematic diagram of the steps involved in a method for intelligently converting a landscape screen to a portrait screen. Figure 2 A schematic diagram of a smart landscape-to-portrait screen system structure provided by the present invention; The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0027] Example 1 Please see Figure 1 This embodiment provides a method for intelligently switching between landscape and portrait modes, including: Perform shot boundary detection on the input landscape video, divide the landscape video into multiple shot segments, and record the shot transition boundary markers; Each shot segment is frame-by-frame extracted, and target detection and scene semantic parsing are performed on the extracted images to obtain the attribute information of each target, as well as the shooting type and scene semantic type of the shot segment. Based on the shooting type and scene semantic type, the weights of multiple preset saliency factors are dynamically adjusted, the comprehensive saliency score of each target is calculated, and the target with the highest score is selected as the cropped follow subject. Determine the initial cropping center point based on the vertical screen frame and the position of the subject being cropped. The initial crop center point sequence within the shot segment is subjected to a hybrid smoothing process based on the shot boundary to obtain a smoothed crop center point sequence. The hybrid smoothing process resets the historical state of the smoothing algorithm when the shot boundary is detected, adaptively adjusts the smoothing coefficient according to the movement speed of the crop center point within the shot, and detects and removes abnormal frames. Based on the smoothed cropping center point sequence, a vertical screen image is cropped from the video frames of the horizontal screen video. The cropped vertical screen image is video encoded and encapsulated with the original audio extracted from the horizontal screen video to output a vertical screen video.

[0028] It should be noted that the intelligent landscape-to-portrait conversion method provided in this embodiment, based on scene adaptive saliency tracking and boundary blending smoothing, adopts a closed-loop processing logic of "scene decomposition - semantic parsing - dynamic subject selection - fixed-point cropping - smoothing optimization - encoding encapsulation". This method first performs fully automatic shot boundary detection and decomposition on the input landscape video, simultaneously recording shot time parameters and boundary markers; Subsequently, each independent shot segment is uniformly framed, and a lightweight deep learning model is used to complete multi-target detection and scene semantic parsing. Based on scene semantic tags, a scene adaptive dynamic weighting mechanism is used to perform non-linear weighted fusion of significant factors such as type, position, size, and frequency of each target to accurately select the core subject following the image. When determining the cropping center point, a face-first protection strategy is implemented for adaptation verification. Next, the cropping center point trajectory is optimized using an EMA hybrid smoothing architecture based on the lens boundary. Finally, vertical screen frame sequences are generated by batch cropping based on the optimized center point coordinates, and standardized video encoding and audio encapsulation are completed to output the final vertical screen video file.

[0029] Furthermore, the plurality of saliency factors include target type score, location score, size score, and frequency of occurrence score; the dynamic adjustment of the weights of the preset plurality of saliency factors includes: Based on a preset scene rule base, the dynamic weight vector of each saliency factor is determined using the shooting type and scene semantic type; The comprehensive significance score of each objective is obtained by weighting and summing the significance factors using the dynamic weight vector.

[0030] Specifically, this embodiment introduces a dynamic weighting mechanism. Instead of applying a fixed weight ratio to all video scenes, it dynamically generates a corresponding weight coefficient for each of the four saliency factors based on a pre-defined scene rule base and the previously parsed shooting type and scene semantic type, forming a dynamic weight vector. Subsequently, the system weights and sums this dynamic weight vector with the scores of each factor to calculate the comprehensive saliency score for each target. This technical solution ensures, at the algorithmic level, that the saliency calculation logic can adaptively reconstruct as scene semantics change, enabling the final selected cropping to accurately adapt to the core composition requirements of different scenes.

[0031] Furthermore, the shooting types include close-ups, medium shots, wide shots, and dynamic tracking shots; the scene semantic types include dialogue, sports events, landscape displays, product demonstrations, and video montages; and the determination of the dynamic weight vector for each saliency factor specifically includes at least one of the following scene adaptive adjustment rules: When the shooting type is a close-up and the scene semantic type is a dialogue, the target type weight is increased, and a facial feature-specific bonus independent of the face position is added, while the position weight is reduced. When the shooting type is panoramic and the scene semantic type is sports event, the target type weight is reduced and the size weight and occurrence frequency weight are increased. When the scene semantic type is landscape display, the position weight is increased to guide the cropping center to fit the composition area of ​​the image; When the semantic type of the scenario is product demonstration, the weight of the target type and the weight of the size are balanced. When the semantic type of the scene is a video montage, the weights of each saliency factor are balanced.

[0032] Specifically, after the system analyzes the specific scene such as "close-up of a dialogue", "panoramic view of a sports event", "scenery display", "product demonstration" or "film and television montage", it does not use a uniform weight allocation formula, but triggers a differentiated, preset non-linear weight mapping strategy.

[0033] Specifically, this embodiment is based on adaptive configuration logic for various typical scenarios, and "expert rules" are customized for the composition requirements of different content.

[0034] For example, in close-up scenes of character dialogue, the rules force a significant increase in the weight of the "target type" factor to a dominant position, and creatively introduce a unique bonus item independent of the face position, while weakening the impact of positional deviation, ensuring that the face is always the absolute focus of the picture.

[0035] In the panoramic scene of sports events, the rules are completely switched, reducing the decisive role of type and instead greatly increasing the weight of "size" and "frequency of occurrence". This allows the algorithm to lock onto the core moving subject that appears continuously and occupies a large part of the screen from among many moving targets, thereby effectively blocking out distracting targets such as spectators and scoreboards.

[0036] By solidifying these prior rules based on photographic composition principles and semantic understanding into algorithmic logic, this embodiment achieves a leap from "general computing" to "scene-specific adaptation," eliminating the problems of subject misjudgment and compositional imbalance in complex scenes caused by fixed-weight algorithms.

[0037] Furthermore, determining the initial cropping center point based on the vertical screen frame and the position of the cropping subject specifically includes: Determine whether the vertical screen frame can completely encompass the cropped and following subject; If it can completely wrap around the subject, then the x-coordinate of the center point of the cut that follows the subject is used as the initial x-coordinate of the cut center point; If the cut cannot completely cover the subject, further determine whether the subject being cut is a person. If it is a person, then the face detection model will be forcibly invoked, and the x-coordinate of the center of the detected face region will be used as the x-coordinate of the initial cropping center point. If the target is a non-human object such as a vehicle, product, or landscape, the x-coordinate of the center point of the cropping subject itself is used as the initial x-coordinate of the cropping center point.

[0038] Specifically, its core lies in introducing a hierarchical and clearly prioritized judgment logic to solve the problem of cropping reference selection when the vertical screen frame and the target subject size are not compatible. This method does not simply apply the target geometric center directly, but first introduces a key "fit verification" step, that is, to determine whether the preset 9:16 vertical screen frame can completely cover the selected cropping subject.

[0039] Based on the verification result, this embodiment sets up decision branches with three gradients.

[0040] First, in the most ideal situation, if the vertical frame can completely accommodate the entire subject, then the center point of the subject should be used as the cropping reference. This is the most efficient and complete composition strategy.

[0041] Secondly, in complex situations where the frame cannot completely encompass the target, the solution demonstrates its core intelligent protection feature: it further determines the type of the target. If the target that cannot be completely encompassed is a person, the system will forcibly switch to a highest priority protection mode, that is, call the face detection model and force the center of the detected face region as the cropping reference. Thus, under the extreme conditions of limited space, it strategically abandons the requirement for the integrity of the person's entire body, fundamentally ensuring that the face, the area with the highest information density, is not cropped.

[0042] Finally, for non-human subjects that cannot be fully covered, such as vehicles and products, the solution adopts a pragmatic fallback strategy, directly using the center point coordinates of the target itself as the cropping benchmark to ensure that the core visual center of gravity is centered.

[0043] Therefore, this embodiment dynamically selects the cropping benchmark by following the sequence of "integrity judgment, character attribute judgment, and face protection triggering," and accurately applies limited computing resources (face detection) to the character truncation scene where serious visual defects are most likely to occur, thus achieving an autonomous balance between compositional integrity and key information protection.

[0044] Furthermore, the hybrid smoothing process based on lens boundaries employs an exponential moving average algorithm, specifically including: Forced reset of lens boundary: When the lens switching boundary marker is detected, the historical cumulative state of the exponential moving average algorithm is cleared, and the initial cropping center point of the first frame of the current lens segment is directly used as the smoothing reference value. Motion speed adaptive attenuation factor adjustment steps: Within a single shot segment, the moving speed of the cropping center point of adjacent frames is calculated in real time, and the smoothing coefficient is dynamically adjusted based on a preset speed threshold; Dynamic threshold abnormal frame detection and removal steps: Calculate the standard deviation of the center point displacement of multiple recent consecutive frames in real time, and dynamically generate a judgment threshold based on the standard deviation; when the deviation between the original cropped center point and the algorithm-predicted center point of the current frame exceeds the dynamic judgment threshold, the frame is judged as an abnormal frame caused by target recognition noise or instantaneous interference, and the original cropped center point of the frame is replaced with the value of the algorithm-predicted center point to complete the frame.

[0045] Specifically, this embodiment uses the exponential moving average algorithm as the core smoothing tool, and innovatively designs a triple-coordinated sub-mechanism to address the inherent defects of traditional single smoothing algorithms in video cropping scenarios.

[0046] The overall technical concept of this embodiment is: instead of treating the video as a continuous whole for uniform smoothing, it dynamically switches the smoothing strategy by sensing the switching boundaries of the shots and the motion state inside the frame in real time, thereby perfectly resolving the contradiction between jumps between shots and shake within the shot.

[0047] First, to address the issues of historical trajectory contamination and abrupt image transitions caused by cross-shot smoothing, this embodiment introduces a "forced shot boundary reset" mechanism. When the system detects a shot transition boundary marker, it immediately and forcibly clears all historical state data accumulated by the exponential moving average algorithm, and directly uses the original crop center point of the first frame of the new shot as the starting point for smoothing. This hard reset operation completely cuts off the influence of the previous shot's motion trajectory on the current shot at the algorithm level, ensuring that the smoothing trajectory of each shot starts from a clean, drag-free initial reference.

[0048] Secondly, regarding the internal workings of the lens, this embodiment achieves a dynamic balance between image stability and tracking sensitivity through the remaining two sub-mechanisms. In the "Motion Speed ​​Adaptive Attenuation Factor Adjustment" step, the system quantifies the subject's motion speed in real time and dynamically adjusts the smoothing coefficient accordingly. When the subject moves quickly, the smoothing intensity is automatically reduced to ensure close tracking, while when the subject moves slowly or is stationary, the smoothing intensity is increased to filter out minor shakes.

[0049] Meanwhile, the "dynamic threshold abnormal frame detection and rejection" step analyzes the standard deviation of the center point displacement in recent multi-frame images to generate a judgment threshold that adapts to the stability of the image. This threshold accurately identifies and discards abnormal center point data caused by instantaneous noise in target detection, and then completes the data with predicted values. The coordinated operation of these three mechanisms ultimately generates a smooth trajectory that is both sensitive to subject movement and effectively suppresses various types of image jitter.

[0050] Furthermore, in the motion speed adaptive attenuation factor adjustment step, the dynamic adjustment of the smoothing coefficient satisfies the following relationship: ; Where α is the dynamically adjusted smoothing coefficient. The base smoothness is defined as a value ranging from 0.7 to 0.9, where v is the moving speed. The preset speed threshold is defined as follows.

[0051] Specifically, this embodiment introduces an upper limit truncation function. This ensures that even when the subject moves at extremely high speeds, the attenuation of the smoothing coefficient is controlled between 20% and 100% of the original base value, thus preserving the necessary smoothing ability under extreme conditions and preventing the image from shaking violently without restraint.

[0052] Specifically, the base smoothness in the formula represents the system's strongest smoothing ability when the subject is stationary or in an ideal state. Its value is limited to an empirically optimal range verified by numerous experiments, namely between 0.7 and 0.9. The parameter v is the real-time calculated movement speed of the cropping center point between adjacent frames, which reflects the intensity of the current subject's movement. The preset speed threshold is a sensitivity benchmark that can be configured according to the video type.

[0053] When the subject is stationary or moving slowly, v approaches 0, and α is approximately equal to... The system provides maximum smoothness to filter out minor jitters; when the subject moves quickly, the v value increases while α adaptively decreases, weakening the historical inertia of the smoothing algorithm and ensuring that the cropping window can quickly keep up with the target displacement. This technical solution achieves a dynamic optimal balance between image stability and subject-following sensitivity autonomously and smoothly through a simple computational paradigm.

[0054] Furthermore, the dynamic determination threshold is the product of the standard deviation of the displacement of the center point in the most recent consecutive frames and a preset coefficient.

[0055] Specifically, the dynamic judgment threshold is not a fixed constant, but is dynamically determined by the product of the "standard deviation of the displacement of the center point in recent consecutive frames" and a "preset coefficient".

[0056] Standard deviation is a statistic that measures the degree of dispersion of data. Here, we calculate the standard deviation of the displacement of the center point in recent multiple frames, which is essentially a real-time quantification of the smoothness of the motion in the current video segment.

[0057] When the image is static or moving at a slow, uniform speed, the displacement values ​​between frames are highly consistent and the standard deviation is very small. The dynamic judgment threshold generated based on this is very strict (i.e., the value is very small). At this time, even slight shaking of the recognition box can easily cause the deviation to exceed the threshold, thus being effectively captured and eliminated by the system.

[0058] Conversely, when the subject is undergoing violent or large-scale irregular movement, the inter-frame displacement itself changes drastically, the calculated standard deviation will increase significantly, and the dynamic threshold will also be relaxed (i.e., the value will increase), so as to avoid wrongly judging the normal, high-speed positional changes of the subject as an abnormality and wrongly "correcting" them.

[0059] By using this mechanism of dynamically setting the threshold for anomaly detection based on the recent motion state of the image itself, this embodiment solves the technical contradiction that a fixed threshold cannot simultaneously achieve both "high sensitivity in capturing micro-shakes" and "high tolerance in adapting to large motions".

[0060] Therefore, this embodiment enables the abnormal frame detection system to have scene-adaptive judgment capabilities, which can accurately filter out minor noise in smooth scenes and tolerate large displacements in violent motion scenes, ensuring the robustness and accuracy of smooth processing in all scenes.

[0061] Furthermore, the attribute information of each target includes target type, bounding box pixel coordinates, center coordinates, occupied area size, and frequency of appearance within the shot segment.

[0062] Specifically, in this embodiment, the five types of attribute information together constitute the data foundation for all subsequent saliency calculations, each undertaking a specific technical function. Among them, "target type" is used to distinguish different semantic categories such as people and vehicles, and is a prerequisite for performing type score calculation and scene rule matching; "bounding box pixel coordinates" and "occupied area size" together define the spatial range and size of the target in the picture, and are the geometric basis for calculating the size score and judging whether the vertical screen frame can completely enclose the target; "center coordinates" directly serve the calculation of position score, and together with "occurrence frequency", provide quantitative support for evaluating the compositional value and temporal stability of the target.

[0063] By standardizing and regulating target attributes into the aforementioned standardized and structured information set, this embodiment ensures a clear, complete, and consistent data interface from the target detection module to the saliency calculation module and the cropping center point determination module. This explicit data structure definition enables subsequent complex logic such as dynamic weight calculation and face priority protection determination to operate based on stable and predictable input data. It is a crucial foundation for ensuring the reliable operation of the entire automated process, from underlying data perception to high-level semantic decision-making.

[0064] Furthermore, the height of the vertical screen frame is consistent with the original screen height of the horizontal screen video, and the aspect ratio is 9:16; the step of cropping the vertical screen image from the video frame of the horizontal screen video includes: keeping the original screen height unchanged, using the horizontal coordinate of the center point in the smoothed cropping center point sequence as the cropping center, and cropping out a 9:16 vertical screen image.

[0065] Specifically, this embodiment first clarifies the baseline setting rules for the vertical screen cropping frame from a spatial perspective. That is, the height of the vertical screen frame is completely consistent with the height of the original horizontal video frame, and its aspect ratio is strictly locked at 9:16. The technical significance of this size definition is that it ensures that the cropped vertical screen image can completely retain all the vertical information of the original video, while only cropping proportionally from both sides in the horizontal direction, thereby minimizing content loss caused by redirection.

[0066] Based on this, during frame-by-frame cropping, the system maintains the original video frame height unchanged, using only the horizontal coordinate of the center point in the cropping center point sequence after blending and smoothing optimization as the sole positioning reference in the horizontal direction, to precisely crop a 9:16 vertical screen area from the original horizontal frame. Essentially, this operation accurately applies the stable trajectory obtained through all the aforementioned intelligent analysis and smoothing processes to the actual output of each frame.

[0067] By spatially binding the smoothed center point trajectory to a fixed-size vertical screen frame, this embodiment ensures the stability and consistency of the final video output at the execution level. The complex semantic understanding and trajectory optimization work at the front end is transformed and converged into a clear, repeatable geometric cropping action, achieving a closed-loop connection between intelligent decision-making and precise execution. Based on a strict cropping method using optimized coordinates, the final vertical screen video can stably and smoothly lock onto the core subject, delivering a high-quality visual viewing experience.

[0068] Example 2 Please see Figure 2Corresponding to the method in Embodiment 1 above, this embodiment also provides an intelligent landscape-to-portrait screen conversion system based on scene adaptive saliency tracking and boundary blending smoothing. This system comprises seven functional modules, each operating independently and interacting with each other through data flow to form a fully automated processing closed loop: The video segmentation preprocessing module is used to perform shot boundary detection on the input landscape video, divide the landscape video into multiple shot segments, and record shot transition boundary markers. The scene and target parsing module is used to extract frames from each shot segment, perform target detection and scene semantic parsing on the extracted frames, and obtain the attribute information of each target as well as the shooting type and scene semantic type of the shot segment. The scene adaptive saliency calculation module is used to dynamically adjust the weights of multiple preset saliency factors based on the shooting type and scene semantic type, calculate the comprehensive saliency score of each target, and select the target with the highest score as the cropping follow subject; The cropping center point determination module is used to determine the initial cropping center point based on the position of the vertical screen frame and the cropping follow-the-subject; wherein, when the vertical screen frame cannot completely wrap the cropping follow-the-subject and the subject is a person, the face detection model is forcibly invoked with the center of the face region as the reference. The boundary blending smoothing processing module is used to apply a blending smoothing process based on the lens boundary to the initial cropping center point sequence within the shot segment to obtain a smoothed cropping center point sequence. The blending smoothing process resets the historical state of the smoothing algorithm when the lens boundary is detected, adaptively adjusts the smoothing coefficient according to the moving speed of the cropping center point inside the shot, and detects and removes abnormal frames caused by target recognition noise. The intelligent cropping and encoding module is used to crop vertical screen images from the video frames of the horizontal video based on the smoothed cropping center point sequence, and then encode the video. The audio and video encapsulation module is used to encapsulate the encoded video stream with the original audio extracted from the landscape video and output a portrait video.

[0069] Hardware Implementation Environment: This embodiment has no special dependency on dedicated hardware and has good versatility. Its methods and system modules can be deployed on mainstream computing devices such as general-purpose servers, personal computer terminals, and cloud GPU computing instances. It runs as a software program and can flexibly adapt to various application scenarios such as offline local deployment and cloud batch processing.

[0070] Step 1: Video Segmentation Processing (The video segmentation preprocessing module is responsible for performing Step 1, completing the shot splitting and boundary marking) First, input a raw 16:9 landscape video. The system quantifies the degree of visual change in the video content using inter-frame pixel difference comparison and intra-frame texture change detection algorithms. Combined with preset shot transition threshold judgment logic, it completes fully automatic video segmentation, cutting the complete video stream into several temporally continuous but content-independent shot segments. For each segment, the system simultaneously records its start and end timestamps, as well as the continuous frame sequence information it contains, thus accurately marking all shot transition boundaries in the video. These boundary markers provide basic data support for subsequent fine-tuning of shot segmentation and smooth resetting of cross-shot strategies.

[0071] Step 2: Shot-by-shot frame extraction, target recognition, and scene semantic parsing (the scene and target parsing module is responsible for executing Step 2, completing shot-by-shot frame extraction, target detection, and the generation of two-dimensional scene semantic labels). For each independent shot segment separated in step 1, the system performs uniform frame extraction at a pre-configurable time interval. This mechanism avoids wasting computing power by processing a large number of redundant frames while ensuring the integrity of the features required for analysis.

[0072] For each extracted frame, a pre-trained, lightweight, high-precision object detection model is used for inference to identify and output the attribute information of all salient objects within the frame. This attribute information specifically includes: object type (such as people, vehicles, scenery, text, products, etc.), pixel coordinates of the bounding box, pixel coordinates of the object's center point, area occupied by the bounding box, and core parameters such as the frequency of continuous occurrence of the object type within the shot segment.

[0073] Simultaneously, a separate lightweight scene classification network is employed to perform semantic parsing on representative keyframes of each shot, outputting core labels in two dimensions. The first dimension is the shot type, used to describe the framing or motion state of the scene, specifically including close-ups, medium shots, wide shots, and dynamic tracking shots; the second dimension is the scene semantic type, used to describe the content attributes of the scene, specifically including dialogue, sports events, landscape displays, product demonstrations, film montages, and documentary shooting. Through these two-dimensional labels, the system defines clear semantic attributes for each shot, providing accurate semantic priors for subsequent dynamic weight adaptive adjustments.

[0074] Step 3: Scene Adaptive Dynamic Saliency Calculation and Target Selection (The scene adaptive saliency calculation module is responsible for executing Step 3. It has a built-in scene weight rule library and dynamic interpolation algorithm to complete the weighted calculation of saliency factors and the selection of the optimal following subject.) This step is one of the core inventions, abandoning the traditional fixed-weight fusion mode and innovatively introducing a dynamic weight nonlinear fusion mechanism based on shot type and scene semantics.

[0075] First, four basic and standardized significance factors are constructed. The values ​​of all factors are normalized to eliminate the impact of differences in the magnitude of values ​​across different dimensions on the calculation. (1) Target type score Ti, whose priority is preset as people > vehicles > text > general scenery / products; (2) Position score Pi is calculated using Gaussian weighting based on the distance between the target center point and the intersection of the geometric center of the image and the golden ratio third line. The closer to the core composition area, the higher the score. (3) Size score Si is the normalized percentage of the area of ​​the outer frame of a target relative to the total area of ​​the entire frame. (4) Frequency score Fi is the normalized value of the frequency of a target in all frames of the shot segment.

[0076] Next, define a dynamic weight vector. These correspond to the weights of the type, location, size, and frequency factors mentioned above, respectively. The specific values ​​of these weights are not fixed but are dynamically generated by interpolating from a pre-defined scene rule base lookup table and the scene confidence score output by the scene classification network, achieving differentiated adaptation for different scenes. An example of the core mapping rules is as follows: Close-up scenes of character dialogue: Weighting the target type Elevate it to a dominant position (e.g., 0.5), and add an additional bonus based on facial features, while appropriately reducing the weight of position. To reduce sensitivity to positional deviations, priority is given to ensuring the complete and centered display of the main face. Panoramic scenes of sports events: Reduce type weights and significantly increase size weight. Frequency weight This allows the algorithm to focus on the core moving subjects that appear continuously in the picture and have a wide range of movement (such as a group of athletes on the track or a single athlete in the lead position), and avoids interfering targets such as scoreboards and scattered spectators. Scenery display scenarios: Prioritize increasing position weight This guides the cropping center point to align with the golden ratio composition area of ​​the image, which conforms to photographic aesthetics, in order to preserve the best aesthetic features of the image. Product demonstration scenario: A balanced strategy is adopted to balance the weight of target type and size in order to prioritize and fully display the core product in the picture, ensuring that the product appearance is not truncated in any way; Film and television montage scenes: An adaptive equalization strategy is adopted to balance the four types of weights to take into account the main characters in the rapidly changing shots, the stability of the picture composition, and the continuity of the target movement, so as to adapt to complex shots with fast-paced switching of multiple elements.

[0077] Finally, the comprehensive saliency score of each detected target in the current frame is calculated using this dynamic weight vector. The calculation formula is as follows: ;in, The system selects the target with the highest overall score among all targets within a single shot as the sole subject to be cropped and followed in that shot. This mechanism enables intelligent and adaptive subject selection for different video scenarios.

[0078] Step 4: Determine the vertical screen frame and cropping center point (the cropping center point determination module is responsible for executing Step 4, which incorporates a face priority protection strategy and combines the 9:16 vertical screen frame rule to complete the calculation of the initial cropping center point coordinates) This step introduces a face-priority protection strategy to determine the initial cropping center point for each frame. Specifically, a standard portrait frame with the same height as the original landscape video and an aspect ratio of 9:16 is used to adapt and validate the salience-optimal target selected in the previous step. The specific judgment logic is as follows: Case 1: If the 9:16 vertical screen frame can completely wrap around the entire target subject, then the x-coordinate of the center point of the target subject is directly used as the x-coordinate of the initial cropping center point of this frame. Scenario 2: If the vertical frame cannot completely encompass the target due to its large size or off-center position, the target type will be further determined. If the target is identified as a person, the system will forcibly invoke a built-in face detection model, using the center horizontal coordinate of the detected face region as the cropping reference, prioritizing the complete display of the face, and fundamentally avoiding the problem of the face being truncated by the frame edge; Scenario 3: If the vertical frame cannot completely encompass the target, and the target is determined to be a vehicle, product, large landscape, or other non-human subject, then the horizontal coordinate of the center point of the target's own bounding box is directly used as the initial cropping center point to ensure that the core visual center is centered.

[0079] Step 5: EMA blending and smoothing strategy based on lens boundary (the boundary blending and smoothing processing module is responsible for executing step 5, integrating the triple sub-algorithm of lens boundary reset, adaptive attenuation, and outlier removal to optimize the trajectory of the crop center point) This step is another core invention point. In response to the triple defects of "camera switching jumps", "in-camera tracking lag" and "target recognition jitter" that occur when the traditional exponential moving average algorithm is directly applied to video cropping, a hybrid smoothing architecture with three sub-mechanisms is designed.

[0080] Sub-step 5-1: Force reset of lens boundaries The system monitors the shot transition boundary marker signal generated in step 1 in real time. Once a new shot segment is detected, the algorithm immediately forces the clearing of all historical accumulated states of the EMA algorithm and directly uses the initial crop center point calculated from the first frame of the new shot segment as the initial reference value for the current smoothing process. This sub-step completely eliminates any "dragging" or residual effect of the motion trajectory of the previous shot on the smoothing calculation of the current shot, fundamentally solving the problem of image jumps during cross-shot transitions.

[0081] Sub-step 5-2: Adjustment of motion speed adaptive decay factor Within the same shot segment, the system calculates in real-time the movement speed v (in pixels) of the crop center point between adjacent video frames. Combined with a pre-configurable speed threshold v_th, the smoothing coefficient α of the EMA algorithm is dynamically adjusted. The specific formula for adjustment is: Among them, basic smoothness This is an empirical preset value, ranging from 0.7 to 0.9, which can be adapted to the vast majority of video scenarios; α is the sensitivity threshold for pixel displacement. The logic of this adjustment mechanism is as follows: when the subject moves quickly, resulting in a large v value, α will adaptively decrease, weakening the constraint of the smoothing algorithm and ensuring that the cropping window can sensitively keep up with the subject's displacement; when the subject is stationary or moves slowly, resulting in a small v value, α will increase accordingly, strengthening the smoothing effect, thereby effectively suppressing screen jitter caused by slight shaking of the detection box.

[0082] Sub-step 5-3: Dynamic threshold abnormal frame detection and removal Based on the standard deviation of the cropping center point displacement over recent consecutive frames (e.g., 5 to 10 frames), the system dynamically generates a deviation judgment threshold. This threshold is the product of the aforementioned standard deviation and a preset coefficient, thus automatically adjusting the sensitivity of anomaly detection according to the current stability of the image. The system compares the deviation between the original calculated center point of the current frame and the center point predicted by the EMA algorithm frame by frame. Once this deviation exceeds the dynamically generated judgment threshold, the current frame is judged as an abnormal frame caused by target recognition noise or transient interference. For such abnormal frames, the system directly discards their original, noisy center point data and replaces it with the algorithm's prediction value at the current moment. This mechanism completely eliminates instantaneous image jumps caused by single-point recognition errors.

[0083] Through the synergistic effect of the above three mechanisms, this step ultimately outputs a smooth cropping center point trajectory sequence that combines high stability (no image jitter) and high sensitivity (no lag in following).

[0084] Step 6: Crop the vertical screen image according to smooth coordinates (The intelligent cropping and encoding module is responsible for executing steps 6 to 8, completing image cropping, frame sequence integration, and H.264 standardized encoding according to smooth coordinates) Using the horizontal coordinate of the center point of each frame after blending and smoothing optimization in step 5 as the cropping reference, and maintaining the same height as the original landscape video frame, a standard 9:16 aspect ratio portrait-sized image area is precisely cropped from each original landscape frame. This area retains the core subject and good compositional information of the image to the maximum extent, thereby generating a continuous single-frame portrait image.

[0085] Step 7: Batch processing of all lenses The system iterates through all the shot segments according to the original video time sequence recorded in step 1, and sequentially executes steps 2 to 6 (i.e., semantic parsing, subject selection, center point calculation, smoothing optimization, and image cropping). After all shot segments have been processed, the generated vertical screen frame image sequence is integrated according to the original timeline to finally form a vertical screen video frame sequence that is continuous in time and complete in content.

[0086] Step 8: H.264 video encoding For the complete vertical screen frame sequence integrated in step 7, a standardized H.264 encoding is performed using a video encoder. During the encoding process, parameters such as bitrate, resolution, and frame rate of the target video are configured uniformly to ensure that the final generated video stream has clear image quality and broad terminal playback compatibility.

[0087] Step 9: Audio Extraction and MP4 Packaging (The audio and video packaging module is responsible for executing Step 9, completing the extraction of the original audio stream and the multiplexing and packaging of the audio and video streams, and outputting the final MP4 format vertical video file) From the original input landscape video file, the original audio stream data is extracted losslessly, preserving all information such as the original sound, background music, and vocals, eliminating any compression distortion. Finally, the H.264 standard video stream generated in step 8 is multiplexed and packaged with the extracted original audio stream to generate a standard MP4 format file, which serves as the final output 9:16 portrait video.

[0088] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for intelligently converting a landscape screen to a portrait screen, characterized in that, include: Perform shot boundary detection on the input landscape video, divide the landscape video into multiple shot segments, and record the shot transition boundary markers; Each shot segment is frame-by-frame extracted, and target detection and scene semantic parsing are performed on the extracted images to obtain the attribute information of each target, as well as the shooting type and scene semantic type of the shot segment. Based on the shooting type and scene semantic type, the weights of multiple preset saliency factors are dynamically adjusted, the comprehensive saliency score of each target is calculated, and the target with the highest score is selected as the cropped follow subject. Determine the initial cropping center point based on the vertical screen frame and the position of the subject being cropped. The initial crop center point sequence within the shot segment is subjected to a hybrid smoothing process based on the shot boundary to obtain a smoothed crop center point sequence. The hybrid smoothing process resets the historical state of the smoothing algorithm when the shot boundary is detected, adaptively adjusts the smoothing coefficient according to the movement speed of the crop center point within the shot, and detects and removes abnormal frames. Based on the smoothed cropping center point sequence, a vertical screen image is cropped from the video frames of the horizontal screen video. The cropped vertical screen image is video encoded and encapsulated with the original audio extracted from the horizontal screen video to output a vertical screen video.

2. The method for intelligent landscape-to-portrait screen conversion according to claim 1, characterized in that, The multiple saliency factors include target type score, location score, size score, and frequency of occurrence score; The dynamic adjustment of the weights of multiple preset saliency factors includes: Based on a preset scene rule base, the dynamic weight vector of each saliency factor is determined using the shooting type and scene semantic type; The comprehensive significance score of each objective is obtained by weighting and summing the significance factors using the dynamic weight vector.

3. The method for intelligent landscape-to-portrait screen conversion according to claim 2, characterized in that, The shooting types include close-ups, medium shots, wide shots, and dynamic tracking shots; the scene semantic types include dialogue, sports events, landscape displays, product demonstrations, and video montages; the determination of the dynamic weight vector for each saliency factor specifically includes at least one of the following scene adaptive adjustment rules: When the shooting type is a close-up and the scene semantic type is a dialogue, the target type weight is increased, and a facial feature-specific bonus independent of the face position is added, while the position weight is reduced. When the shooting type is panoramic and the scene semantic type is sports event, the target type weight is reduced and the size weight and occurrence frequency weight are increased. When the scene semantic type is landscape display, the position weight is increased to guide the cropping center to fit the composition area of ​​the image; When the semantic type of the scenario is product demonstration, the weight of the target type and the weight of the size are balanced. When the semantic type of the scene is a video montage, the weights of each saliency factor are balanced.

4. The method for intelligent landscape-to-portrait screen conversion according to claim 1, characterized in that, The step of determining the initial cropping center point based on the position of the vertical screen frame and the cropping-following-the-subject body specifically includes: Determine whether the vertical screen frame can completely encompass the cropped and following subject; If it can completely wrap around the subject, then the x-coordinate of the center point of the cut that follows the subject is used as the initial x-coordinate of the cut center point; If the cut cannot completely cover the subject, further determine whether the subject being cut is a person. If it is a person, then the face detection model will be forcibly invoked, and the x-coordinate of the center of the detected face region will be used as the x-coordinate of the initial cropping center point. If the target is a non-human object such as a vehicle, product, or landscape, the x-coordinate of the center point of the cropping subject itself is used as the initial x-coordinate of the cropping center point.

5. The method for intelligent landscape-to-portrait screen conversion according to claim 1, characterized in that, The lens boundary-based blending smoothing process employs an exponential moving average algorithm, specifically including: Forced reset of lens boundary: When the lens switching boundary marker is detected, the historical cumulative state of the exponential moving average algorithm is cleared, and the initial cropping center point of the first frame of the current lens segment is directly used as the smoothing reference value. Motion speed adaptive attenuation factor adjustment steps: Within a single shot segment, the moving speed of the cropping center point of adjacent frames is calculated in real time, and the smoothing coefficient is dynamically adjusted based on a preset speed threshold; Dynamic threshold abnormal frame detection and removal steps: Calculate the standard deviation of the center point displacement of multiple recent consecutive frames in real time, and dynamically generate a judgment threshold based on the standard deviation; when the deviation between the original cropped center point and the algorithm-predicted center point of the current frame exceeds the dynamic judgment threshold, the frame is judged as an abnormal frame caused by target recognition noise or instantaneous interference, and the original cropped center point of the frame is replaced with the value of the algorithm-predicted center point to complete the frame.

6. The method for intelligent landscape-to-portrait screen conversion according to claim 5, characterized in that, In the motion speed adaptive attenuation factor adjustment step, the dynamic adjustment of the smoothing coefficient satisfies the following relationship: ； Where α is the dynamically adjusted smoothing coefficient. The base smoothness is defined as a value ranging from 0.7 to 0.9, where v is the moving speed. The preset speed threshold is defined as follows.

7. A method for intelligent landscape-to-portrait screen conversion according to claim 5, characterized in that, The dynamic determination threshold is the product of the standard deviation of the center point displacement in the most recent consecutive frames and a preset coefficient.

8. The method for intelligent landscape-to-portrait screen conversion according to claim 1, characterized in that, The attribute information of each target includes target type, bounding box pixel coordinates, center coordinates, occupied area size, and frequency of appearance in the shot segment.

9. A method for intelligent landscape-to-portrait screen conversion according to claim 1, characterized in that, The height of the vertical screen frame is consistent with the original screen height of the horizontal screen video, and the aspect ratio is 9:16; the step of cropping the vertical screen image from the video frame of the horizontal screen video includes: keeping the original screen height unchanged, using the horizontal coordinate of the center point in the smoothed cropping center point sequence as the cropping center, and cropping the 9:16 vertical screen image.

10. A system for intelligent landscape-to-portrait screen conversion, used to implement the method for intelligent landscape-to-portrait screen conversion as described in any one of claims 1 to 9, characterized in that, include: The video segmentation preprocessing module is used to perform shot boundary detection on the input landscape video, divide the landscape video into multiple shot segments, and record shot transition boundary markers. The scene and target parsing module is used to extract frames from each shot segment, perform target detection and scene semantic parsing on the extracted frames, and obtain the attribute information of each target as well as the shooting type and scene semantic type of the shot segment. The scene adaptive saliency calculation module is used to dynamically adjust the weights of multiple preset saliency factors based on the shooting type and scene semantic type, calculate the comprehensive saliency score of each target, and select the target with the highest score as the cropping follow subject; The cropping center point determination module is used to determine the initial cropping center point based on the position of the vertical screen frame and the cropping follow-the-subject; wherein, when the vertical screen frame cannot completely wrap the cropping follow-the-subject and the subject is a person, the face detection model is forcibly invoked with the center of the face region as the reference. The boundary blending smoothing processing module is used to apply a blending smoothing process based on the lens boundary to the initial cropping center point sequence within the shot segment to obtain a smoothed cropping center point sequence. The blending smoothing process resets the historical state of the smoothing algorithm when the lens boundary is detected, adaptively adjusts the smoothing coefficient according to the moving speed of the cropping center point inside the shot, and detects and removes abnormal frames caused by target recognition noise. The intelligent cropping and encoding module is used to crop vertical screen images from the video frames of the horizontal video based on the smoothed cropping center point sequence, and then encode the video. The audio and video encapsulation module is used to encapsulate the encoded video stream with the original audio extracted from the landscape video and output a portrait video.

Images