Shooting adjustment method based on AI intelligent optimization and shooting robot

Abstract

The invention discloses an AI-based intelligent optimized shooting adjustment method and a shooting robot, and relates to the technical field of intelligent shooting, and the method comprises the steps: obtaining a frame rate change video segment in an automatic following shooting process of a shooting main body moving at a high speed; extracting a first-frame shot picture and a tail-frame shot picture of the frame rate change video segment, and respectively extracting first-frame relative position data and tail-frame relative position data of the shot subject from the first-frame shot picture and the tail-frame shot picture; generating a fitting motion route based on an AI model; dividing a plurality of video adjustment segments in the frame rate change video segment, and generating sub-fitting routes corresponding to the plurality of video adjustment segments based on an AI model; and integrating each sub-fitting route and the fitting motion route for analysis, and inserting more than one compensation frame picture in each video adjustment segment. And dynamic frame insertion compensation is performed by generating the motion track fitting route and combining the sub-fitting routes of the multiple video adjustment segments, so that the video fluency and the image quality in the high-speed motion scene are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent shooting technology, and in particular to a shooting adjustment method based on AI intelligent optimization and a shooting robot. Background Art

[0002] Existing sports shooting videos, especially those involving sports ball games, require shooting equipment with high precision and high stability. Since the players and the ball need to be captured during the shooting process, the camera equipment needs to follow the players or the ball to move and shoot, which leads to frame drops during the shooting process, affecting the shooting effect. Especially in fast-moving scenes, it is difficult for traditional shooting equipment to capture the complete trajectory of the moving subject in real time, which can easily cause problems such as image freeze and blur. In addition, the existing technology has a relatively simple way of handling frame rate changes, and is unable to perform intelligent compensation according to the actual changes of the moving subject, resulting in uneven quality of the final output video. At the same time, existing shooting robots lack the ability to detect and adaptively adjust frame rate changes in real time, making it difficult to meet the actual needs of professional-grade high-speed motion and high-quality real-time shooting. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings of the existing technology. The present invention provides an AI-based intelligent optimization shooting adjustment method and a shooting robot. By real-time detection of frame rate change video segments and generating motion trajectory fitting routes, dynamic interpolation compensation is performed by combining sub-fitting routes of multiple video adjustment segments, effectively solving the problems of picture freezes and incomplete trajectory capture in high-speed motion scenes, and having the advantages of improving video smoothness and picture quality in high-speed motion scenes.

[0004] The present invention provides a shooting adjustment method based on AI intelligent optimization, comprising the steps of:

[0005] a. Automatically follow the high-speed moving subject during shooting, detect the frame rate changes of the shooting picture in real time, and obtain the frame rate change video segment;

[0006] b. extracting the first frame and the last frame of the video segment with a variable frame rate, and extracting the first frame relative position data and the last frame relative position data of the subject from the first frame and the last frame respectively;

[0007] c. Generate a fitted motion route based on the AI ​​model and the relative position data of the first frame and the last frame;

[0008] d. Divide the frame rate-changing video segment into several video adjustment segments, and generate sub-fitting routes corresponding to the several video adjustment segments based on the AI ​​model;

[0009] e. Integrate each sub-fitting route and fitting motion route for analysis, and insert one or more compensation frame images into each video adjustment segment.

[0010] Furthermore, the step a includes:

[0011] Set a detection sliding window to perform sliding window detection on the real-time shooting picture during the shooting process to obtain detection data;

[0012] Calculating the standard deviation of the frame rate of the detection data to obtain a variation index value;

[0013] When the change index value is greater than a preset detection threshold, the detected data is marked as a frame rate change video segment.

[0014] Furthermore, the step b includes:

[0015] b1. Select a panoramic background image based on the shooting scene involved in the frame rate change video segment;

[0016] b2. Extracting the subject from the first frame of the image and obtaining the first frame relative position data of the subject in the panoramic background image;

[0017] b3. Take a picture at the last frame to extract the subject, and obtain the last frame relative position data of the subject in the panoramic background picture.

[0018] Furthermore, the step c includes:

[0019] c1. Use the AI ​​model to analyze the motion trend of the subject based on the relative position data of the first frame and the relative data of the last frame;

[0020] c2. Insert several blank frames between the first and last frames according to the preset shooting frame rate, and insert the predicted position of the subject in each blank frame based on the motion trend of the subject;

[0021] c3. Construct a fitted motion route based on the predicted position points of several blank frame images.

[0022] Furthermore, the step c2 includes:

[0023] The panoramic background image is taken as the image content of the blank frame, and the AI ​​model is used to combine the motion trend of the subject and the relative position data of the subject in the current frame to predict the position of the subject in the next frame, and the predicted position point of the subject in the next frame is obtained.

[0024] Furthermore, the step d includes:

[0025] d1. Extracting several reference objects from the frame rate-varying video segment, and extracting several process images from the frame rate-varying video segment based on the distribution of the reference objects.

[0026] d2. Divide the frame rate change video segment into a number of video adjustment segments according to the shooting time sequence of the plurality of process pictures;

[0027] d3. Obtain the relative position change data between the shooting reference object and the shooting subject in each video adjustment segment, and generate a sub-fitting route based on the relative position change data through the AI ​​model.

[0028] Furthermore, the step e includes:

[0029] e1. Divide the fitted motion route into a number of fitted route segments, where the fitted route segments correspond to the number of fitted sub-routes in a one-to-one manner;

[0030] e2. Determine the interpolation compensation position point based on the comparative analysis of the fitted route segment and the corresponding fitted sub-route;

[0031] e3. Generate motion estimation values ​​based on the fitted route segments, generate motion sub-data based on the fitted sub-route, and output compensated frame images based on the motion estimation values ​​and motion sub-data through the AI ​​model.

[0032] Furthermore, the step e3 includes:

[0033] Analyze the motion parameters of the shooting subject and the shooting reference object according to the shooting content of each fitting route segment;

[0034] Compare the motion parameters with the preset threshold value and classify the interpolated frame motion level;

[0035] The corresponding frame interpolation method is selected according to the interpolation frame motion level, and a compensation frame image is generated.

[0036] Furthermore, the operation of dividing the interpolation frame motion level is as follows:

[0037] like ,and , the frame interpolation motion level is classified as high-difficulty frame interpolation;

[0038] like ,and , classifying the interpolation frame motion level as low difficulty interpolation frame;

[0039] in, 、 is the proportionality coefficient, is the motion rate of the subject, is the motion rate of the reference object. is the moving direction angle of the subject, The moving direction angle of the reference object.

[0040] The present invention also provides a shooting robot based on AI intelligent optimization shooting adjustment, characterized in that the shooting robot is used to execute the shooting adjustment method based on AI intelligent optimization, and the shooting robot includes:

[0041] The shooting component is used for real-time shooting and transmitting the shooting images to the frame rate processing component in real time;

[0042] The frame rate processing component is used to detect frame rate changes in the captured image and obtain frame rate change video segments;

[0043] The frame rate compensation component is used to perform frame analysis on video segments with frame rate changes and generate compensated frame images.

[0044] The present invention provides an AI-based intelligent optimization shooting adjustment method and a shooting robot. By capturing the status of the shooting subject and the shooting reference object, the method automatically follows the high-speed moving shooting subject during the shooting process, and combines the shooting reference object to perform AI intelligent frame filling operations on the dropped frame video segments. The method can output high-quality motion shooting videos and achieve efficient video shooting optimization and adjustment effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the AI-based intelligent optimization shooting adjustment method in an embodiment of the present invention;

[0046] Figure 2 is a flow chart of the method of step b in an embodiment of the present invention;

[0047] Figure 3 is a flow chart of the method of step c in an embodiment of the present invention;

[0048] Figure 4 is a flow chart of the method of step d in an embodiment of the present invention;

[0049] Figure 5 is a flow chart of the method of step e in an embodiment of the present invention;

[0050] Figure 6 2 is a schematic diagram of the working system of a shooting robot based on AI intelligent optimization shooting adjustment in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] Example 1:

[0053] Figure 1 A flowchart of a shooting adjustment method based on AI intelligent optimization in an embodiment of the present invention is shown, and the adjustment method includes steps a to e.

[0054] Step a: automatically following a high-speed moving subject during shooting, detecting frame rate changes of the shooting image in real time, and obtaining a frame rate change video segment;

[0055] Set a detection sliding window to perform sliding window detection on the real-time shooting picture during the shooting process to obtain detection data;

[0056] Calculating the standard deviation of the frame rate of the detection data to obtain a variation index value;

[0057] When the change index value is greater than a preset detection threshold, the detected data is marked as a frame rate change video segment.

[0058] Among them, the window length of the detection sliding window can be 5 to 15 frames, and the window sliding step can be 1 to 3 frames. The window coverage range is dynamically adjusted to adapt to the detection needs under different motion speeds. During the standard deviation calculation process, the discrete degree of the frame rate data in the window is converted into a continuous numerical indicator through a mathematical formula. For example, the sample standard deviation formula is used to calculate the change index value. The setting of the detection threshold can be dynamically adjusted according to the historical frame rate data. For example, 1.5 to 2.5 times the average value of the historical frame rate standard deviation is taken as the threshold benchmark. The combination of window sliding and standard deviation calculation effectively captures the spatiotemporal continuity of frame rate fluctuations, avoiding the detection blind spots caused by fixed interval sampling.

[0059] Specifically, during real-time shooting, the detection sliding window slides along the time axis at a preset step size, and the frame rate data of all frames in the window are extracted after each slide. By calculating the standard deviation of the frame rate in the window, the discrete frame rate fluctuations are converted into quantitative indicators. For example, when the window contains 10 frames of data, the standard deviation calculation result can reflect the frame rate stability within the 10 frames. When the change index value exceeds the dynamically set detection threshold, it is determined that the video segment corresponding to the current window has abnormal fluctuations. For example, in the shooting of a basketball game, when the athlete suddenly accelerates and causes the frame rate to drop sharply from 60fps to 45fps, the standard deviation in the window increases significantly and triggers a mark. By combining continuous sliding window detection with statistical quantitative analysis, the risk of missing instantaneous fluctuations in a single sampling is avoided, and conventional fluctuation noise is filtered out through quantitative indicators to ensure the accurate positioning of abnormal segments.

[0060] Step b: extracting the first frame and the last frame of the frame rate variable video segment, and extracting the first frame relative position data and the last frame relative position data of the subject from the first frame and the last frame respectively. Step b is specifically implemented by steps b1-b3.

[0061] b1. Select a panoramic background image based on the shooting scene involved in the frame rate change video segment;

[0062] When the subject is photographed in motion, a camera unit at a fixed position is used to perform panoramic photography of the motion scene in which the subject is located, thereby obtaining the motion shooting scene of the subject and extracting the panoramic background image corresponding to the subject during high-speed motion from the panoramic photography image data.

[0063] b2. Extracting the subject from the first frame of the image and obtaining the first frame relative position data of the subject in the panoramic background image;

[0064] Specifically, Figure 2 A flowchart of the method in step b in an embodiment of the present invention is shown; identification of the subject of the first frame image and the last frame image can be achieved through image recognition algorithms, such as the Faster R-CNN model (Fast Region-based Convolutional Neural Networks), the SSD model (Single Shot MultiBox Detector), the YOLO model (You Only Look Once), and other image algorithms, which are not specifically limited here.

[0065] b3. Take a picture at the last frame to extract the subject, and obtain the last frame relative position data of the subject in the panoramic background picture.

[0066] Furthermore, the subject is also extracted from the last frame of the image, and its relative position data in the last frame of the panoramic background image is obtained. In this way, the position data of the first and last frames are based on the same panoramic background coordinate system, ensuring spatial consistency. A unified panoramic background reference system is established based on the panoramic background image, effectively solving the problem of inconsistent coordinate bases in dynamic scenes. By mapping the subject position to the panoramic background coordinate system, base drift caused by lens movement or perspective changes is eliminated. This dual-frame coordinate mapping method based on the panoramic background isolates local image interference, provides a precise spatiotemporal reference for subsequent motion path fitting, and improves the accuracy of motion path fitting.

[0067] Specifically, during dynamic shooting, when a video segment with frame rate changes is detected, the pre-stored panoramic background image is first called as a spatial reference based on the video content features. The first frame is aligned with the panoramic background through image registration technology, and the target detection model is used to locate the bounding box of the shooting subject. The coordinate value of its center point in the panoramic coordinate system is calculated and stored as the starting position. When processing the last frame, the same panoramic background is reused, and the optical flow method is used to track the displacement trajectory of the subject, and finally the end position data based on the unified coordinate system is output. This process controls the error of the head and tail position data within 0.5 pixels by eliminating the difference in local picture perspective, providing an accurate spatiotemporal reference for subsequent motion route fitting. For example, in the shooting of a football game, even if the camera follows the movement of the players and causes the background to shift, the player's displacement data in the standard field coordinate system can still be accurately obtained through the panoramic field map.

[0068] Step c: Generate a fitted motion path based on the AI ​​model combined with the first frame relative position data and the last frame relative position data. Specifically, Figure 3 A method flow chart of step c in an embodiment of the present invention is shown, including steps c1-c3.

[0069] c1. Use the AI ​​model to analyze the motion trend of the subject based on the relative position data of the first frame and the relative data of the last frame;

[0070] Furthermore, the AI ​​model can be trained based on a neural network model, and the AI ​​model can be trained using different sample data of multiple shooting subjects, so that the AI ​​model can generate the movement route of the shooting subject based on the first frame and the last frame.

[0071] c2. Insert several blank frames between the first and last frames according to the preset shooting frame rate, and insert the predicted position of the subject in each blank frame based on the motion trend of the subject;

[0072] c3. Construct a fitted motion route based on the predicted position points of several blank frame images.

[0073] Specifically, step c2 includes: taking the panoramic background picture as the image content of the blank frame, using the AI ​​model to combine the motion trend of the subject and the relative position data of the subject in the current frame picture to predict the position of the subject in the next frame picture, and obtain the predicted position point of the subject in the next frame picture.

[0074] Specifically, the AI ​​model is configured to analyze nonlinear motion features, such as acceleration or sudden changes in direction, in the first and last frame position data, and extract the underlying patterns of the motion trajectory through a deep learning algorithm. The number of blank frames inserted is set by the actual frame rate of the original video segment. For example, when the target frame rate is 60fps, 58 blank frames need to be inserted between the first and last frames. This allows the AI ​​model to compensate for the required frame rate and simulate the motion path of the subject at the preset frame rate.

[0075] Furthermore, the generation of predicted position points adopts a time series prediction model, which takes the first and last frame positions as input and outputs the position coordinates of the intermediate moments. The loss function used in model training may include velocity continuity constraints and acceleration smoothness constraints, so as to generate the motion path of the subject in continuous frame images.

[0076] Specifically, when a blank frame is inserted between the first and last frames, the timestamp of each blank frame is evenly distributed according to the preset frame rate, for example, 60 equally spaced time points are inserted within a 1-second interval. The AI ​​model analyzes the position difference between the first and last frames and predicts the position coordinates of the intermediate moments in combination with kinematic equations or neural networks, such as using LSTM networks to capture time-dependent features. In the process of generating predicted position points, an error correction mechanism can be introduced. For example, when the speed change of adjacent predicted points exceeds a threshold, the generation parameters of subsequent predicted points are automatically adjusted. The predicted position points in the blank frame image are converted into specific pixel positions in the image through a coordinate mapping algorithm, such as projecting the three-dimensional space coordinates onto the two-dimensional image plane.

[0077] The fitted motion path is constructed using a spline interpolation algorithm, connecting discrete predicted position points into a continuous trajectory. The interpolation algorithm parameters are dynamically adjusted based on the motion trend, such as using Bezier curve fitting in high-speed rotation scenarios. This effectively improves the accuracy and real-time performance of motion compensation by processing the first and last frame data in stages, dynamically generating intermediate prediction points, and optimizing the trajectory fitting algorithm.

[0078] Furthermore, the panoramic background picture is generated by performing 360-degree scene capture in the pre-shooting stage, and its resolution is consistent with the real-time shooting picture. The AI ​​model adopts a convolutional neural network architecture, and the input end receives a data packet containing a motion trend vector and the relative position coordinates of the current frame. The motion trend vector is composed of a speed component and a direction angle, and the relative position coordinates of the current frame are based on the panoramic background picture coordinate system.

[0079] During the prediction process, the AI ​​model updates the position benchmark data in real time at a processing frequency of 30 frames per second, and the fusion weight of the motion trend vector and the position coordinates is dynamically adjusted according to the complexity of the scene.

[0080] Step d: Divide the frame rate change video segment into several video adjustment segments, and generate sub-fitting routes corresponding to the several video adjustment segments based on the AI ​​model. Specifically, Figure 4 A method flow chart of step d in an embodiment of the present invention is shown, which specifically includes steps d1-d3.

[0081] d1. Extract several shooting reference objects from the frame rate-varying video segment, and extract several process shooting pictures within the frame rate-varying video segment based on the distribution of the shooting reference objects. The extraction of the shooting reference objects can be implemented using an algorithm based on edge detection or feature point matching, such as using the SIFT algorithm to identify static objects in the scene as reference objects.

[0082] d2. Divide the frame rate-varied video segment into several video adjustment segments based on the chronological order of the process images. The sampling interval for extracting process images can be set based on the reference object distribution density. For example, when the reference object distribution density exceeds 5 feature points per square pixel, extract process images at a rate of 10 frames per second. The division of video adjustment segments should take into account both the chronological order and the reference object distribution characteristics. For example, consecutive segments of process images with a time interval of less than 0.5 seconds can be combined into a single video adjustment segment.

[0083] d3. Obtain the relative position change data between the shooting reference object and the shooting subject in each video adjustment segment, and generate a sub-fitting route based on the relative position change data through the AI ​​model.

[0084] The acquisition of relative position change data can be achieved by calculating the displacement vector between the photographed subject and the reference object through the optical flow method. The AI ​​model can use the LSTM network to analyze the temporal change pattern of the displacement vector and generate a sub-fitting route that matches the fitted motion route.

[0085] Specifically, in video segments with varying frame rates, a computer vision algorithm first identifies objects with stable spatial positions in the scene as reference objects, such as billboards or fixed structures in stadiums. Based on the spatial distribution density of the reference objects, a pre-set sampling frequency is used to extract process images containing the complete distribution characteristics of the reference objects, for example, one frame every 0.3 seconds. The video segments corresponding to the consecutive process images are then divided into video adjustment segments in chronological order, with each segment ranging in length from 0.5 to 2 seconds. For each video adjustment segment, a dynamic dataset containing three-dimensional coordinate changes is generated by tracking the relative displacement between the subject and the reference object. The AI ​​model analyzes the direction, velocity, and acceleration parameters of the displacement vectors in this dataset to predict the subject's motion trajectory within the sub-time period and generate a sub-fitting route that is spatially continuous with the overall fitted motion route. This expands the basis for segmenting video adjustment segments from a purely temporal dimension to a spatial feature dimension, enabling the sub-fitting routes to dynamically reflect the coordinated motion patterns of the subject and the scene reference objects, providing a precise spatial positional reference for compensatory frame insertion.

[0086] Step e: Integrate each sub-fitting route and the fitting motion route for analysis, and insert one or more compensation frame images into each video adjustment segment. Specifically, Figure 5 A method flow chart of step e in an embodiment of the present invention is shown, including steps e1-e3.

[0087] e1. Divide the fitted motion route into a number of fitted route segments, where the fitted route segments correspond to the number of fitted sub-routes in a one-to-one manner;

[0088] e2. Determine the interpolation compensation position point based on the comparative analysis of the fitted route segment and the corresponding fitted sub-route;

[0089] e3. Generate motion estimation values ​​based on the fitted route segments, generate motion sub-data based on the fitted sub-route, and output compensated frame images based on the motion estimation values ​​and motion sub-data through the AI ​​model.

[0090] Step e3 specifically includes analyzing the motion parameters of the subject and reference object based on the captured content of each fitted route segment. Specifically, computer vision algorithms can be used to track the position changes of the subject and reference object in consecutive frames to obtain parameters such as their motion speed, acceleration, and direction. For example, optical flow methods or object tracking algorithms can be used to implement this process.

[0091] Motion parameters are compared with preset thresholds to classify interpolated frames into motion levels. Multiple thresholds can be set to define different motion levels. For example, motion speeds less than 5 pixels / frame can be defined as low-speed motion, 5-15 pixels / frame as medium-speed motion, and greater than 15 pixels / frame as high-speed motion.

[0092] The corresponding frame interpolation method is selected according to the interpolation frame motion level, and a compensation frame image is generated.

[0093] Motion parameters may include at least one of velocity and directional angle. Preset thresholds can be dynamically adjusted based on different scenarios. For example, in sports events, the velocity threshold is set to 5 m / s, and the directional angle difference threshold is set to 30 degrees. The interpolation motion level can be divided using a linear weighting method. For example, the weighted sum of the velocities of the subject and the reference object is compared with a preset threshold. When the weighted sum exceeds the threshold, the interpolation level is classified as high difficulty. The selection of the interpolation method may include switching the interpolation algorithm type or generation logic. For example, at a high difficulty level, the optical flow method combined with bidirectional motion estimation is used to generate compensation frames, while at a low difficulty level, a linear interpolation algorithm is used.

[0094] Specifically, after integrating the main route and sub-routes, the motion parameters of the subject and the reference object are extracted by analyzing the shooting content within each route. For example, the displacement of the subject between consecutive frames is calculated by feature point matching, and the motion trajectory of the reference object is obtained by background optical flow analysis. When comparing the motion parameters with the preset thresholds, a multi-dimensional evaluation model is used. For example, the speed and direction angle are assigned different weight coefficients and then a comprehensive score is performed. The score results are mapped into three levels of frame supplementation: high, medium, and low. When selecting the frame supplementation method based on the level, the pre-trained high-precision motion prediction model is called to generate compensation frames in high-difficulty scenarios. The number of compensation frames can be dynamically increased to 1.5 times the number of original frames; in low-difficulty scenarios, a lightweight algorithm based on position interpolation is used to generate compensation frames. By dynamically matching the frame supplementation strategy and motion complexity, the matching error between the compensation frame and the actual motion trajectory can be reduced to within 3 pixels, thereby ensuring the continuity and stability of the video image in high-speed motion scenarios.

[0095] The division operation of the interpolation frame motion level is as follows:

[0096] like ,and , the frame interpolation motion level is classified as high-difficulty frame interpolation;

[0097] like ,and , classifying the interpolation frame motion level as low difficulty interpolation frame;

[0098] in, 、 is the proportionality coefficient, is the motion rate of the subject, is the motion rate of the reference object. is the moving direction angle of the subject, The moving direction angle of the reference object.

[0099] Furthermore, during the motion change data generation phase, the subject's motion trajectory extracted from the first and last frame images is quantized into parameters S1 and θ1, while the reference object's motion trajectory extracted from the subsequent images is quantized into parameters S2 and θ2. When segmenting the motion-compensated video, the corresponding α and β coefficients are dynamically configured based on the scene type.

[0100] Specifically, during the processing of motion-compensated video segments, the coordinate change sequence of the subject in consecutive frames is first extracted through a feature matching algorithm, and the displacement of fixed reference objects in the background is detected at the same time.

[0101] The subject's velocity is calculated by dividing the difference in displacement between consecutive frames by the time interval. For example, if a football player is detected to move 15 pixels in 0.1 seconds, the motion rate is 150 pixels / second. The direction angle is calculated from the angle formed by the coordinate points of three consecutive frames. If a basketball player suddenly stops and changes direction, causing the angle to change by more than 45 degrees, it is considered a sudden change in direction.

[0102] During the frame supplementation level division stage, the motion parameters of the subject and the reference object are input into a weighted calculation formula. For example, when the athlete's rate weight α is set to 0.7 and the reference object's rate weight β is set to 0.3, the high-difficulty frame supplementation mode is triggered when the comprehensive motion index exceeds the preset threshold of 8.5.

[0103] Furthermore, the frame interpolation method selection module calls the corresponding algorithm library based on the level identifier. In high-difficulty mode, a frame interpolation model based on PWC-Net optical flow estimation is enabled, while in low-difficulty mode, phase correlation motion compensation is used. By dynamically adjusting the frame interpolation strategy, the video processing system can flexibly select the interpolation mode while maintaining computational efficiency, improving the efficiency of the overall system operation.

[0104] Specifically, the shooting adjustment method further includes:

[0105] Before filming, take photos of the filming environment to obtain environmental image data;

[0106] The background of each frame of the captured image is optimized based on the environmental image data.

[0107] Scene capture can include selecting multiple sampling points within a scene for multi-angle capture. The sampling point spacing can be set to 0.5-1.2 meters, and the number of scene captures can be 3-5. Environmental image data can include parameters such as light intensity, color temperature distribution, and background texture characteristics.

[0108] Furthermore, the background optimization process may include dividing the background area in the real-time picture into 8×8 pixel blocks, performing feature comparison with pre-stored environmental data, triggering noise reduction processing when the matching degree is lower than 85%, and performing optimization processing through brightness compensation, contrast correction, noise reduction intensity adjustment, etc., so that the captured video picture can be clear and smooth.

[0109] Specifically, before shooting, the shooting environment can be photographed to obtain a panoramic background picture, and the background of each frame of the shooting picture can be optimized based on the panoramic background picture, so that the overall quality of the shot video is improved.

[0110] Scene capture can be done using either surround multi-angle or fixed-point scanning methods. For example, a drone equipped with a multi-lens array can fly 360 degrees around the scene to collect image data covering the entire scene. This image data is processed using a 3D reconstruction algorithm to generate a panoramic background image. The resolution can be set to 4K, and the size of a single image is controlled within 8000×6000 pixels. During background optimization, an image segmentation algorithm is used to separate the dynamic subject from the static background. For example, a U-Net neural network model is used to perform semantic segmentation on each frame. The segmented dynamic subject area is then fused with the panoramic background at the pixel level. The transparency parameter is set to 0.9 during fusion to ensure a natural transition between the subject edges.

[0111] Specifically, before filming begins, a camera equipped with a wide-angle lens captures the target scene from multiple angles. For example, a moving track along the roof of the auditorium in a stadium captures a panoramic image covering the playing field. During capture, an exposure time of 1 / 1000 second is set to eliminate motion blur, and three sets of images with different exposure parameters are captured using HDR mode. An image stitching algorithm is used to combine these images into a panoramic background image, which is stored as a PNG file with a transparency channel. During subsequent video processing, a background difference method is used to extract the outline of the moving subject from each frame. This outline is then mapped to the corresponding coordinates of the panoramic background, and bilinear interpolation is used to eliminate stitching gaps. When inserting compensation frames, the panoramic background is directly used as the underlying image, and the predicted trajectory of the moving subject is combined to generate a complete compensation frame image, ensuring spatial consistency of background elements across the timeline. By establishing a unified background reference library, this solution ensures that video frames at different time points share the same background data, effectively eliminating background jumps caused by lighting changes or temporary obstructions.

[0112] The core innovation of this application lies in combining dynamic frame rate detection with multi-level motion trajectory analysis. Through a dual mechanism of segmented fitting and global path integration, this approach enables adaptive generation of compensation frames in high-speed motion scenes. This solution breaks through the limitations of traditional interpolation techniques, which rely on fixed algorithms, and utilizes AI models to intelligently predict nonlinear motion, effectively eliminating image lag while reducing computational load. It is particularly suitable for optimizing the shooting of complex motion trajectories such as sporting events.

[0113] The operating process and principle of this application are as follows: during the automatic tracking of a high-speed moving subject, the frame rate changes of the captured image are first detected in real time to obtain a frame rate change video segment. This step is performed by setting a detection sliding window to perform sliding window detection on the real-time captured image, obtaining detection data, and calculating the standard deviation of the detection data's frame rate to obtain a change index value. When the change index value exceeds a preset detection threshold, the detection data is marked as a frame rate change video segment.

[0114] Next, the first and last frames of the frame rate-varying video segment are extracted, and the subject's first-frame relative position data and last-frame relative position data are extracted from these two images, respectively. This operation involves selecting a panoramic background image based on the scene captured by the frame rate-varying video segment, extracting the subject from the first and last frames, and obtaining the subject's relative position data within the panoramic background image.

[0115] Then, the AI ​​model combines the relative position data of the first and last frames to generate a fitted motion path. The AI ​​model first analyzes the subject's motion trends, inserts several blank frames between the first and last frames based on the preset shooting frame rate, and inserts the predicted subject position points in each blank frame. Finally, the fitted motion path is constructed based on these predicted position points.

[0116] Furthermore, the frame rate-varying video segment is divided into several video adjustment segments, and sub-fitting routes corresponding to these video adjustment segments are generated based on the AI ​​model. This step involves extracting several reference objects from the frame rate-varying video segment, extracting several process images based on the distribution of the reference objects, and dividing the video adjustment segments based on the chronological order of these images. For each video adjustment segment, data on the relative position change between the reference object and the subject is obtained, and sub-fitting routes are generated using the AI ​​model.

[0117] Finally, each sub-fitting route and fitted motion route are integrated and analyzed, and one or more compensation frame images are inserted into each video adjustment segment. This operation involves dividing the fitted motion route into several fitting route segments, which correspond one to each sub-fitting route. Comparative analysis determines the insertion compensation points, generates motion estimates based on the fitting route segments, and generates motion sub-data based on the fitting sub-routes. Finally, the AI ​​model outputs the compensation frame images.

[0118] Example 2:

[0119] Figure 6 A schematic diagram of the working system of a shooting robot based on AI intelligent optimization shooting adjustment in an embodiment of the present invention is shown. The shooting robot is used to perform the shooting adjustment method based on AI intelligent optimization. The shooting robot includes:

[0120] Shooting component 10: used for real-time shooting and transmitting the shooting images to the frame rate processing component in real time;

[0121] The shooting component 10 can use a CMOS sensor or a CCD sensor to achieve real-time image acquisition, and send the original image data to the frame rate processing component through the MIPI interface at a transmission rate of not less than 30fps.

[0122] Frame rate processing component 20: used to detect frame rate changes of the captured image and obtain frame rate change video segments;

[0123] The frame rate processing component 20 can integrate an FPGA chip or a dedicated ASIC processor. After receiving the image data, it uses a built-in frame rate detection algorithm to scan the image stream once every millisecond. When the standard deviation of the time interval between adjacent frames exceeds a preset threshold, it immediately triggers a video segment marking mechanism and caches the abnormal video segment to the DDR4 memory module. The frame rate compensation component can also be equipped with a GPU acceleration unit.

[0124] Frame rate compensation component 30: used to perform frame analysis on the frame rate-varied video segment and generate compensated frame images.

[0125] The OpenCL framework is used to analyze the motion vectors of the marked video segments, and the motion trajectories of the subject and background in the picture are calculated by the optical flow method to generate intermediate compensation frames.

[0126] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0127] In addition, the above provides a detailed introduction to the embodiments of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A shooting adjustment method based on AI intelligent optimization, characterized in that: Including steps: a. Automatically follow the high-speed moving subject during shooting, detect the frame rate changes of the shooting picture in real time, and obtain the frame rate change video segment; b. extracting the first frame and the last frame of the video segment with a variable frame rate, and extracting the first frame relative position data and the last frame relative position data of the subject from the first frame and the last frame respectively; c. Generate a fitted motion route based on the AI ​​model and the relative position data of the first frame and the last frame; d. Divide the frame rate-changing video segment into several video adjustment segments, and generate sub-fitting routes corresponding to the several video adjustment segments based on the AI ​​model; e. Integrate each sub-fitting route and fitting motion route for analysis, and insert one or more compensation frame images into each video adjustment segment.

2. The AI-based intelligent optimization shooting adjustment method according to claim 1, characterized in that: The step a comprises: Set a detection sliding window to perform sliding window detection on the real-time shooting picture during the shooting process to obtain detection data; Calculating the standard deviation of the frame rate of the detection data to obtain a variation index value; When the change index value is greater than a preset detection threshold, the detected data is marked as a frame rate change video segment.

3. The AI-based intelligent optimization shooting adjustment method according to claim 1, characterized in that: The step b comprises: b1. Select a panoramic background image based on the shooting scene involved in the frame rate change video segment; b2. Extracting the subject from the first frame of the image and obtaining the first frame relative position data of the subject in the panoramic background image; b3. Take a picture at the last frame to extract the subject, and obtain the last frame relative position data of the subject in the panoramic background picture.

4. The AI-based intelligent optimization shooting adjustment method according to claim 1, characterized in that: The step c comprises: c1. Use the AI ​​model to analyze the motion trend of the subject based on the relative position data of the first and last frames; c2. Insert several blank frames between the first and last frames according to the preset shooting frame rate, and insert the predicted position of the subject in each blank frame based on the motion trend of the subject; c3. Construct a fitted motion route based on the predicted position points of several blank frame images.

5. The AI-based intelligent optimization shooting adjustment method according to claim 4, characterized in that: The step c2 comprises: The panoramic background image is taken as the image content of the blank frame, and the AI ​​model is used to combine the motion trend of the subject and the relative position data of the subject in the current frame to predict the position of the subject in the next frame, and the predicted position point of the subject in the next frame is obtained.

6. The AI-based intelligent optimization shooting adjustment method according to claim 1, characterized in that: The step d comprises: d1. Extracting several reference objects from the frame rate-varying video segment, and extracting several process images from the frame rate-varying video segment based on the distribution of the reference objects. d2. Divide the frame rate change video segment into a number of video adjustment segments according to the shooting time sequence of the plurality of process pictures; d3. Obtain the relative position change data between the shooting reference object and the shooting subject in each video adjustment segment, and generate a sub-fitting route based on the relative position change data through the AI ​​model.

7. The AI-based intelligent optimization shooting adjustment method according to claim 1, characterized in that: The step e comprises: e1. Divide the fitted motion route into a number of fitted route segments, where the fitted route segments correspond to the number of fitted sub-routes in a one-to-one manner; e2. Determine the interpolation compensation position point based on the comparative analysis of the fitted route segment and the corresponding fitted sub-route; e3. Generate motion estimation values ​​based on the fitted route segments, generate motion sub-data based on the fitted sub-route, and output compensated frame images based on the motion estimation values ​​and motion sub-data through the AI ​​model.

8. The AI-based intelligent optimization shooting adjustment method according to claim 7, characterized in that: The step e3 comprises: Analyze the motion parameters of the shooting subject and the shooting reference object according to the shooting content of each fitting route segment; Compare the motion parameters with the preset threshold value and classify the interpolated frame motion level; The corresponding frame interpolation method is selected according to the interpolation frame motion level, and a compensation frame image is generated.

9. The AI-based intelligent optimization shooting adjustment method according to claim 8, characterized in that: The division operation of the interpolation frame motion level is as follows: like ,and , the frame interpolation motion level is classified as high-difficulty frame interpolation; like ,and , classifying the interpolation frame motion level as low difficulty interpolation frame; in, 、 is the proportionality coefficient, is the motion rate of the subject, is the motion rate of the reference object. is the moving direction angle of the subject, The moving direction angle of the reference object.

10. A shooting robot based on AI intelligent optimization shooting adjustment, characterized in that: The shooting robot is used to execute the AI ​​intelligent optimization shooting adjustment method according to any one of claims 1 to 9, and the shooting robot includes: The shooting component is used for real-time shooting and transmitting the shooting images to the frame rate processing component in real time; The frame rate processing component is used to detect frame rate changes in the captured image and obtain frame rate change video segments; The frame rate compensation component is used to perform frame analysis on video segments with frame rate changes and generate compensated frame images.

Images