Video clip processing method based on artificial intelligence

Abstract

The application discloses a video clip processing method based on artificial intelligence and relates to the technical field of video processing, and comprises the following steps: establishing a light flash feature description rule for a light rapid jump scene, converting each time of luminance mutation appearing in a video frame into a light flash event, and forming a light flash event sequence from continuous light flash events to construct a light flash track; screening time segments with the largest luminance change amplitude on the light flash track, and extracting segments with stable human expression and action from the time segments, taking the screened segments as emotional benchmark segments to form an emotional baseline. The application enables the expression of emotional changes to be closer to the real state of a character through light flash track, emotional baseline and emotional curve compression, and weakens emotional interference caused by light jump; meanwhile, self-adjusting editing rhythm control is added in the smooth emotional band, so that the rhythm can be adjusted according to emotional changes and light disturbance can be avoided, thereby improving the coherent rhythm and emotional expression accuracy of a finished film.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and more specifically to a video editing method based on artificial intelligence. Background Technology

[0002] Artificial intelligence-based video editing refers to the use of deep learning models, computer vision algorithms, and semantic understanding technology to automatically identify and structurally analyze multi-dimensional features in the original video, such as characters, scenes, camera movements, audio rhythm, and emotional changes. Based on a set editing style, rhythm rules, or target task, it performs processing steps such as shot segmentation, highlight extraction, rhythm rearrangement, transition generation, audio-visual synchronization, and keyframe optimization. This automatically generates a coherent, aesthetically pleasing video product that conforms to a specific expressive intent without requiring manual frame-by-frame manipulation. Essentially, this technology enables artificial intelligence to understand "what the video is saying," "where the key points are," and "how it should be presented," achieving an intelligent, automated, and even stylized video editing process, significantly improving editing efficiency and final product quality.

[0003] Existing technologies suffer from the following shortcomings: Current AI video editing algorithms typically rely on changes in screen brightness, rhythmic distribution, and facial posture to determine the emotional trend of a scene, using this as a crucial basis for controlling the editing rhythm. However, when video content is in environments with rapidly fluctuating lighting, such as stage performances, light shows, or fire rescue scenes, where screen brightness experiences rapid flickering, localized overexposure, or instantaneous darkening within a very short time, existing technologies often fail to effectively distinguish between "lighting change signals" and "emotional change signals." Because the model lacks a robust correction mechanism for light disturbances during training, AI systems are prone to misinterpreting intense flickering as drastic emotional fluctuations, leading to a reversed amplification of the emotional curve and causing the algorithm to incorrectly determine that the character is in a state of panic, tension, or confusion. This misjudgment further affects the editing logic structure, resulting in an unrealistic emotional trajectory in the generated video rhythm, disrupting narrative coherence and visual emotional consistency. For finished scenes requiring high emotional guidance, such as propaganda, reports, and emergency recordings, this misjudgment can be seriously misleading, even causing the public to misunderstand the nature of the event, the characters' emotions, or the emergency response.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an artificial intelligence-based video editing method to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a video editing processing method based on artificial intelligence, comprising the following steps:

[0007] For scenes with rapid changes in lighting, a light flicker feature description rule is established. Every brightness change that occurs in a video frame is converted into a light flicker event, and consecutive light flicker events are formed into a light flicker event sequence to construct a light flicker trajectory, which is used to distinguish between changes in lighting and changes in emotion in subsequent processing.

[0008] The time segments with the largest brightness changes are selected from the light flash trajectory, and segments where the character's facial expressions and movements remain stable are extracted from these time segments. The selected segments are used as emotional baseline segments to form an emotional baseline, which is used to establish a benchmark reference for emotional changes.

[0009] Based on the emotion baseline, segment-by-segment facial expression and action comparison is performed on all time periods covered by the light flash trajectory. Time periods that are accompanied only by the light flash trajectory and do not show significant changes in facial expression and action are identified and recorded as a light disturbance list to mark the time intervals of interference caused by light.

[0010] Within the time interval corresponding to the light disturbance list, the original emotion curve is subjected to amplitude compression processing, compressing the fluctuation amplitude of the emotion curve in the light disturbance interval to a change level consistent with the emotion baseline, so as to generate a smooth emotion band compatible with light disturbance.

[0011] By incorporating a self-regulating editing rhythm control strategy into the smooth mood band, the editing rhythm is adjusted according to changes in the characters' facial expressions and actions during the process. It also automatically avoids the light flash trajectories corresponding to the lighting disturbance list during the rhythm control process, thereby constructing a dynamic avoidance editing process for scenes with rapid changes in lighting.

[0012] Preferably, the process for constructing the light flash trajectory is as follows:

[0013] The video to be processed is read frame by frame in chronological order, and the average brightness value, the local brightness value of multiple divided regions, and the pixel number distribution of brightness at different gray levels are extracted from each frame. The brightness information of the current frame is compared with the brightness information of the previous frame item by item to identify brightness jumps.

[0014] Each brightness jump is converted into a flash event, and the numerical difference before and after the brightness change, the direction of the brightness change, the range of the brightness change area, the duration of the brightness change in frames, and the video content scene type before and after the brightness change are associated with the same flash event.

[0015] All flash events are constructed into a flash event time chain in chronological order, and adjacent flash events are combined into a flash event sequence based on differences in light intensity, consistency of light direction, and continuity of video content.

[0016] All light flash event sequences are connected along the time axis to form a light flash trajectory, and the distribution location of light jumps, the duration of light jumps, and the order of light jumps are obtained based on the light flash trajectory.

[0017] Preferably, the steps for selecting the time segments with the largest brightness changes along the light flash trajectory and extracting segments where the character's facial expressions and movements remain stable to form an emotional baseline are as follows:

[0018] The sequence of light flash events in the light flash trajectory is analyzed one by one. The difference in brightness before and after the change of brightness for each light flash event is extracted and sorted according to the magnitude of brightness change to obtain the time segment with the largest brightness change. The start time, end time, direction of light change, and area of ​​light change of the time segment are recorded.

[0019] Within the time segment with the greatest brightness variation, observe frame by frame the changes in the shape of the eyes, the opening and closing of the mouth, the position of the eyebrows, the shape of the cheek muscles, the tilt of the head, and the changes in body posture. Identify frames where facial expressions and movements remain stable and combine them into stable facial expression and movement segments.

[0020] All stable facial expression segments are arranged according to their order of appearance in the light flash trajectory, and combined to form emotional baseline segments based on the common characteristics of maintaining calm facial expressions and movements.

[0021] The emotional expression corresponding to the emotional baseline segment is mapped onto a unified emotional reference line and connected to form an emotional baseline for subsequent comparison of emotional changes.

[0022] Preferably, among the arranged stable facial expression segments, only segments where the character's eye opening and closing state remains constant, the mouth curve remains smooth, the eyebrow position does not change abruptly, the head posture is continuous and consistent, and the background content is stable are included in the emotional baseline segment. This ensures that the emotional baseline segment maintains a consistent emotional expression throughout the entire lighting transition process, thereby improving the stability and reference accuracy of the emotional baseline.

[0023] Preferably, the steps for performing segment-by-segment facial expression and action comparison and generating a list of illumination perturbations based on the emotional baseline across the entire time period covered by the light flash trajectory are as follows:

[0024] Based on the light flash trajectory, the start time, end time, duration of light change, area covered by light change, and intensity of light change of each light flash event sequence are read and organized into time intervals to be compared;

[0025] Within each comparison time interval, the range of eye opening and closing, mouth contour changes, eyebrow shape changes, head rotation angle and body posture changes of the person are extracted frame by frame, and compared with the emotional level of the emotional baseline to identify frames where facial expressions and movements remain stable.

[0026] Within each comparison time interval, find a continuous frame sequence in which facial expressions and emotions remain consistent with the baseline, and record the start time, end time, duration, intensity of light changes, and the stable expression and body posture of the person in the sequence to form interference time intervals.

[0027] Record the start and end times, duration, sequence number of light flash events, amplitude of light change, spatial area of ​​light change, and baseline level of mood for all disturbance periods as a light disturbance list and arrange them in chronological order.

[0028] Preferably, in the step of forming the lighting disturbance list, the lighting intensity characteristics of each disturbance time period and the stable performance of the character's facial expressions and movements are used as recording conditions. After recording, all disturbance time periods are rearranged according to their chronological order, so that the lighting disturbance list can continuously cover all lighting change positions and provide an accurate reference for subsequent emotional curve compression processing.

[0029] Preferably, the steps for performing amplitude compression processing on the original mood curve and forming a smooth mood band within the time interval corresponding to the illumination disturbance list are as follows:

[0030] Obtain the original mood curve and mark the start and end times in the light disturbance list as the intervals of the mood curve to be processed, and set the front buffer time and back buffer time for each interval.

[0031] Within each interval of the emotion curve to be processed, the emotion baseline value at the corresponding time point is read, and the upper and lower limit emotion values ​​are set as the compression range based on the changing trend of the emotion baseline in the interval.

[0032] Each interval of the emotion curve to be processed is divided into sub-segments according to the natural change pattern of the original emotion curve, and the original emotion value is adjusted to fall within the compression range at each time point, and a continuous transition is established between adjacent time points.

[0033] Each compressed, locally smoothed emotion curve segment replaces the corresponding interval of the original emotion curve, and transition processing is performed at the connection points to form a smooth emotion band covering the entire video.

[0034] Preferably, the steps for incorporating a self-regulating editing rhythm control strategy into a smooth emotional band are as follows:

[0035] Based on the emotional intensity level, direction of emotional change, speed of emotional change, and pattern of emotional fluctuations of the smooth emotional band on the time axis, the smooth emotional band is divided into multiple emotional stages and forms a rhythm reference band.

[0036] Based on the rhythm reference, self-adjusting rules for editing rhythm are set for each emotional stage, and the shot dwell time, number of shot cuts and image presentation are set according to the emotional rise, emotional stability, emotional fall and slow emotional fluctuation.

[0037] The list of light disturbances and the light flash trajectory are superimposed on the rhythm reference band, and when generating rhythm change points, the time interval of the light disturbances is compared, and the rhythm change points that fall into the disturbance time interval are adjusted to outside the disturbance time interval.

[0038] All rhythmic change points are arranged in chronological order, and editing operations are assigned to each rhythmic change point to ensure that the editing rhythm is consistent with the emotional state corresponding to the smooth mood band and avoids the time interval of lighting disturbance.

[0039] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0040] This invention constructs light flicker trajectories, establishes an emotional baseline, and compresses the amplitude of lighting disturbances, making the emotional curve more closely resemble the true emotional trajectory of a character. When rapid changes in lighting cause drastic fluctuations in screen brightness, the emotional curve no longer exhibits irrelevant fluctuations with these brightness changes, but instead maintains a stable and continuous change pattern. The resulting emotional expression accurately reflects the character's emotional changes throughout the video content, providing a reliable foundation for subsequent editing rhythm control and making the final film more consistent with the logic of real emotions in its narrative expression. Through this method, false emotional fluctuations caused by lighting interference are effectively weakened, ensuring the consistency of the final film's visual emotional presentation.

[0041] This invention introduces a self-adjusting editing rhythm control strategy within a smooth emotional band, allowing the editing rhythm to naturally adjust with changes in the character's emotions. During editing, it automatically avoids lighting disturbances and flickering light trajectories, achieving synchronization between rhythm changes and the character's emotions. In scenes with strong, jittery lighting, the editing rhythm is not disrupted by brightness jumps, but maintains a continuous, stable rhythmic trajectory that aligns with narrative needs. The resulting video exhibits improved rhythm presentation, content expression, and visual stability, avoiding abrupt shot transitions or emotional rhythm shifts caused by lighting fluctuations, resulting in a final product with higher expressive credibility and visual comfort. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0043] Figure 1This is a flowchart of the video editing processing method based on artificial intelligence according to the present invention. Detailed Implementation

[0044] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0045] This invention provides, for example Figure 1 The AI-based video editing method shown includes the following steps:

[0046] For scenes with rapid changes in lighting, a light flicker feature description rule is established. Every brightness change that occurs in a video frame is converted into a light flicker event, and consecutive light flicker events are formed into a light flicker event sequence to construct a light flicker trajectory, which is used to distinguish between changes in lighting and changes in emotion in subsequent processing.

[0047] To establish flicker feature description rules in scenes with rapid changes in lighting, and to convert each brightness change in a video frame into a flicker event, and then form a flicker event sequence from consecutive flicker events to construct a flicker trajectory, which is used to distinguish between lighting changes and emotional changes in subsequent processing, the specific operations can be carried out as follows:

[0048] The video to be processed is read frame by frame in chronological order. For each frame, the brightness information within the spatial range of that frame is extracted. The brightness information can include the average brightness value of the entire frame, the local brightness values ​​of multiple divided regions within the frame, and the distribution of the number of pixels at different gray levels within the frame. After extracting the brightness information of the current frame, the brightness information of the current frame is compared item by item with the brightness information of the immediately preceding frame. By comprehensively observing the changes in average brightness, local brightness, and the trend of brightness distribution, if a significant jump in overall brightness or local brightness is found between the current frame and the previous frame, and this jump is completed within a single frame time interval, a brightness mutation is recorded at the time position of the current frame. The time of occurrence of the brightness mutation, the corresponding video frame number, the difference in brightness before and after the change, the direction of the brightness change, and the spatial region where the brightness change is concentrated are all archived and stored. This ensures that each brightness mutation has clear temporal, spatial, and intensity attributes, thus providing complete basic information for subsequently converting brightness mutations into light flicker events.

[0049] After marking all brightness abrupt changes in the video, a corresponding flash event is constructed for each brightness abrupt change record. The flash event is used as the basic unit to describe the behavior of a single brightness jump. When constructing the flash event, all information in the aforementioned brightness abrupt change record, such as the time position, video frame number, brightness values ​​before and after the change, direction of the brightness change, image area range involved in the brightness change, number of frames the brightness change lasts, and video content scene type before and after the brightness change, are associated with the same flash event. This ensures that each flash event not only reflects the change in light intensity but also the correspondence between the light change and the video content. Based on this, all flash events are sorted according to the video playback timeline, with earlier flash events appearing before later flash events. In this way, a time chain of flash events arranged in chronological order is formed, making the positional relationship of each flash event on the timeline clearly visible. This provides conditions for subsequent combination of flash events from the perspective of temporal continuity.

[0050] After obtaining the time chain of light flash events arranged chronologically, the time chain is further refined. A joint judgment is made based on the time interval between adjacent light flash events, the difference in light intensity, the consistency of the direction of light change, and the continuity of the scene in which the light changes occur. When several light flash events occur consecutively within a short time interval, and these events exhibit the same direction of light change or alternate between increasing and decreasing intensity, and the video content corresponding to these events belongs to the same scene type or is in a continuous scene transition process, this group of events that is closely related in time is considered a time-series event. Furthermore, light flash events that are intrinsically related in terms of illumination change are grouped into the same light flash event sequence. When constructing a light flash event sequence, the sequence is assigned attributes such as start time, end time, number of light flash events, order of light flash events in the sequence, overall trend of illumination change in the sequence, and distribution density of illumination change in different time slices. This makes the light flash event sequence not only a simple collection of several light flash events, but also capable of reflecting the characteristics of illumination jump process. Thus, the illumination abrupt change behavior that was originally scattered at multiple times is integrated into a time-continuous illumination change process description unit.

[0051] After obtaining multiple flash event sequences covering the entire video content, all flash event sequences are reconnected along the video timeline. The start and end times of each flash event sequence are mapped to the overall video timeline. The time interval between the end time of one flash event sequence and the start time of the next flash event sequence is recorded as a flash-free segment or a relatively stable lighting segment. All flash event sequences and these flash-free segments are then combined chronologically to form a flash trajectory that runs through the entire video. In the flash trajectory, the flash event sequence portion represents the time interval where the lighting changes rapidly in the video, while the flash-free portion represents the time interval where the lighting changes rapidly. The flash segment is used to represent the time interval in a video where the lighting remains relatively stable. By observing the flash trajectory as a whole, we can intuitively obtain the distribution of rapid lighting changes throughout the entire video time range, the time periods in which lighting changes occur in clusters, the duration of the lighting changes, and the order in which the lighting changes occur. This provides a clear reference for distinguishing lighting changes from emotional changes based on the flash trajectory, enabling subsequent emotion analysis and processing to effectively eliminate the influence of lighting disturbances once the lighting change trajectory is understood. The emotional change information used to control the editing rhythm can more closely reflect the real emotional changes of the characters.

[0052] The time segments with the largest brightness changes are selected from the light flash trajectory, and segments where the character's facial expressions and movements remain stable are extracted from these time segments. The selected segments are used as emotional baseline segments to form an emotional baseline, which is used to establish a benchmark reference for emotional changes.

[0053] To select time segments with the largest brightness variations based on the light flash trajectory, and to extract segments from these time segments where the character's facial expressions and movements remain stable, so that these segments can serve as emotional baseline segments to form an emotional baseline that can be used for subsequent comparison of emotional changes, the specific implementation steps are as follows:

[0054] With the flash trail already constructed, each flash event sequence within it is analyzed. The analysis begins by examining the illumination change amplitude for each flash event in the sequence. This amplitude is derived from the difference in brightness before and after the change, reflecting the actual range of light intensity change as it transitions from one brightness level to another. After extracting the amplitude values ​​for brightness changes in each sequence, all flash events within a sequence are arranged from largest to smallest, placing the event with the largest amplitude at the beginning of the sequence. Subsequently, by observing the differences in the amplitude ranking within and between flash event sequences, the time segments with the highest amplitude values ​​across the entire flash trail are selected. These time segments typically exhibit sudden brightness jumps and continuous, concentrated changes in light intensity. To make the boundaries of these time segments clearer, the start and end times, the direction of light change, the video area involved in the light change, and the continuity of brightness changes between consecutive frames for each time segment are all recorded in a uniform way, so that each time segment with the largest brightness change has complete, clear, and accurate attributes that can be accurately referenced in subsequent processing.

[0055] After identifying the time segments with the greatest brightness variations, detailed observation of the areas containing the person is necessary to find frames within these segments where the person's emotions remain stable. This involves first analyzing each time segment with the greatest brightness variation frame-by-frame in sequence, observing changes in the person's facial features such as eye shape, mouth opening and closing, eyebrow position, cheek muscle shape, and head tilt. Simultaneously, changes in the person's shoulder position, arm position, and upper body posture are also observed. By fully recording these changes in facial and body features across consecutive frames, it's possible to determine whether the person maintains emotional stability during rapid changes in lighting. When a frame or a series of frames shows facial features remaining largely unchanged from the previous frame or several frames, and no significant changes in body movement are observed, these frames are marked as stable facial expression frames. Subsequently, by combining consecutive stable expression frames in their original chronological order, a video segment is created where the person's facial expressions and movements remain largely unchanged even under rapid changes in lighting. This segment serves as the foundational material for constructing subsequent emotional baseline segments.

[0056] After obtaining segments showing stable facial expressions and movements, these selected segments need to be rearranged according to their actual order of appearance in the light trails to ensure they can be used as a basis for emotion judgment. During the rearrangement, the start frame number, end frame number, frame time position, facial features, body posture, and background content of each stable facial expression and movement segment are recorded individually, ensuring each segment possesses independently identifiable attributes. After rearrangement, common characteristics among all stable facial expression and movement segments are observed, such as a calm facial expression, stable eye opening and closing, minimal changes in mouth shape, and no body posture movements accompanying emotional changes. Segments sharing these common characteristics are combined to form emotional baseline segments. These emotional baseline segments originate from time periods of intense light fluctuations where the character's emotion did not change significantly; therefore, they accurately reflect the character's emotional state under lighting interference and are crucial for constructing the emotional baseline.

[0057] After obtaining the emotional baseline segments, the emotional expression levels of multiple emotional baseline segments need to be mapped onto the same emotional reference line to form an emotional baseline. During the mapping process, frames where the character's facial expressions and movements remain stable in each emotional baseline segment are individually mapped to the positions on the emotional reference line. These corresponding points are then connected according to the order of the emotional baseline segments on the timeline, so that the connection curve shows the character's emotional stability throughout the rapid changes in lighting. By continuously unfolding the character's emotional state across multiple lighting transitions, the emotional baseline possesses the complete characteristic of covering the entire light flash trajectory. Once the emotional baseline is constructed, it can serve as a reference curve for judging the character's true emotional changes under lighting interference, ensuring that subsequent editing rhythm processing based on the emotional curve is no longer affected by lighting changes.

[0058] Based on the emotion baseline, segment-by-segment facial expression and action comparison is performed on all time periods covered by the light flash trajectory. Time periods that are accompanied only by the light flash trajectory and do not show significant changes in facial expression and action are identified and recorded as a light disturbance list to mark the time intervals of interference caused by light.

[0059] Based on the emotional baseline, segment-by-segment facial expression and action comparison is performed across the entire time period covered by the light flash trajectory to identify time periods that only accompany the light flash trajectory without significant changes in facial expression and action. These time periods are recorded as a light perturbation list to mark the time intervals of interference caused by light. This can be carried out by following these steps, provided that the light flash trajectory and emotional baseline have already been obtained:

[0060] After constructing the flash trail, each flash event sequence recorded in the flash trail is read one by one. For each flash event sequence, the start time, end time, duration of illumination change, coverage area of ​​the illumination change in the video frame, and intensity of the illumination change are recorded and organized into multiple independent time intervals for comparison. Then, all time intervals are arranged in ascending order of start time, ensuring that each time interval has a definite start and end point on the overall video timeline. After arrangement, a continuity check is performed on adjacent time intervals. If the time interval between the end time of one time interval and the start time of the next time interval is very short, and the direction of illumination change within these two time intervals is consistent, these two time intervals can be merged into a longer comparison time interval, thus reducing the analysis fragmentation caused by time interval fragmentation. In this way, a set of comparison time intervals covering all illumination change locations can be obtained based on the flash trail, with each time interval corresponding to a portion of the video content that may be affected by illumination disturbances. At the same time, these time intervals to be compared are time-aligned with the emotional baseline, and the start and end times of each time interval are marked on the emotional baseline to prepare for subsequent comparison of facial expressions and actions within the same time reference frame.

[0061] After obtaining the set of time intervals to be compared, for each time interval, facial expressions and motion information are extracted frame by frame according to the natural playback order of the video frames. Specifically, in each frame, the area where the person is located is located, and the facial features are observed, focusing on the opening and closing of the eyes, changes in eyelid position, changes in eyebrow shape, changes in the contour curve of the mouth, changes in the upward or downward position of the corners of the mouth, changes in the tension and relaxation of the cheek contour, and changes in the angle of head rotation. At the same time, the body features are observed, including the undulation of the shoulders, changes in the posture of the upper body leaning forward or backward, changes in the posture of raising or lowering the arms, and changes in the position of the hands. The state of these facial and motion features in each frame is recorded as the facial and motion description information for that frame. Subsequently, at the same time point, the corresponding emotional reference state in the emotional baseline is extracted. The emotional level represented by the emotional baseline at that time point is compared with the facial expression and action description information recorded in the current frame. If the character's eyes are stable, the mouth contour curve remains basically unchanged, the head posture does not shift drastically, and the body posture continues the state of the previous frame, and the emotional baseline also shows a stable emotional trend at the same time point, then it can be determined that the character's emotion in that frame is consistent with the emotional baseline. Based on this method, within a comparison time interval, multiple consecutive frames can be compared frame by frame, and those frames that are consistent with the emotional baseline in terms of facial expressions and actions are marked. Through this segment-by-segment comparison process, a one-to-one correspondence is established between the character's facial expressions and action states in each comparison time interval and the emotional baseline, providing sufficient basis for subsequent identification of interference time periods caused only by changes in lighting.

[0062] After comparing each frame within the entire comparison time interval with the emotional baseline, the comparison results can be centrally organized. For each comparison time interval, a sequence of consecutive frames whose facial expressions and movements are highly consistent with the emotional baseline is identified. If, within a certain consecutive frame sequence, the degree of eye opening and closing remains approximately within a certain range, the mouth contour does not exhibit strong indications of emotional excitement or depression, the head orientation in space changes very little compared to the previous time period, and the shoulder and upper body posture remains stable in a standing, sitting, or walking position, and this consecutive frame sequence falls within a rapidly changing illumination range in the light flash trajectory, and the light flash trajectory shows significant fluctuations in illumination intensity within a short period, then the time period corresponding to this consecutive frame sequence can be identified as an interference period that only appears with the light flash trajectory without any actual emotional change. For each identified interference period, the start time, end time, duration, corresponding light flash event sequence number, intensity characteristics of illumination changes, specific manifestations of stable facial expressions, specific manifestations of stable body movements, and the emotional baseline level within that time period need to be recorded in detail. In this way, a set of multiple interference time periods can be obtained. Each time period in this set satisfies two conditions: on the one hand, it is within the range of rapid changes in illumination represented by the light flash trajectory; on the other hand, the characters' expressions and movements remain stable after being compared with the emotional baseline. Thus, it can be confirmed that the changes in the picture during these time periods mainly come from lighting disturbances, rather than changes in the characters' emotions.

[0063] After obtaining all the interfering time periods in the entire video that meet the above conditions, these time periods need to be uniformly registered as a lighting disturbance list. Specifically, each interfering time period is assigned a unique identifier. The start time, end time, duration, corresponding flash event sequence information, amplitude of lighting changes, spatial distribution area of ​​lighting changes in the frame, description of stable facial expressions, description of stable body movements, and emotional baseline at the corresponding time period are recorded as attribute information for that interfering time period. This information is stored in the lighting disturbance list in a tabular or structured format. Subsequently, all entries in the lighting disturbance list are sorted according to the start time of the interfering time periods, so that the sorted lighting disturbance list can intuitively reflect the distribution of lighting interference along the entire video playback timeline. In this way, the lighting disturbance list becomes a set of markers covering multiple time intervals. When adjusting the amplitude of the mood curve in the later stage, as long as the time position falls into any disturbance time period in the lighting disturbance list, the corresponding mood curve can be regarded as the interval affected by the lighting disturbance. Thus, in the subsequent processing, special mood curve compression and mood curve smoothing strategies can be adopted for these intervals, so that the final mood change trend used to control the editing rhythm is closer to the real mood change of the character, rather than the visual disturbance caused by the rapid change of lighting.

[0064] Within the time interval corresponding to the light disturbance list, the original emotion curve is subjected to amplitude compression processing, compressing the fluctuation amplitude of the emotion curve in the light disturbance interval to a change level consistent with the emotion baseline, so as to generate a smooth emotion band compatible with light disturbance.

[0065] To perform amplitude compression processing on the original mood curve within the time interval corresponding to the lighting perturbation list, ensuring that the fluctuation amplitude of the mood curve within the lighting perturbation interval maintains a consistent level of change with the mood baseline, and on this basis, to form a smooth mood band that covers the entire video content, the following steps can be taken based on the existing original mood curve, mood baseline, and lighting perturbation list.

[0066] The complete original mood curve is obtained, and the start and end times of each illumination perturbation time interval in the illumination perturbation list are accurately marked on the original mood curve, giving each illumination perturbation time interval a clear time range. After completing the time range marking, a front buffer time and a back buffer time are set for each illumination perturbation time interval, so that the leading and trailing edges of the illumination perturbation interval on the time axis have transition regions. This ensures that the mood changes can be naturally connected during the subsequent mood curve compression process, without producing discontinuous changes due to directly entering or leaving the compression region. In this way, a set of mood curve intervals to be processed with clear time positions, complete boundaries, and transition regions can be formed, so that all time periods in the original mood curve that may be affected by illumination perturbation can be accurately located.

[0067] After calibrating the time range of each emotional curve interval to be processed, to provide a reference basis for compression processing, it is necessary to extract the corresponding emotional baseline value on the time axis for each emotional curve interval to be processed from the emotional baseline. Specifically, the start and end times of each emotional curve interval to be processed are mapped onto the emotional baseline curve, thereby obtaining the emotional baseline values ​​at all time points within that interval. After obtaining the emotional baseline values, by observing the trend of the emotional baseline within the interval, its overall rate of change, direction of change, and magnitude of change can be extracted, and an emotional value limit range can be established for that interval based on these emotional baseline characteristics. This limit range consists of an upper limit emotional value and a lower limit emotional value, both of which revolve around the trend of the emotional baseline within the time interval, so that the compression processing of the original emotional curve can revolve around the emotional baseline, thereby ensuring that the overall change level of the compressed emotional curve is consistent with the emotional baseline. In addition, if the emotional baseline has a continuous trend of change before and after the light disturbance time interval, when determining the limit range, the boundary of the limit range needs to be appropriately raised or lowered at the end of the time interval according to the trend of the emotional baseline, so that the compressed emotional curve can naturally connect with the emotional changes outside the compressed interval.

[0068] After clearly defining the baseline limits for each interval of the emotion curve to be processed, the original emotion curve needs to be divided into several sub-segments within that interval according to its natural variation. Each sub-segment can correspond to an upward trend, a period of stagnation, a slow decline, or a combination of multiple subtle fluctuations in the original emotion curve. After division, the emotion value is read at each time point within each sub-segment, and this emotion value is compared with the emotion value limits determined in the previous step. When the original emotion value at a certain time point is higher than the upper limit emotion value, the emotion value is adjusted to fall within the range between the upper and lower limits; when the original emotion value at a certain time point is lower than the lower limit emotion value, the emotion value is adjusted to fall within the limit range; when the emotion value at a certain time point is already within the limit range, the emotion value can be slightly processed based on the subtle changes in the emotion baseline at that time point, so that the compressed emotion value better matches the overall trend of the emotion baseline. After adjusting the emotional values ​​at each time point, in order to maintain the visual continuity of the compressed emotional curve, it is necessary to construct a continuous transition between adjacent time points so that the compressed emotional values ​​do not jump or break, thereby maintaining the smoothness of the overall curve change.

[0069] After compressing the original emotion curve for each illumination disturbance time interval and obtaining multiple locally smoothed emotion curve segments, these locally smoothed emotion curve segments need to be placed back into their corresponding positions on the time axis of the original emotion curve, replacing the original emotion curve segments. After the local replacement, the entire emotion curve will consist of two types of parts: one is the locally smoothed emotion curve segments that have been compressed and are limited to the range of emotion baseline changes; the other is the emotion curve segments for non-illumination disturbance time intervals that retain the original change pattern. To ensure that the entire emotion curve has continuous change characteristics, a transition region needs to be set at the connection point between the locally smoothed emotion curve segments and the original emotion curve segments. This allows the curve to gradually transition from uncompressed to compressed change, or from compressed change to uncompressed change, at multiple time points at the connection point, thereby ensuring that the entire emotion curve remains smooth, consistent, and without abrupt changes in visual presentation. The emotion curve obtained through the above steps constitutes a smooth emotion band covering the entire video content. This smooth emotion band can reflect the real emotional changes of the characters under lighting disturbances, providing an accurate emotional basis for subsequent editing rhythm control, and enabling the editing rhythm to be dynamically adjusted based on the accurate emotional trends of the characters even in scenes with rapid changes in lighting.

[0070] A self-regulating editing rhythm control strategy is added to the smooth mood band, so that the editing rhythm is adjusted according to the changes in the character's facial expressions and actions during the operation, and automatically avoids the light flash trajectories corresponding to the lighting disturbance list during the rhythm control process, thereby constructing a dynamic avoidance editing process for scenes with rapid changes in lighting.

[0071] To incorporate a self-adjusting editing rhythm control strategy into the smooth mood band, prioritizing adjustments based on changes in character facial expressions and movements during editing, and automatically avoiding light flicker trajectories corresponding to the lighting disturbance list during rhythm control, thus constructing a dynamic avoidance editing process for scenes with rapid lighting changes, the following steps can be taken, assuming the smooth mood band, lighting disturbance list, and light flicker trajectories are already obtained:

[0072] After obtaining the smooth emotional band, a full scan of the smooth emotional band is performed to determine the emotional rhythm reference framework upon which the editing rhythm control is based. Specifically, the smooth emotional band is divided into several continuous time intervals along the video playback timeline from start to end. Each time interval can correspond to a relatively independent emotional phase, such as a gradually rising emotional phase, a stable emotional phase, a gradually falling emotional phase, and a slowly fluctuating emotional phase. During the division process, the emotional intensity level, direction of emotional change, speed of emotional change, and morphological characteristics of emotional fluctuations within each time interval are recorded. Subsequently, editing rhythm reference parameters are established for each time interval. The gradually rising emotional interval is marked as a rhythmic area where the frequency of shot changes needs to be gradually increased; the stable emotional interval is marked as a rhythmic area where the shot rhythm remains uniform; the gradually falling emotional interval is marked as a rhythmic area where shot changes gradually slow down; and the slowly fluctuating emotional interval is marked as a rhythmic area where slight rhythmic adjustments are allowed while maintaining an overall smooth transition. This process creates a rhythmic reference band that corresponds one-to-one with the smooth emotional band. This rhythmic reference band provides a clear target range for the self-regulating behavior of subsequent editing rhythm control strategies, ensuring that the overall editing rhythm remains synchronized with the main emotional thread of the character.

[0073] Based on the established rhythmic reference bands, specific and executable self-adjusting rules for editing rhythm are established for different emotional stages, allowing the editing rhythm to naturally tighten or loosen according to changes in the smooth emotional band. Specifically, in the time interval where the smooth emotional band shows a continuous upward trend, the self-adjusting rules for this time interval may include: appropriately shortening the shot dwell time, increasing the number of shot transitions, prioritizing the use of dynamic shot combinations, and increasing the perceived intensity of the rhythm in the shots. In the time interval where the smooth emotional band shows a relatively stable trend, the self-adjusting rules for this time interval may include: maintaining a relatively fixed shot dwell time, controlling the number of shot transitions to a moderate level, prioritizing structurally complete shot content, and emphasizing the coherent display of characters' expressions and actions. In the time interval where the smooth emotional band shows a downward trend, the self-adjusting rules for this time interval may include: extending the shot dwell time, reducing the number of shot transitions, emphasizing environmental atmosphere shots, and weakening the rhythmic impact. In the time interval where the smooth emotional band is in a slow fluctuating state, the self-adjusting rules for this time interval may include: while maintaining an overall stable rhythm, allowing slight adjustments to the frequency of local shot transitions with minor emotional fluctuations to ensure a natural transition. By defining self-adjusting rules for editing rhythm for each emotional stage, the editing rhythm control can automatically select the corresponding rhythm behavior according to the different forms of the smooth emotional band in the subsequent execution stage, thereby realizing the active following of the editing rhythm to the changes in the character's emotions.

[0074] After the rhythm self-adjustment rules are determined, the lighting disturbance list and flash trails are associated with the aforementioned rhythm reference band and rhythm self-adjustment rules. This allows the editing rhythm to actively avoid interference time intervals caused by lighting jumps during execution. Specifically, each record in the lighting disturbance list is first read, and its start time, end time, corresponding flash trail segment position, description of lighting change intensity, and description of the character's stable emotion within that time are all converted into marked intervals on the video timeline. A smooth emotion band and rhythm reference band are then overlaid on the same timeline. Subsequently, when preparing to generate an editing rhythm arrangement for a specific time interval, the appropriate rhythm self-adjustment rule is first determined based on the time interval's position on the smooth emotion band. Then, it is checked whether this time interval overlaps with interference time intervals recorded in the lighting disturbance list. If a candidate time interval for a rhythm change is found to be within or partially overlaps with an interference time interval, the rhythm avoidance logic is triggered. The shot transition time is adjusted forward to a safe time interval before the start of the interference time interval, or backward to a safe time interval after the end of the interference time interval. Simultaneously, the time positions of adjacent rhythm change points are corrected to maintain the overall rhythm structure's coherence. In this way, during the generation of editing rhythm, any key rhythm changes no longer fall directly on the high-intensity lighting jump positions corresponding to the flash trajectory, but naturally avoid these positions, so that lighting changes no longer become the dominant factor in rhythm changes, but only exist as background changes in the picture. The editing rhythm always takes the changes in the characters' emotions reflected by the smooth emotional band as the core basis.

[0075] After comprehensively introducing the rhythm reference band, self-adjustment rules, and lighting avoidance mechanism, the editing rhythm control results across the entire video timeline are organized. All rhythm change points processed by self-adjustment and avoidance are uniformly arranged to construct a dynamic avoidance editing process that runs throughout the entire film. In this sub-step, the adjusted rhythm change points are sorted chronologically, and a corresponding editing operation type is assigned to each rhythm change point, such as cutting to a close-up of a character, switching to a distant view of the environment, changing the perspective within the same scene, or connecting different scenes. Subsequently, these specific editing operations are mapped one by one to the emotional states on the smooth emotion band, ensuring that each editing action reflects the emotional intent at the corresponding moment. At the same time, the relationship between the time interval involved in each editing operation and the lighting disturbance list is checked again to confirm that all editing actions have successfully avoided the lighting interference time interval marked by the light flash trajectory, ensuring that the final editing rhythm conforms to the emotional direction throughout the entire film while avoiding the visual impact caused by lighting jumps that deviate from the character's true emotions. The dynamic avoidance editing process formed by the above steps ensures that the final video maintains a continuous and natural emotional expression path even in scenes with rapidly changing lighting. The editing rhythm closely revolves around the changes in the characters' facial expressions and actions, thereby improving the overall viewing experience and the accuracy of emotional expression in complex lighting environments.

[0076] This invention constructs light flicker trajectories, establishes an emotional baseline, and compresses the amplitude of lighting disturbances, making the emotional curve more closely resemble the true emotional trajectory of a character. When rapid changes in lighting cause drastic fluctuations in screen brightness, the emotional curve no longer exhibits irrelevant fluctuations with these brightness changes, but instead maintains a stable and continuous change pattern. The resulting emotional expression accurately reflects the character's emotional changes throughout the video content, providing a reliable foundation for subsequent editing rhythm control and making the final film more consistent with the logic of real emotions in its narrative expression. Through this method, false emotional fluctuations caused by lighting interference are effectively weakened, ensuring the consistency of the final film's visual emotional presentation.

[0077] This invention introduces a self-adjusting editing rhythm control strategy within a smooth emotional band, allowing the editing rhythm to naturally adjust with changes in the character's emotions. During editing, it automatically avoids lighting disturbances and flickering light trajectories, achieving synchronization between rhythm changes and the character's emotions. In scenes with strong, jittery lighting, the editing rhythm is not disrupted by brightness jumps, but maintains a continuous, stable rhythmic trajectory that aligns with narrative needs. The resulting video exhibits improved rhythm presentation, content expression, and visual stability, avoiding abrupt shot transitions or emotional rhythm shifts caused by lighting fluctuations, resulting in a final product with higher expressive credibility and visual comfort.

[0078] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A video editing and processing method based on artificial intelligence, characterized in that, Includes the following steps: For scenes with rapid changes in lighting, a light flicker feature description rule is established. Each brightness change that occurs in a video frame is converted into a light flicker event, and consecutive light flicker events are formed into a light flicker event sequence to construct a light flicker trajectory. The time segments with the largest brightness changes are selected from the light flash trajectory, and segments where the character's facial expressions and movements remain stable are extracted from these time segments. The selected segments are used as emotional benchmark segments to form an emotional baseline. The steps to select the time segments with the largest brightness changes along the light flash trajectory and extract segments where the human facial expressions and movements remain stable to form an emotional baseline are as follows: The sequence of light flash events in the light flash trajectory is analyzed one by one. The difference in brightness before and after the change of brightness for each light flash event is extracted and sorted according to the magnitude of brightness change to obtain the time segment with the largest brightness change. The start time, end time, direction of light change, and area of ​​light change of the time segment are recorded. Within the time segment with the greatest brightness variation, observe frame by frame the changes in the shape of the eyes, the opening and closing of the mouth, the position of the eyebrows, the shape of the cheek muscles, the tilt of the head, and the changes in body posture. Identify frames where facial expressions and movements remain stable and combine them into stable facial expression and movement segments. All stable facial expression segments are arranged according to their order of appearance in the light flash trajectory, and combined to form emotional baseline segments based on the common characteristics of maintaining calm facial expressions and movements. Map the emotional expressions corresponding to the emotional baseline segments onto a unified emotional reference line and connect them to form an emotional baseline. Based on the emotion baseline, segment-by-segment facial expression and action comparisons were performed on all time periods covered by the light flash trajectory to identify time periods that were only accompanied by the light flash trajectory and did not show significant changes in facial expression and action, and these time periods were recorded as a list of illumination perturbations. The steps for performing segment-by-segment facial expression and action comparison and generating a list of illumination perturbations based on the emotion baseline across the entire time period covered by the light flash trajectory are as follows: Based on the light flash trajectory, the start time, end time, duration of light change, area covered by light change, and intensity of light change of each light flash event sequence are read and organized into time intervals to be compared; Within each comparison time interval, the range of eye opening and closing, mouth contour changes, eyebrow shape changes, head rotation angle and body posture changes of the person are extracted frame by frame, and compared with the emotional level of the emotional baseline to identify frames where facial expressions and movements remain stable. Within each comparison time interval, find a continuous frame sequence in which facial expressions and emotions remain consistent with the baseline, and record the start time, end time, duration, intensity of light changes, and the stable expression and body posture of the person in the sequence to form interference time intervals. Record the start and end times, duration, sequence number of light flash events, amplitude of light change, spatial area of ​​light change, and baseline level of mood for all interference periods as a light disturbance list and arrange them in chronological order. Within the time interval corresponding to the light disturbance list, the original emotion curve is subjected to amplitude compression processing, compressing the fluctuation amplitude of the emotion curve in the light disturbance interval to a change level consistent with the emotion baseline, so as to generate a smooth emotion band compatible with light disturbance. By incorporating a self-regulating editing rhythm control strategy into the smooth mood band, the editing rhythm is adjusted according to changes in the characters' facial expressions and actions during the process. It also automatically avoids the light flash trajectories corresponding to the lighting disturbance list during the rhythm control process, thereby constructing a dynamic avoidance editing process for scenes with rapid changes in lighting.

2. The video editing processing method based on artificial intelligence according to claim 1, characterized in that, The process of constructing the light flash trajectory is as follows: The video to be processed is read frame by frame in chronological order, and the average brightness value, the local brightness value of multiple divided regions, and the pixel number distribution of brightness at different gray levels are extracted from each frame. The brightness information of the current frame is compared with the brightness information of the previous frame item by item to identify brightness jumps. Each brightness jump is converted into a flash event, and the numerical difference before and after the brightness change, the direction of the brightness change, the range of the brightness change area, the duration of the brightness change in frames, and the video content scene type before and after the brightness change are associated with the same flash event. All flash events are constructed into a flash event time chain in chronological order, and adjacent flash events are combined into a flash event sequence based on differences in light intensity, consistency of light direction, and continuity of video content. All light flash event sequences are connected along the time axis to form a light flash trajectory, and the distribution location of light jumps, the duration of light jumps, and the order of light jumps are obtained based on the light flash trajectory.

3. The video editing processing method based on artificial intelligence according to claim 1, characterized in that, In the completed stable facial expression segments, only segments with constant eye opening and closing, smooth mouth curves, no sudden changes in eyebrow position, continuous and consistent head posture, and stable background content are included in the emotional baseline segments. This ensures that the emotional baseline segments maintain a consistent emotional expression throughout the lighting transitions, thereby improving the stability and reference accuracy of the emotional baseline.

4. The video editing processing method based on artificial intelligence according to claim 1, characterized in that, In the process of creating the lighting disturbance list, the characteristics of lighting intensity changes during each disturbance period and the stable performance of the character's facial expressions and movements are used as recording conditions. After recording, all disturbance periods are rearranged according to their chronological order, so that the lighting disturbance list can continuously cover all lighting change positions and provide an accurate reference for subsequent emotional curve compression processing.

5. The video editing processing method based on artificial intelligence according to claim 1, characterized in that, The steps for performing amplitude compression processing on the original sentiment curve and forming a smooth sentiment band within the time interval corresponding to the illumination perturbation list are as follows: Obtain the original mood curve and mark the start and end times in the light disturbance list as the intervals of the mood curve to be processed, and set the front buffer time and back buffer time for each interval. Within each interval of the emotion curve to be processed, the emotion baseline value at the corresponding time point is read, and the upper and lower limit emotion values ​​are set as the compression range based on the changing trend of the emotion baseline in the interval. Each interval of the emotion curve to be processed is divided into sub-segments according to the natural change pattern of the original emotion curve, and the original emotion value is adjusted to fall within the compression range at each time point, and a continuous transition is established between adjacent time points. Each compressed, locally smoothed emotion curve segment replaces the corresponding interval of the original emotion curve, and transition processing is performed at the connection points to form a smooth emotion band covering the entire video.

6. The video editing processing method based on artificial intelligence according to claim 1, characterized in that, The steps to incorporate self-regulating editing rhythm control strategies into smooth mood bands are as follows: Based on the emotional intensity level, direction of emotional change, speed of emotional change, and pattern of emotional fluctuations of the smooth emotional band on the time axis, the smooth emotional band is divided into multiple emotional stages and forms a rhythm reference band. Based on the rhythm reference, self-adjusting rules for editing rhythm are set for each emotional stage, and the shot dwell time, number of shot cuts and image presentation are set according to the emotional rise, emotional stability, emotional fall and slow emotional fluctuation. The list of light disturbances and the light flash trajectory are superimposed on the rhythm reference band, and when generating rhythm change points, the time interval of the light disturbances is compared, and the rhythm change points that fall into the disturbance time interval are adjusted to outside the disturbance time interval. All rhythmic change points are arranged in chronological order, and editing operations are assigned to each rhythmic change point to ensure that the editing rhythm is consistent with the emotional state corresponding to the smooth mood band and avoids the time interval of lighting disturbance.

Images