Short video auxiliary shooting scoring method based on AI multi-mode model

Abstract

The invention discloses a short video auxiliary shooting scoring method based on an AI multi-mode model. The short video auxiliary shooting scoring method comprises the following steps: S10, video acquisition; s20, video analysis: performing multi-dimensional fusion analysis on the standardized video data by adopting a vision-audio-semantic multi-modal fusion analysis algorithm to obtain a multi-dimensional fusion analysis result; s30, score generation: according to a multi-dimensional fusion analysis result, calculating by adopting a dynamic weight adaptive algorithm to obtain a comprehensive score of the to-be-analyzed short video; s40, bright spot identification: extracting index data meeting a quantitative excellent standard from a multi-dimensional fusion analysis result, and identifying and summarizing shooting bright spots of the short video by combining content characteristics of the short video; s50, defect identification is carried out; s60, optimization suggestion generation: on the basis of the identified shooting defects, a targeted overall optimization suggestion is generated in combination with the reservation demand of the shooting bright spots; and S70, outputting a result.

Description

Technical Field

[0001] This invention belongs to the field of video content understanding and intelligent recommendation technology, specifically relating to a short video assisted shooting and scoring method based on an AI multimodal model. Background Technology

[0002] With the rapid development of mobile internet technology and the widespread adoption of smart devices, short videos, with their brevity, vivid content, and ease of dissemination, have become a mainstream form of information dissemination and content creation. More and more ordinary users are participating in short video creation, using them to record their lives, share their opinions, or achieve commercial monetization. However, for the vast majority of non-professional creators, due to a lack of professional shooting knowledge and composition skills, their short videos often suffer from poor image quality, disorganized content, and insufficient appeal, making it difficult to achieve the desired dissemination effect.

[0003] In the existing technology, some short video shooting tools only provide basic shooting functions and simple post-editing templates, which cannot provide a comprehensive and professional evaluation of the short videos shot by users; a few tools with evaluation functions have a single evaluation dimension, usually only focusing on basic indicators such as image clarity, which cannot accurately identify the shooting highlights and shortcomings, let alone provide targeted optimization suggestions. Summary of the Invention

[0004] In view of the above-mentioned problems, this invention provides a short video assisted shooting and scoring method based on an AI multimodal model. It achieves in-depth analysis of short videos through multimodal fusion analysis algorithm, quantifies the criteria for judging highlights and shortcomings, generates a comprehensive score using a dynamic weight adaptive algorithm, and provides targeted overall optimization suggestions based on a clear technical path, helping users to clearly understand the strengths and weaknesses of their creations and continuously improve the quality of their short video creations.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A short video assisted shooting and scoring method based on an AI multimodal model, characterized by the following steps: S10, Video Acquisition: Receive short videos uploaded by users to be analyzed, perform format parsing and preprocessing on the short videos, and obtain standardized video data; S20, Video Analysis: A visual-audio-semantic multimodal fusion analysis algorithm is used to perform multi-dimensional fusion analysis on standardized video data to obtain multi-dimensional fusion analysis results; S30, Score Generation: Based on the results of multi-dimensional fusion analysis, a dynamic weight adaptive algorithm is used to calculate the comprehensive score of the short video to be analyzed; S40, Highlight Identification: Extract indicator data that meets the quantitative excellence standard from the multi-dimensional fusion analysis results, and combine it with the content characteristics of the short video to identify and summarize the shooting highlights of the short video; S50, Defect Identification: Extract indicator data that are below the quantitative qualification standard from the multi-dimensional fusion analysis results of the video analysis module, and combine them with the creation logic of the short video to identify and summarize the shooting defects of the short video. S60, Optimization suggestion generation: Based on the identified shooting shortcomings and the need to retain shooting highlights, targeted overall optimization suggestions are generated; S70 output: The overall score, shooting highlights, shooting shortcomings, and overall optimization suggestions are presented to the user in a visual format.

[0006] One possible implementation further includes: S80, Historical Record Storage: Utilizes cloud storage to store user-uploaded short video data, multi-dimensional analysis results, comprehensive scores, optimization suggestions, and user modification behavior data; allows users to view past creative analysis through the historical record interface, and the system automatically generates charts showing trends in creative quality changes.

[0007] In one possible implementation, S10 specifically includes: receiving the short video to be analyzed uploaded by the user through a mobile APP interface or a web page upload interface, extracting the video stream and audio stream, performing size standardization and frame rate unification processing on the video stream, performing sampling rate standardization processing on the audio stream, and finally outputting standardized video data.

[0008] In one possible implementation, S20 specifically includes: It includes a built-in image feature extraction model based on Convolutional Neural Network (CNN), an audio feature extraction model based on Long Short-Term Memory Network (LSTM), and a semantic feature extraction model based on BERT. The image feature extraction model extracts image features including edge contours, color distribution, and pixel density of video frames. The audio feature extraction model extracts human voice and ambient sound features including volume changes, frequency distribution, and noise characteristics. The semantic feature extraction model performs text encoding on video titles, subtitles, and image semantic tags to obtain semantic features. A three-dimensional feature vector is constructed, and an attention mechanism is introduced to assign weights and fuse three types of features: image, audio, and semantics, thereby enhancing the feature expression of key information. The analysis dimensions include image quality, composition, content completeness, rhythm, and audio quality.

[0009] In one possible implementation, the image quality dimension analysis is as follows: using an edge detection algorithm to calculate the gradient value standard sharpness, using the statistical mean of pixel grayscale values ​​to represent brightness, using the ratio of the range to the mean of pixel grayscale values ​​to represent contrast, and using the mean color of the S channel in the HSV color space to represent saturation.

[0010] In one possible implementation, the composition dimension analysis is as follows: the pixel distance between the center of the subject and the golden ratio point is used to represent the position of the subject; the pixel distribution difference rate of the left and right / up and down areas of the image is used to represent the balance of the image; and the proportion of the blank area to the total area of ​​the image is used to represent the rationality of the blank space.

[0011] In one possible implementation, S30 specifically includes: S301, preset basic weight coefficients: image quality dimension 0.3, composition dimension 0.2, content integrity dimension 0.2, rhythm dimension 0.1, audio quality dimension 0.2; S302, Content Type Recognition: Automatically identify the content type of short videos by fusing semantic features and visual features; S303, Dynamic Weight Adjustment: Adaptively adjusts the weights of each dimension based on the identified content type; S304, User Preference Learning: Based on user modification behavior data in the historical record storage module, the weight allocation is optimized through the gradient descent algorithm; S305, Comprehensive score calculation: The comprehensive score is calculated using a weighted summation formula. The comprehensive score = picture quality score × dynamic weight 1 + composition score × dynamic weight 2 + content integrity score × dynamic weight 3 + rhythm score × dynamic weight 4 + audio quality score × dynamic weight 5. Each dimension score is a quantitative value of 0-100 points.

[0012] In one possible implementation, S40 specifically includes: Image quality rating: Sharpness gradient value ≥250, corresponding score ≥95; average brightness in the range of [120, 180], corresponding score ≥90; contrast ratio ≥1.8, corresponding score ≥90; color saturation in the range of [55, 75], corresponding score ≥90. Composition dimension score: The distance between the main subject position and the golden ratio point is ≤30px, the difference rate of the image balance is ≤10%, and the reasonableness of the white space ratio is in the range of [20%, 35%]; The scoring criteria for completeness of content are as follows: Topic clarity and semantic similarity ≥90%, corresponding to a score ≥90 points; Content coherence and relevance ≥85%, corresponding to a score ≥85 points. Rhythm dimension score: The frequency of camera transitions is adapted to the content type; Audio quality rating criteria: standard deviation of volume stability ≤ 5dB, noise interference percentage ≤ 5%, and signal-to-noise ratio of human voice clarity ≥ 35dB; Highlights Summary: Extract indicator data that meets any two or more of the above criteria, and generate structured commentary text based on content characteristics.

[0013] In one possible implementation, S50 specifically includes: Image quality rating: Sharpness gradient value <120, corresponding score <60 points; average brightness <80 or >220, corresponding score <60 points; contrast ratio <0.8, corresponding score <60 points; color saturation <30 or >90, corresponding score <60 points. Composition dimension rating: The distance between the subject position and the golden ratio point is greater than 100px, the difference rate of the image balance is greater than 30%, and the reasonableness of the white space ratio is less than 10% or greater than 50%; Content completeness score: Theme clarity and semantic similarity <60%, corresponding score <65 points; Content coherence and relevance <50%, corresponding score <60 points; Pace-based scoring: Camera switching frequency exceeds the adaptation range by 2 times or more; core content duration accounts for less than 40% of the total time. Audio quality rating criteria: volume stability standard deviation > 15dB, noise interference percentage > 20%, and voice clarity signal-to-noise ratio < 15dB. Summary of shortcomings: Extract indicator data that are below the above standards, distinguish between objective quality defects and room for improvement, and generate accurate review copy.

[0014] In one possible implementation, S60 specifically includes: Constructing a Defect-Solution Knowledge Graph: The knowledge graph consists of three layers: the defect type layer, the solution layer, and the operation guidance layer. Integrating large language model to generate suggestions: A lightweight large language model is fine-tuned, taking quantitative data of shortcomings indicators, data of highlights indicators, and content type tags as input, and combining them with the related data in the knowledge graph to generate structured optimization suggestions; To optimize image quality, specific numerical adjustment suggestions are provided based on the parameters output by the image feature extraction model. Optimize the composition: Combine the subject position data identified by the target detection algorithm to generate coordinate adjustment suggestions; Optimize content completeness: Based on the semantic similarity analysis results, it is recommended to add or delete relevant content; Optimize the pacing: Provide duration adjustment suggestions based on the frequency of camera transitions and the suitability of the content type; Audio optimization: Based on spectrum analysis data, suggestions for noise reduction and volume adjustment are provided; Highlight retention suggestions: Generate suggestions for enhancing and extending the identified highlights in the shot.

[0015] The present invention has the following beneficial effects: (1) Multimodal fusion for accurate analysis: The visual-audio-semantic multimodal fusion algorithm is adopted, and the key features are enhanced by the attention mechanism. Compared with single-dimensional or single-modal analysis, the comprehensiveness and accuracy of video evaluation are greatly improved, providing a more reliable basis for scoring and optimization suggestions.

[0016] (2) Quantitative standards ensure practicality: By using clear numerical thresholds and calculation methods, the highlights and shortcomings are quantitatively defined, which solves the problem of subjective judgment in traditional evaluation, ensures the universality and operability of the evaluation standards, and meets the unified reference needs of different users.

[0017] (3) Strong adaptability of dynamic scoring: The dynamic weight adaptive algorithm can automatically adjust the scoring weight according to the short video content type and user preferences. Compared with fixed weight or manual adjustment, it is more in line with diverse creative scenarios, and the scoring results are more personalized and have more reference value.

[0018] (4) The optimization suggestions are highly implementable: By using the technical approach of knowledge graph + large language model, and combining specific quantitative data, targeted optimization suggestions with clear operation steps are generated. This not only informs users where changes are needed, but also clarifies how to make changes, greatly reducing the optimization threshold for ordinary users.

[0019] (5) Excellent ease of use and scalability: The standardized video preprocessing process and the visual output format reduce the difficulty of use; it supports user preference learning and historical data tracing, which can adapt to the creative needs of different users and the changes in industry technology development, and has a wide range of application scenarios. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of a short video assisted shooting and scoring method based on an AI multimodal model, according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Referring to Figure 1, a flowchart of a short video assisted shooting and scoring method based on an AI multimodal model according to an embodiment of the present invention is shown, including the following steps: S10, Video Acquisition: Receive short videos uploaded by users to be analyzed, perform format parsing and preprocessing on the short videos, and obtain standardized video data; S20, Video Analysis: A visual-audio-semantic multimodal fusion analysis algorithm is used to perform multi-dimensional fusion analysis on standardized video data to obtain multi-dimensional fusion analysis results; S30, Score Generation: Based on the results of multi-dimensional fusion analysis, a dynamic weight adaptive algorithm is used to calculate the comprehensive score of the short video to be analyzed; S40, Highlight Identification: Extract indicator data that meets the quantitative excellence standard from the multi-dimensional fusion analysis results, and combine it with the content characteristics of the short video to identify and summarize the shooting highlights of the short video; S50, Defect Identification: Extract indicator data that are below the quantitative qualification standard from the multi-dimensional fusion analysis results of the video analysis module, and combine them with the creation logic of the short video to identify and summarize the shooting defects of the short video. S60, Optimization suggestion generation: Based on the identified shooting shortcomings and the need to retain shooting highlights, targeted overall optimization suggestions are generated; S70 output: The overall score, shooting highlights, shooting shortcomings, and overall optimization suggestions are presented to the user in a visual format.

[0023] Another embodiment of the present invention provides a short video assisted shooting and scoring method based on an AI multimodal model, further referring to Figure 1, which may further include, based on S10 to S70: S80, Historical Record Storage: Utilizes cloud storage to store user-uploaded short video data, multi-dimensional analysis results, comprehensive scores, optimization suggestions, and user modification behavior data; allows users to view past creative analysis through the historical record interface, and the system automatically generates charts showing trends in creative quality changes.

[0024] In one application example, S10 specifically includes: receiving short videos uploaded by users for analysis via a mobile APP interface or a web upload interface, which can be in common formats such as MP4 and MOV; extracting video and audio streams; standardizing the size of the video stream (e.g., adjusting it to 1080P resolution) and the frame rate (e.g., adjusting it to 30fps); standardizing the sampling rate of the audio stream (e.g., adjusting it to 44.1kHz); and finally outputting standardized video data.

[0025] In one application example, S20 specifically includes: It includes a built-in image feature extraction model based on Convolutional Neural Network (CNN), an audio feature extraction model based on Long Short-Term Memory Network (LSTM), and a semantic feature extraction model based on BERT. The image feature extraction model extracts image features including edge contours, color distribution, and pixel density of video frames. The audio feature extraction model extracts human voice and ambient sound features including volume changes, frequency distribution, and noise characteristics. The semantic feature extraction model performs text encoding on video titles, subtitles, and image semantic tags to obtain semantic features. A three-dimensional feature vector is constructed, and an attention mechanism is introduced to assign weights and fuse three types of features: image, audio, and semantics, thereby enhancing the feature expression of key information. The analysis dimensions include image quality, composition, content completeness, rhythm, and audio quality.

[0026] Furthermore, the image quality dimensions are analyzed as follows: the gradient value is calculated using an edge detection algorithm to represent standard sharpness; the average value of pixel grayscale values ​​is used to represent brightness; the ratio of the range to the mean of pixel grayscale values ​​is used to represent contrast; and the average value of the S channel in the HSV color space is used to represent saturation.

[0027] The composition dimensions are analyzed as follows: the pixel distance between the center of the subject and the golden ratio point is used to represent the position of the subject; the pixel distribution difference rate of the left and right / up and down areas of the image is used to represent the balance of the image; and the proportion of the blank area to the total area of ​​the image is used to represent the rationality of the blank space.

[0028] The content integrity dimension analysis is as follows: the clarity of the theme is represented by the semantic similarity between the image semantics and the title / description, and the content coherence is represented by the semantic relevance between shots; The rhythm dimension analysis is as follows: including the shot switching frequency represented by the number of shot switching times per unit time, and the rationality of the time allocation represented by the proportion of core content segment duration; The audio quality dimensions are analyzed as follows: volume stability represented by the standard deviation of volume fluctuation, noise interference level represented by the energy ratio of noise frequency bands, and voice clarity represented by the signal-to-noise ratio of the human voice frequency band.

[0029] In one application example, S30 specifically includes: S301, preset basic weight coefficients: image quality dimension 0.3, composition dimension 0.2, content integrity dimension 0.2, rhythm dimension 0.1, audio quality dimension 0.2; S302, Content Type Recognition: By fusing semantic features and visual features, it automatically identifies the content type of short videos, including vlogs, product promotions, popular science, music performances, photography exhibitions, etc. S303, Dynamic Weight Adjustment: The weights of each dimension are adaptively adjusted based on the type of content being identified. For example, for music performances: the weight of the audio quality dimension is increased to 0.3, and the weight of the composition dimension is decreased to 0.15; for photography exhibitions: the weight of the image quality dimension is increased to 0.4, and the weight of the audio quality dimension is decreased to 0.1; for popular science: the weight of the content completeness dimension is increased to 0.3, and the weight of the audio quality dimension is adjusted to 0.2. S304, User Preference Learning: Based on user modification behavior data in the historical record storage module, the weight allocation is optimized through the gradient descent algorithm. For example, if a user adopts composition optimization suggestions multiple times, the weight of the composition dimension is gradually increased. S305, Comprehensive score calculation: The comprehensive score is calculated using a weighted summation formula. The comprehensive score = picture quality score × dynamic weight 1 + composition score × dynamic weight 2 + content integrity score × dynamic weight 3 + rhythm score × dynamic weight 4 + audio quality score × dynamic weight 5. Each dimension score is a quantitative value of 0-100 points.

[0030] In one application example, S40 specifically includes: Image quality rating: Sharpness gradient value ≥250, corresponding score ≥95; average brightness in the range of [120, 180], corresponding score ≥90; contrast ratio ≥1.8, corresponding score ≥90; color saturation in the range of [55, 75], corresponding score ≥90. Composition dimension score: The distance between the main subject position and the golden ratio point is ≤30px, the difference rate of the image balance is ≤10%, and the reasonableness of the white space ratio is in the range of [20%, 35%]; The scoring criteria for completeness of content are as follows: Topic clarity and semantic similarity ≥90%, corresponding to a score ≥90 points; Content coherence and relevance ≥85%, corresponding to a score ≥85 points. Pace-based scoring: The frequency of camera cuts should be appropriate for the content type, for example, 3-5 times / minute for vlogs and 2-4 times / minute for product promotions; the core content should account for ≥70% of the total time. Audio quality rating criteria: standard deviation of volume stability ≤ 5dB, noise interference percentage ≤ 5%, and signal-to-noise ratio of human voice clarity ≥ 35dB; Highlights Summary: Extract indicator data that meets any two or more of the above criteria, and generate structured review text based on content characteristics. For example, the video image clarity gradient value reaches 268, the color saturation is 62, and the visual presentation is excellent; the main subject position deviates from the golden ratio point by only 22px, the white space ratio is 28%, and the composition is professional and harmonious.

[0031] In one application example, S50 specifically includes: Image quality rating: Sharpness gradient value <120, corresponding score <60 points; average brightness <80 or >220, corresponding score <60 points; contrast ratio <0.8, corresponding score <60 points; color saturation <30 or >90, corresponding score <60 points. Composition dimension rating: The distance between the subject position and the golden ratio point is greater than 100px, the difference rate of the image balance is greater than 30%, and the reasonableness of the white space ratio is less than 10% or greater than 50%; Content completeness score: Theme clarity and semantic similarity <60%, corresponding score <65 points; Content coherence and relevance <50%, corresponding score <60 points; Pace-based scoring: Camera switching frequency exceeds the adaptation range by 2 times or more; core content duration accounts for less than 40% of the total time. Audio quality rating criteria: volume stability standard deviation > 15dB, noise interference percentage > 20%, and voice clarity signal-to-noise ratio < 15dB. Summary of shortcomings: Extract indicator data that are below the above standards, distinguish between objective quality defects and room for optimization, and generate accurate review text. For example, the video clarity gradient value is only 105, which is lower than the qualified standard (120), and the picture is blurry; the audio noise interference level reaches 28%, which exceeds the qualified threshold (20%), affecting the auditory experience; the main body position deviates from the golden ratio point by 125px, and the composition is unbalanced.

[0032] In one application example, S60 specifically includes: Construct a defect-solution knowledge graph: consisting of a defect type layer (e.g., blurry image, unbalanced composition), a solution layer (e.g., parameter adjustment, post-processing), and an operation guidance layer (e.g., tool selection, parameter configuration), storing 200+ preset related data entries; Integrating large language model to generate suggestions: A lightweight large language model is fine-tuned, taking quantitative data of shortcomings indicators, data of highlights indicators, and content type tags as input, and combining them with the related data in the knowledge graph to generate structured optimization suggestions; To optimize image quality, specific numerical adjustment suggestions are given based on the parameters output by the image feature extraction model. For example, if the sharpness gradient value is 105, it is recommended to optimize it using a bilateral filtering algorithm in post-production software with a filter radius of 3px; if the average brightness is 75, it is recommended to increase the exposure parameter by 0.8EV. Optimize composition: Combine the subject position data identified by the object detection algorithm to generate coordinate adjustment suggestions. For example, if the subject position deviates from the golden section point by 125px, it is recommended to move the subject 100px to the right horizontally and 25px down vertically to align the center of the subject with the golden section point. Optimize content completeness: Based on the semantic similarity analysis results, it is recommended to add or delete relevant content. For example, if the semantic similarity of topic clarity is 58%, it is recommended to add a 3-second topic subtitle at the beginning of the video and add the core description. Optimize the pacing: Based on the frequency of camera switching and the suitability of the content type, give suggestions for adjusting the duration. For example, if the frequency of camera switching is 12 times / minute, which exceeds the adaptability range of vlog (3-5 times / minute), it is recommended to remove 4 redundant shots and extend the duration of the core shot from 8 seconds to 12 seconds. Audio optimization: Based on spectrum analysis data, suggestions for noise reduction and volume adjustment are given. For example, if the noise interference level is 28%, it is recommended to use spectral subtraction noise reduction and set the noise reduction threshold to -25dB; if the standard deviation of volume stability is 18dB, it is recommended to adjust the volume to 85dB and control the fluctuation range within ±3dB. Highlight Preservation Suggestions: For the identified shooting highlights, suggestions for enhancement and continuation are generated. For example, if the color saturation of the image is 62, which meets the excellent standard, it is recommended to retain this color parameter in post-processing. You can also appropriately increase the warm tone by 10% to enhance the visual experience.

[0033] In one application example, on the S70, the specific presentation format may include an app interface pop-up window or a web-based analysis report, allowing users to view, download, or share the analysis results; it provides post-operation guidance corresponding to optimization suggestions, including recommended compatible post-editing tools (such as CapCut, Audacity) and screenshots or video tutorial links for specific operation steps; it allows users to bookmark comments and optimization suggestions as adopted, facilitating subsequent tracking of optimization effects.

[0034] In a specific application example, the S80 can use cloud storage to store user-uploaded short video data, multi-dimensional analysis results, comprehensive scores, optimization suggestions, and user modification behavior data. Users can view past creative analysis through the history interface, and the system automatically generates creative quality change trend charts (including score change curves for each dimension and comprehensive score trend charts) to help users summarize creative experience and continuously improve their creative level.

[0035] An application example may include the following process: It receives short MP4 videos uploaded by users, extracts video and audio streams after format parsing, adjusts the video stream to 1080P resolution and 30fps frame rate, adjusts the audio stream to 44.1kHz sampling rate, and outputs standardized video data. The algorithm uses a CNN model to extract image features such as edge gradient values ​​and grayscale values ​​from video frames, an LSTM model to analyze audio features such as volume fluctuations and noise frequencies, and a BERT model to semantically encode video titles, such as "Handmade Tabby Cat Doll". A three-dimensional feature vector is constructed and fused using an attention mechanism to obtain quantitative data for each dimension: sharpness gradient value 105, average brightness 75, contrast ratio 0.7, color saturation 58, subject position deviation from the golden ratio point 125px, semantic similarity of theme clarity 58%, shot switching frequency 12 times / minute, and noise interference level 28%. The video was identified as a product promotion video based on its content type. Weighting coefficients were dynamically adjusted: image quality 0.35, composition 0.2, content completeness 0.25, pacing 0.1, and audio quality 0.1. Combining the scores from each dimension (image quality 55, composition 50, content completeness 60, pacing 45, and audio quality 40), the overall score was calculated as follows: 55 × 0.35 + 50 × 0.2 + 60 × 0.25 + 45 × 0.1 + 40 × 0.1 = 52.75 points. The color saturation of 58 was detected, which meets the excellent standard (55-75 range). All other indicators did not reach the excellent threshold. Highlight comment: The video color saturation of 58 is in the excellent range, and the visual color presentation is comfortable. The extracted data shows a sharpness gradient value of 105 (<120), an average brightness of 75 (<80), a contrast ratio of 0.7 (<0.8), a subject position deviation of 125px (>100px), a theme clarity similarity of 58% (<60%), a shot switching frequency of 12 times / minute (exceeding the product promotion category's adaptation range of 2-4 times / minute), and a noise interference level of 28% (>20%). The resulting shortcomings are: a video sharpness gradient value of 105, an average brightness of 75, and a contrast ratio of 0.7, resulting in a dark and blurry image; a subject position deviation of 125px from the golden ratio, leading to an unbalanced composition; a theme clarity semantic similarity of 58%, indicating unclear core expression; a shot switching frequency of 12 times / minute, resulting in a chaotic rhythm; and a background noise interference level of 28%, affecting the auditory experience. Input the aforementioned data on shortcomings, highlights, and product promotion tags into the fine-tuned large language model, and combine it with knowledge graph-related data to generate optimization suggestions: 1. Visual optimization: Use the bilateral filtering function (filter radius 3px) in video editing software to improve clarity, increase exposure parameters by 0.8EV, and adjust contrast to 1.2 times; 2. Composition optimization: In CapCut, use the image cropping tool to move the subject 100px to the right horizontally and 25px down vertically, aligning the subject's center with the golden ratio point; 3. Content optimization: Add a 3-second title tag at the beginning of the video, such as 'The entire process of making a handmade ceramic cup,' and add a 20-second segment of core production steps; 4. Rhythm optimization: Delete 4 redundant transition shots, extend the duration of core shots from 8 seconds to 12 seconds, and control the shot switching frequency to 3 times / minute; 5. Audio optimization: Use spectral subtraction noise reduction (threshold -25dB) in audio editing software to uniformly adjust the volume to 85dB, controlling the fluctuation range within ±3dB; 6. Highlights Preserved: Retains the current color saturation parameters and can add a 10% warm tone filter to enhance the visual effect; The report is presented in the form of a web-based analysis report, showing an overall score of 52.75, highlights, shortcomings, and the above optimization suggestions, and supports users in saving and downloading the report. The video data, analysis results, optimization suggestions, and subsequent user modifications (such as adopting audio optimization suggestions) are stored on a cloud server. Users can later view before-and-after comparison data of the video, and the system automatically generates a curve showing the change in the video's creative quality. The analysis and optimization data are used as iterative knowledge graphs for subsequent video analysis.

[0036] The AI-based multimodal model-based short video shooting scoring method described above achieves in-depth analysis of short videos through multimodal fusion analysis algorithms, quantifies the criteria for judging highlights and shortcomings, generates a comprehensive score using a dynamic weight adaptive algorithm, and provides targeted overall optimization suggestions based on a clear technical path, helping users clearly understand the strengths and weaknesses of their creations and continuously improve the quality of their short video creations.

[0037] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A short video assisted shooting and scoring method based on an AI multimodal model, characterized in that, Includes the following steps: S10, Video Acquisition: Receive short videos uploaded by users to be analyzed, perform format parsing and preprocessing on the short videos, and obtain standardized video data; S20, Video Analysis: A visual-audio-semantic multimodal fusion analysis algorithm is used to perform multi-dimensional fusion analysis on standardized video data to obtain multi-dimensional fusion analysis results; S30, Score Generation: Based on the results of multi-dimensional fusion analysis, a dynamic weight adaptive algorithm is used to calculate the comprehensive score of the short video to be analyzed; S40, Highlight Identification: Extract indicator data that meets the quantitative excellence standard from the multi-dimensional fusion analysis results, and combine it with the content characteristics of the short video to identify and summarize the shooting highlights of the short video; S50, Defect Identification: Extract indicator data that are below the quantitative qualification standard from the multi-dimensional fusion analysis results of the video analysis module, and combine them with the creation logic of the short video to identify and summarize the shooting defects of the short video. S60, Optimization suggestion generation: Based on the identified shooting shortcomings and the need to retain shooting highlights, targeted overall optimization suggestions are generated; S70 output: The overall score, shooting highlights, shooting shortcomings, and overall optimization suggestions are presented to the user in a visual format.

2. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 1, characterized in that, Further includes: S80, Historical Record Storage: It uses cloud storage to store user-uploaded short video data, multi-dimensional analysis results, comprehensive scores, optimization suggestions, and user modification behavior data; Users can view past creative analysis through the history interface, and the system automatically generates charts showing the trend of changes in creative quality.

3. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 1 or 2, characterized in that, S10 specifically includes: receiving short videos uploaded by users for analysis via mobile APP interface or web upload interface, extracting video stream and audio stream, standardizing the size and frame rate of the video stream, standardizing the sampling rate of the audio stream, and finally outputting standardized video data.

4. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 1 or 2, characterized in that, S20 specifically includes: It includes a built-in image feature extraction model based on Convolutional Neural Network (CNN), an audio feature extraction model based on Long Short-Term Memory Network (LSTM), and a semantic feature extraction model based on BERT. The image feature extraction model extracts image features including edge contours, color distribution, and pixel density of video frames. The audio feature extraction model extracts human voice and ambient sound features including volume changes, frequency distribution, and noise characteristics. The semantic feature extraction model performs text encoding on video titles, subtitles, and image semantic tags to obtain semantic features. A three-dimensional feature vector is constructed, and an attention mechanism is introduced to assign weights and fuse three types of features: image, audio, and semantics, thereby enhancing the feature expression of key information. The analysis dimensions include image quality, composition, content completeness, rhythm, and audio quality.

5. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 4, characterized in that, The image quality dimensions are analyzed as follows: the gradient value is calculated using an edge detection algorithm to represent standard sharpness; the average value of pixel grayscale values ​​is used to represent brightness; the ratio of the range to the mean of pixel grayscale values ​​is used to represent contrast; and the average value of the S channel in the HSV color space is used to represent saturation.

6. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 4, characterized in that, The composition dimensions are analyzed as follows: the pixel distance between the center of the subject and the golden ratio point is used to represent the position of the subject; the pixel distribution difference rate of the left and right / up and down areas of the image is used to represent the balance of the image; and the proportion of the blank area to the total area of ​​the image is used to represent the rationality of the blank space.

7. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 4, characterized in that, S30 specifically includes: S301, preset basic weight coefficients: image quality dimension 0.3, composition dimension 0.2, content integrity dimension 0.2, rhythm dimension 0.1, audio quality dimension 0.2; S302, Content Type Recognition: Automatically identify the content type of short videos by fusing semantic features and visual features; S303, Dynamic Weight Adjustment: Adaptively adjusts the weights of each dimension based on the identified content type; S304, User Preference Learning: Based on user modification behavior data in the historical record storage module, the weight allocation is optimized through the gradient descent algorithm; S305, Comprehensive score calculation: The comprehensive score is calculated using a weighted summation formula. The comprehensive score = picture quality score × dynamic weight 1 + composition score × dynamic weight 2 + content integrity score × dynamic weight 3 + rhythm score × dynamic weight 4 + audio quality score × dynamic weight 5. Each dimension score is a quantitative value of 0-100 points.

8. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 7, characterized in that, S40 specifically includes: Image quality rating: Sharpness gradient value ≥250, corresponding score ≥95; average brightness in the range of [120, 180], corresponding score ≥90; contrast ratio ≥1.8, corresponding score ≥90; color saturation in the range of [55, 75], corresponding score ≥90. Composition dimension score: The distance between the main subject position and the golden ratio point is ≤30px, the difference rate of the image balance is ≤10%, and the reasonableness of the white space ratio is in the range of [20%, 35%]; The scoring criteria for completeness of content are as follows: Topic clarity and semantic similarity ≥90%, corresponding to a score ≥90 points; Content coherence and relevance ≥85%, corresponding to a score ≥85 points. Rhythm dimension score: The frequency of camera transitions is adapted to the content type; Audio quality rating criteria: standard deviation of volume stability ≤ 5dB, noise interference percentage ≤ 5%, and signal-to-noise ratio of human voice clarity ≥ 35dB; Highlights Summary: Extract indicator data that meets any two or more of the above criteria, and generate structured commentary text based on content characteristics.

9. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 4, characterized in that, The S50 specifically includes: Image quality rating: Sharpness gradient value <120, corresponding score <60 points; average brightness <80 or >220, corresponding score <60 points; contrast ratio <0.8, corresponding score <60 points; color saturation <30 or >90, corresponding score <60 points. Composition dimension rating: The distance between the subject position and the golden ratio point is greater than 100px, the difference rate of the image balance is greater than 30%, and the reasonableness of the white space ratio is less than 10% or greater than 50%; Content completeness score: Theme clarity and semantic similarity <60%, corresponding score <65 points; Content coherence and relevance <50%, corresponding score <60 points; Pace-based scoring: Camera switching frequency exceeds the adaptation range by 2 times or more; core content duration accounts for less than 40% of the total time. Audio quality rating criteria: volume stability standard deviation > 15dB, noise interference percentage > 20%, and voice clarity signal-to-noise ratio < 15dB. Summary of shortcomings: Extract indicator data that are below the above standards, distinguish between objective quality defects and room for improvement, and generate accurate review copy.

10. The short video assisted shooting and scoring method based on an AI multimodal model as described in claim 4, characterized in that, The S60 specifically includes: Constructing a Defect-Solution Knowledge Graph: The knowledge graph consists of three layers: the defect type layer, the solution layer, and the operation guidance layer. Integrating large language model to generate suggestions: A lightweight large language model is fine-tuned, taking quantitative data of shortcomings indicators, data of highlights indicators, and content type tags as input, and combining them with the related data in the knowledge graph to generate structured optimization suggestions; To optimize image quality, specific numerical adjustment suggestions are provided based on the parameters output by the image feature extraction model. Optimize the composition: Combine the subject position data identified by the target detection algorithm to generate coordinate adjustment suggestions; Optimize content completeness: Based on the semantic similarity analysis results, it is recommended to add or delete relevant content; Optimize the pacing: Provide duration adjustment suggestions based on the frequency of camera transitions and the suitability of the content type; Audio optimization: Based on spectrum analysis data, suggestions for noise reduction and volume adjustment are provided; Highlight retention suggestions: Generate suggestions for enhancing and extending the identified highlights in the shot.

Images