A short video-based customer mining method and system

Abstract

The application discloses a short video-based customer mining method and system, comprising the following steps: obtaining public content index data and original data stream of a short video through a short video platform interface; generating a structure segmentation section according to a structure conversion point in a time sequence text, a lens mutation point in a picture and an audio energy jump point; constructing a structure section type identification model to identify the type label of each structure section; constructing a content feature weight distribution model to determine a video control vector and generate a short video script; loading a customer voiceprint model file, a digital human model and visual supplementary materials based on the short video script to generate short video content; and obtaining short video behavior data to evaluate the contribution of content features to the behavior data and adjust the video control vector. The application can significantly improve the pertinence, effectiveness and continuous optimization capability of short video content in the customer mining scene, and improve the capacity of short video mass production and precise reach of potential customers.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information processing technology, and in particular to a customer mining method and system based on short videos. Background Technology

[0002] With the widespread adoption of mobile internet and smart devices, short video platforms have gradually become an important medium for information dissemination and commercial promotion. Compared to traditional text and images or long videos, short videos are characterized by shorter production cycles, faster dissemination, and higher user acceptance, and have been widely used in brand promotion, product promotion, customer outreach, and potential customer acquisition. Especially in marketing and customer acquisition, companies hope to attract user attention, stimulate interaction, and further guide users to complete conversions through short video content, thereby improving customer acquisition efficiency. However, existing customer acquisition methods based on short videos mostly rely on manual experience or simple data statistics rules for content planning and placement. On the one hand, content creators typically design short videos based on subjective experience, lacking refined analysis of the internal structure and semantic layers of the videos, resulting in significant uncertainty in the pacing, emotional progression, and information presentation of the generated video content. Furthermore, with the increasing homogenization of short video content, simply adding "trigger points" or strong stimulating elements is no longer sufficient to continuously attract users. Existing short video generation or optimization solutions often frequently or blindly insert viral content without fully considering the video's content carrying capacity and structural rationality. This can easily lead to user fatigue and even interfere with the overall content expression, reducing the actual effectiveness of the short video. Without structural analysis and constraint mechanisms, the configuration of viral points often fails to effectively match the video content, contextual information, and target audience characteristics. Therefore, how to structurally deconstruct and multimodally understand short video content in customer mining scenarios, achieve effective linkage between content modeling and viral point configuration at the structural segment level, and construct a short video content generation mechanism to improve the targeting and effectiveness of short videos in customer mining scenarios has become a pressing technical problem to be solved in this field. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a customer mining method and system based on short videos.

[0004] The first aspect of this invention provides a customer mining method based on short videos, mainly comprising: Obtain the public content index data and raw data stream of short videos through the short video platform interface; Based on the structural transition points in the temporal text, the abrupt change points in the image, and the audio energy jump points, generate structural segmentation sections; Based on the structural features of the segmented sections, a structural segment type identification model is constructed to identify the type label of each structural segment and generate a structural label matrix. Based on the content features and contextual information of the structural segments, a content feature weight allocation model is constructed to assign weights to each feature of each structural segment, determine the video control vector, and generate short video script text. Based on the generated short video script, the client's voiceprint model file is loaded for speech synthesis, and a voice timestamp alignment vector is generated based on the synthesized speech. The digital human model is then used for synchronous rendering and insertion of visual supplementary materials to generate short video content. Short video behavior data is obtained by calling the data feedback interface of the content distribution platform. The timeline alignment mechanism is used to match structural segments and behavior events, calculate the response probability of structural segments, evaluate the contribution of content features to the behavior data, and adjust the video control vector.

[0005] Furthermore, the step of obtaining the publicly available content index data and raw data stream of short videos through the short video platform interface includes: Based on the industry keywords and content tag groups set by the user, the system obtains the public content index data and raw data stream of short videos through the short video platform interface within a preset time window, and sorts the popularity of short videos within the preset time window. The public content index data includes short video cover features, short video content links, title text, comment summary, publication timestamp, number of views, interaction data and author identifier. Interaction data includes the number of likes, comments and reposts. The raw data stream includes audio stream, short video frame data and subtitle information.

[0006] Furthermore, the step of generating structural segmentation segments based on structural transition points in the short video temporal text, abrupt change points in the image, and audio energy jump points includes: The audio parsing module extracts speech features from short videos and transcribes them into time-series text data. Frame-level image slicing is performed on the short video image content to obtain a sequence of visual information including character poses, shot transition frequencies, background music (BGM) changes, and subtitle appearances. Image and speech information are time-aligned, and a spatiotemporal sequence graph is established based on frame-level positional mapping. Sentiment analysis and text grammar parsing tools are used to identify structural transition points in the time-series text. Based on the short video frame data, the SSIM algorithm is used to compare the similarity between adjacent frames to detect shot transitions. Short-time Fourier transform is used to extract the audio's spectral information, and energy jump points in the audio are identified based on the root mean square (RMS) energy. Based on the structural transition points in the time-series text, combined with shot transition points in the image and audio energy jump points, structural segmentation segments are generated. The structural features of these segments are obtained and stored in a structural segmentation database. These structural features include keyword density, sentiment fluctuation values, and visual jump degrees.

[0007] Furthermore, the step of constructing a structural segment type identification model based on the structural features of the segmented sections, identifying the type label of each structural segment, and generating a structural label matrix includes: Historical data on the structural features of segmented sections are obtained from the structural segment database. Each segment is labeled with a type tag, and a random forest algorithm is used to train the model to build a structural segment type recognition model. The type tags include introductory segments, value display segments, emotional progression segments, and behavioral call-to-action segments. Based on the real-time acquired structural features of segmented sections, the structural segment type recognition model is used to identify the type tag of each segment, generate a structural tag matrix, extract the content features of each segment in the structural tag matrix, and store them in the structural segment database. The content features include emotional fluctuation curves, visual transitions, keyword intent, and semantic context.

[0008] Furthermore, the step of assigning weights to each feature of each structural segment based on the content features of the structural segments and the context information of the short video, determining the video control vector, and generating short video script text in conjunction with viral factor prompts includes: By leveraging a structural segment database, historical data on the content features of structural segments and their type labels are obtained. Combined with the contextual information of the short videos, a recurrent neural network is used for model training to construct a content feature weight allocation model. Contextual information includes the video's industry category, video content style tags, and target audience profiles. Based on the structural segment content features provided by the user, the content feature weight allocation model assigns weights to each feature of each structural segment. The content features and weights of each structural segment are encoded as sparse or dense vector representations and concatenated into a video control vector. Based on the structural segment content features, the potential for each segment to become viral is assessed, and viral point configuration parameters are determined. These parameters include the selection of viral factor type, placement constraints, and viral expression methods. By inputting the video control vector and preset viral factor prompts into a large language model, a short video script tailored to the client's needs is generated. Preset viral factors include visual hooks, stylistic parameters, expression tendency factors, colloquialism control variables, and viral point configuration parameters.

[0009] It also includes, based on the content characteristics of the structural segments, determining the detonability of each structural segment and identifying the detonation point configuration parameters, specifically including: Based on the content characteristics of structural segments, the detonability determination formula is used. Calculate the detonability determination value of the structural segment. ,in, The continuous semantic length of the structural segment represents the temporal length of the structural segment. To preset the minimum threshold for the semantic span that can be triggered, This represents the extreme value of emotion within the structural segment. The effective threshold for emotional fluctuation. This represents the number of visual changes within a structural segment. The minimum visual disturbance threshold, For symbolic functions, The function is a floor function; if the detonability judgment value is less than or equal to the preset judgment threshold, the generation of detonation point configuration parameters in the corresponding structural segment is prohibited; if the detonability judgment value is greater than the preset judgment threshold, the structural segment is determined to be detonable; through the structural segment database, historical data of structural segment content features and structural segment type labels are obtained, and combined with the context information of the structural segment, a recurrent neural network is used to train the model and construct a detonation point configuration parameter prediction model; based on the real-time obtained structural segment content features, type labels and context information, the detonation point configuration parameters of each structural segment are determined.

[0010] Furthermore, based on the generated short video script, the client's voiceprint model file is loaded for speech synthesis, and a speech timestamp alignment vector is generated according to the synthesized speech. Synchronous rendering and visual supplementary material insertion are then performed using a digital human model to generate short video content, including: Based on the generated short video script, the speech synthesis module is invoked to load the client's voiceprint model file, setting the speech rate range, pitch characteristics, and emotion parameter values ​​to synthesize text into speech. A speech timestamp alignment vector is generated from the synthesized speech, using the start and end times of each sentence as subsequent rendering annotation nodes. The client's digital human model is loaded, and the set character appearance, action style, and camera expression template information are read, bound to the speech synthesis output, and a synchronous rendering process is executed. Based on the structural tag matrix, predefined visual supplementary materials, including product shots, environmental transitions, emotional backgrounds, and dynamic subtitles, are inserted during each structural transition. The generated short video content undergoes multi-channel compression, image enhancement, and audio noise reduction post-processing according to structural tags, and is uniformly packaged into a short video publishing format supported by the target platform. The short video content is sent to the bound content distribution platform through the publishing interface, recording metadata at the time of publication, including structural information, short video identifier, and push status markers.

[0011] Furthermore, the process of obtaining short video behavior data based on the data feedback interface of the content distribution platform, matching structural segments with behavior events using a timeline alignment mechanism, calculating the response probability of structural segments, evaluating the contribution of content features to the behavior data, and adjusting the video control vector includes: By calling the data feedback interface of the content distribution platform, behavioral data of short videos within a preset time window is obtained, including play count, completion rate, number of interactions, repost frequency, comment semantic vector, collection rate, and content popularity gradient. The acquired behavioral data is categorized by short video ID and platform identifier using a data aggregation module, establishing a unified data structure table containing timestamps, behavioral event types, behavioral intensity values, and user interaction paths. A timeline alignment mechanism is used to match the time range of each structural segment in the structural tag matrix with the occurrence time of behavioral events, constructing a mapping relationship between structural segments and behavioral events. An event trigger probability model is used to calculate the response probability of each structural segment on different types of behavioral events, forming a structural segment behavioral response matrix to determine the relationship between each structural segment and the behavior. The relevance between events is assessed, with structural segments having a relevance higher than a preset relevance threshold labeled as behavior-driven structures and those lower than the threshold labeled as low-impact structures. Based on the type labels, content features, and response metrics of each structural segment, a multiple linear regression algorithm is used for model training. The contribution of each content feature to the behavioral data is evaluated based on the R² and P-values ​​of the model fit, along with response metrics such as like rate, comment trigger rate, collection rate, and completion rate. If the relevance between a content feature and a behavioral metric is lower than the preset relevance threshold, that content feature is marked as pending update. Based on the marking results, a gradient correction model is constructed using backpropagation to adjust the video control vector input to the large language model, and its mapping relationship is stored in the short video generation database.

[0012] A second aspect of the present invention provides a customer mining system based on short videos, mainly comprising: The short video content acquisition module is used to acquire the public content index data and raw data stream of short videos through the short video platform interface; The video structure segmentation module is used to generate structure segmentation segments based on the structure transition points in the short video time-series text, the lens change points in the image, and the audio energy jump points. The structural segment type identification module is used to construct a structural segment type identification model based on the structural features of the structural segmentation sections, identify the type label of each structural segment, and generate a structural label matrix. The script copy generation module is used to assign weights to each feature of each structural segment based on the content features of the structural segments and the context information of the short video, determine the video control vector, and generate short video script copy by combining the viral factor prompt words. The short video generation module is used to load the customer's voiceprint model file for speech synthesis based on the generated short video script, generate a speech timestamp alignment vector based on the synthesized speech, and generate short video content through synchronous rendering and insertion of visual supplementary materials using a digital human model. The control vector adjustment module is used to obtain short video behavior data by calling the data feedback interface of the content distribution platform, match structural segments and behavior events using a timeline alignment mechanism, calculate the response probability of structural segments, evaluate the contribution of content features to the behavior data, and adjust the video control vector.

[0013] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention provides a customer mining method and system based on short videos. By performing multimodal joint analysis on publicly available short video content index data and raw data streams, this invention automatically divides short videos into multiple structural segments. At the structural segment level, it models content features, type tags, and triggerability, thereby achieving a refined understanding of the internal semantic structure and content carrying capacity of short videos. This invention introduces a content feature weight allocation mechanism and video control vectors, making the generation process of short video scripts structurally controllable and semantically interpretable. Through triggerability judgment and trigger point configuration parameter generation, it avoids ineffective insertion of triggering content into structural segments lacking carrying capacity, improving the matching degree between trigger points and video structure and contextual information. Furthermore, this invention combines digital human rendering and speech synthesis technologies to achieve automated generation and publishing of short video content. Through timeline alignment analysis of user behavior data, it establishes a mapping relationship between structural segments and user behavior, evaluates and corrects the contribution of content features to behavioral responses, thus forming a closed-loop optimization mechanism for short video generation and user behavior feedback. The technical solution provided by this invention can significantly improve the targeting, effectiveness, and continuous optimization capabilities of short video content in customer discovery scenarios, reduce reliance on human experience, and enhance the ability of short videos to be produced on a large scale and accurately reach potential customers. Attached Figure Description

[0014] Figure 1 This is a flowchart of a customer mining method based on short videos according to the present invention; Figure 2 This is a schematic diagram of a customer mining method based on short videos according to the present invention; Figure 3 This is a schematic diagram of a customer mining system based on short videos according to the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] like Figures 1-2 This embodiment of a customer mining method based on short videos may specifically include: Step S101: Obtain the public content index data and raw data stream of the short video through the short video platform interface.

[0017] Based on the industry keywords and content tag groups set by the user, the system obtains the public content index data and raw data stream of short videos through the short video platform interface within a preset time window, and sorts the popularity of short videos within the preset time window. The public content index data includes short video cover features, short video content links, title text, comment summary, publication timestamp, number of views, interaction data and author identifier. Interaction data includes the number of likes, comments and reposts. The raw data stream includes audio stream, short video frame data and subtitle information.

[0018] For example, during the creation and optimization of short video content, an analysis is being conducted on a short video about "how to improve workplace efficiency." The user sets industry keywords ("workplace improvement") and content tags ("time management" and "efficient work") on the platform, and retrieves relevant data within a preset 48-hour time window via the platform's API. The publicly available content index data returned by the platform includes the video's cover image, video link, title, comment summary, publication timestamp, number of views, and interaction data. The resulting video title is "Master these 3 skills, and improving workplace efficiency is not a dream!", and its cover image shows a professional using a time management tool. The cover text is concise and clear, highly consistent with the video's theme. The video has reached 150,000 views, and the interaction data is also very positive, including 20,000 likes, 1,500 comments, and 800 shares, all indicating that the video has received widespread attention and interaction. Furthermore, the platform also provides the video's raw data stream, including the audio stream, short video frame data, and subtitle information. The audio stream contains the narrator's voice, explaining specific methods for "how to manage time effectively" and "enhancing workplace competitiveness." The short video frame data provides information such as the characters' postures, camera transitions, and background changes. Subtitles are displayed synchronously with the audio content, ensuring viewers clearly understand every piece of advice and information in the video, such as keywords like "time management is crucial" and "efficient work significantly improves workplace competitiveness." Based on this publicly available content index data and the raw data stream, the platform ranked the videos by popularity within 48 hours.

[0019] Step S102: Generate structural segmentation segments based on structural transition points in the temporal text, shot abrupt change points in the image, and audio energy jump points.

[0020] The audio parsing module extracts speech features from short videos and transcribes them into time-series text data. Frame-level image slicing is performed on the short video image content to obtain a sequence of visual information including character poses, shot transition frequencies, background music (BGM) changes, and subtitle appearances. Image and speech information are time-aligned, and a spatiotemporal sequence graph is established based on frame-level positional mapping. Sentiment analysis and text grammar parsing tools are used to identify structural transition points in the time-series text. Based on the short video frame data, the SSIM algorithm is used to compare the similarity between adjacent frames to detect shot transitions. Short-time Fourier transform is used to extract the audio's spectral information, and energy jump points in the audio are identified based on the root mean square energy (RMS). Based on the structural transition points in the time-series text, combined with shot transition points in the images and audio energy jump points, structural segmentation segments are generated. The structural features of these segments are obtained and stored in a structural segmentation database. These structural features include keyword density, sentiment fluctuation values, and visual jump degrees.

[0021] For example, based on the raw data stream of the short video, including the audio stream, video frame data, and subtitle information, and assuming the video length is 30 seconds, the public content index data of the video was obtained from the video platform interface. This data includes the video cover, title, 500,000 views, comment summary, user discussions on how to improve work efficiency, 10,000 likes, 3,000 shares, and the video's publication timestamp of June 1, 2020. From this data, the video's content link, i.e., the URL pointing to the video, was also obtained. The audio parsing module extracted the speech features from the video. The spoken content in the video was transcribed into time-series text data, essentially converting the speech into subtitles. For instance, the first few seconds of the video contain the phrase "Today we will discuss how to improve work efficiency through time management." The speech parsing module extracted the corresponding time-series text data and marked the start and end times of each sentence. Simultaneously, frame-level image slicing was performed on the video's image content. Image information in the video includes character poses, camera transition frequency, background music (BGM) changes, and subtitle placement. For example, when a person in the video presents PowerPoint slides, the image processing module extracts each frame and records the character's pose changes across different frames, as well as the timing of subtitle appearances. Furthermore, camera transitions are also recorded; for instance, at the 15-second mark, the camera switches from a PowerPoint presentation to a close-up of the presenter. Image and audio information are time-aligned. For example, the audio text "How to improve efficiency" appears between the 5th and 7th seconds of the video, and the subtitle "How to improve efficiency" also appears in the lower left corner of the screen at that exact moment. This information allows for synchronization of audio and image frames, constructing a spatiotemporal sequence diagram and precisely aligning audio content with video frame data, ensuring consistency between audio and visual information on the timeline. Finally, text grammar analysis using the sentiment analysis tools VADER and spaCy identifies structural transition points in the video. For example, at the 30-second mark of the video, the audio content shifts from "the importance of time management" to "how to create a daily time plan." This is a structural transition point, signifying a shift from one theme to another, and the emotional tone also changes, from slightly excited to calm. This transition point is accurately marked. Based on video frame data, the SSIM algorithm is used to compare the similarity between adjacent frames, detecting shot change points in the video. For instance, at the 15-second mark, the camera switches from the speaker to the PPT presentation; this change point is marked as a shot change point. Simultaneously, short-time Fourier transform is used to extract the audio's spectral information. Based on the root mean square energy (RMS) of the audio, a significant energy jump is identified at the 25-second mark. This point indicates an increase in the speaker's volume, possibly to emphasize a key point.Structural segmentation segments were generated by analyzing structural transition points in temporal text, shot change points in images, and energy jump points in audio. These segments represent different structural parts of the video. The structural features of each segment, including keyword density, sentiment fluctuation value, and visual jump degree, were extracted and stored in the structural segmentation segment database.

[0022] Step S103: Based on the structural features of the segmented sections, construct a structural segment type recognition model, identify the type label of each structural segment, and generate a structural label matrix.

[0023] Historical data on the structural features of segmented sections are obtained from a structural segment database. Each segment is labeled with a type tag, and a random forest algorithm is used to train a model to construct a structural segment type recognition model. Type tags include introductory segments, value demonstration segments, emotional progression segments, and behavioral call-to-action segments. Based on the real-time acquired structural features of segmented sections, the structural segment type recognition model identifies the type tag of each segment, generating a structural tag matrix. Content features of each segment in the structural tag matrix are extracted and stored in the structural segment database. Content features include emotional fluctuation curves, visual transitions, keyword intent, and semantic context.

[0024] For example, historical structural feature data of short videos is obtained through a structural segmentation database. Each structural segment is labeled with a type tag, including introductory segments, value display segments, emotional progression segments, and call-to-action segments. For instance, an introductory segment has a keyword density of 0.6, a low emotional fluctuation value of 0.2, and a visual jump rate of 0.3; a value display segment has a high keyword density of 0.8, a large emotional fluctuation value of 0.7, and a high visual jump rate of 0.75; an emotional progression segment has a keyword density of 0.7, an emotional fluctuation value of 0.85, and a visual jump rate of 0.9; and a call-to-action segment has a keyword density of 0.5, a low emotional fluctuation value of 0.4, but a high visual jump rate of 0.8, and so on, totaling 3000 historical data points. Based on the obtained historical structural feature data and type tags, a random forest algorithm is used to train a model to construct a structural segment type recognition model. This model can automatically label the type of video structural segments based on the input short video feature data. If real-time structural segmentation feature data of a short video is acquired, a structural segment type recognition model automatically labels each structural segment of the video with a type label and generates a structural label matrix. Content features of each structural segment are extracted from the structural label matrix and stored in the structural segmentation database. These content features include emotion fluctuation curves, visual abrupt changes, keyword intent, and semantic context.

[0025] Step S104: Based on the content features of the structural segments and the contextual information of the short video, assign weights to each feature of each structural segment, determine the video control vector, and generate the short video script text by combining the viral factor prompt words.

[0026] By leveraging a structural segment database, historical data on the content features of structural segments and their type labels are obtained. Combined with the contextual information of the short videos, a recurrent neural network is used for model training to construct a content feature weight allocation model. Contextual information includes the video's industry category, video content style tags, and target audience profiles. Based on the structural segment content features provided by the user, the content feature weight allocation model assigns weights to each feature of each structural segment. The content features and weights of each structural segment are encoded as sparse or dense vector representations, which are then concatenated into a video control vector. Based on the structural segment content features, the potential for each segment to become viral is assessed, and detonation point configuration parameters are determined. These parameters include detonation factor type selection, placement constraints, and detonation expression methods. By inputting the video control vector and preset viral factor prompts into a large language model, a short video script tailored to the client's needs is generated. Preset viral factors include visual hooks, stylistic parameters, expression tendency factors, colloquialism control variables, and detonation point configuration parameters.

[0027] For example, by using a structural segmentation database, historical feature data of each structural segment of the short video was extracted. The type label jump level was 0.2, indicating minimal visual variation, primarily showing the speaker introducing the video content. The keyword density was 0.6, with frequent appearances of words such as "welcome," "introduction," and "efficiency improvement." Contextual information was incorporated, including the video's industry category, content style tags such as educational and practical, and target audience profiles such as age, gender, interests, completion rate, number of comments, number of likes, and collection rate. A recurrent neural network was used to train the model, constructing a content feature weight allocation model. By acquiring the structural features of segmented sections in real time, a content feature weighting model is used to encode the content features and weight values ​​of each segment, and then concatenate them into a video control vector. For example, the content features and weights of the opening segment may be encoded into a sparse or dense vector. If the content features of this segment are: emotional fluctuation value 0.3, weight 0.2; visual abruptness 0.2, weight 0.1; keyword density 0.6, weight 0.3, then the control vector for this segment is [0.06, 0.02, 0.18]. Based on the content features of each segment, a triggerability assessment is performed. If a segment is deemed triggerable, the trigger point configuration parameters are determined. Subsequently, the control vector and preset viral factor prompts are input into a large language model to generate a short video script tailored to the client's needs. Viral factors include visual hooks, stylistic features, expressive tendencies, and trigger point configuration parameters. Visual hooks can insert visually impactful scenes in the opening introduction. Trigger point configuration parameters include trigger factor type selection, placement constraints, and trigger expression methods. Trigger expression methods include first-person experiential, comparative, question-guided, emotionally resonant, factual, neutral, sudden reversal, emotionally charged, viewpoint conflict, and rhythmic cut-off. Applying trigger point configuration parameters ensures higher audience engagement and interaction. For example, in the emotional progression segment, an empathy factor is selected, describing how working professionals cope with heavy workloads and time management challenges to evoke emotional resonance in the audience. This empathy factor is inserted into the emotional progression segment of the video to ensure emotional resonance during periods of significant emotional fluctuation. Meanwhile, to enhance the video's practicality, value factors were used in the value demonstration segment, explaining how to improve work efficiency through time block management and emphasizing the enhancement of workplace skills. These triggering factors were combined with emotional changes during speech synthesis, visual effects during camera transitions, and cues in the subtitles to ensure that these factors could fully ignite the audience's interest at appropriate points. Combining video control vectors and triggering factor cues, the large language model generated a short video script tailored to the client's needs.For example, the script generated for the opening introduction might be: "Welcome to today's video! In the next few minutes, we'll share how simple time management techniques can help you improve your competitiveness in the workplace. We'll delve into several key time management methods, hoping these techniques will help you be more efficient at work." As the video progresses into the emotionally charged section, the script gradually becomes more intense: "Next, we'll discuss how to set daily work plans and improve your work efficiency through time block management. We've found that by setting clear goals and plans, work becomes more organized." Based on the content characteristics of the structural segments, the detonability of each structural segment is determined, and the detonation point configuration parameters are determined.

[0028] Based on the content characteristics of structural segments, the detonability determination formula is used. Calculate the detonability determination value of the structural segment. ,in, The continuous semantic length of the structural segment represents the temporal length of the structural segment. To preset the minimum threshold for the semantic span that can be triggered, This represents the extreme value of emotion within the structural segment. The effective threshold for emotional fluctuation. This represents the number of visual changes within a structural segment. The minimum visual disturbance threshold, For symbolic functions, This is a floor function. If the detonability judgment value is less than or equal to a preset judgment threshold, generating detonation point configuration parameters in the corresponding structural segment is prohibited. If the detonability judgment value is greater than the preset judgment threshold, the structural segment is determined to be detonable. Historical data of structural segment content features and structural segment type labels are obtained from the structural segment database. Combined with the context information of the structural segments, a recurrent neural network is used to train the model and construct a detonation point configuration parameter prediction model. Based on the real-time acquired structural segment content features, type labels, and context information, the detonation point configuration parameters for each structural segment are determined.

[0029] For example, when performing structural analysis on short videos from a specific e-commerce industry, a structural segment is identified from a structural segmentation database. This is a value demonstration segment, with a duration of 12 seconds, corresponding to a continuous semantic length of... Also 12 seconds. The minimum detonable semantic span threshold preset in the system. The duration is 5 seconds, indicating that a structural segment must last at least 5 seconds to possess the basic semantic space to carry the trigger point content. After sentiment analysis of this structural segment, the maximum sentiment value was 0.82 and the minimum sentiment value was 0.30. Therefore, the sentiment range of this structural segment is... The value is 0.52. Meanwhile, the effective emotional fluctuation threshold is manually set in the system. The value is 0.40, used to limit the minimum amplitude of emotional change that the user can clearly perceive. Furthermore, through shot change detection of video frames within this structural segment, six obvious visual changes were identified within this segment, corresponding to the number of visual changes... The value is 6, while the preset minimum visual disturbance threshold is... The value of 3 indicates that at least three visual changes are required within a structural segment to form an effective visual stimulus. Under the above conditions, the relevant values ​​are substituted into the detonability determination formula. A judgment is made. Since the ratio of the continuous semantic length of the structural segment to the minimum triggerable semantic span threshold is 12 / 5, rounded down to 2, and the emotional range value of 0.52 is greater than the effective emotional fluctuation threshold of 0.40, the corresponding sign function result is positive, and the number of visual changes of 6 is also greater than the minimum visual perturbation threshold of 3, the corresponding sign function result is also positive. Therefore, the final triggerability judgment value of this structural segment is 2. The preset triggerability judgment threshold is 1. When the judgment value is greater than this threshold, the corresponding structural segment is considered to be qualified for trigger point implantation. Thus, the value display segment is judged to be a triggerable structural segment. After determining that the structural segment is triggerable, historical data is further retrieved from the structural segment database to obtain the historical structural segment content features and trigger point configuration results corresponding to the value display segment type. The model is then trained by combining the context information of the current short video. The context information includes the industry to which the short video belongs, the content style tag of "rational explanation type", and the target audience feature profile of "price-sensitive consumers". Building upon this foundation, a recurrent neural network is used to learn the relationship between the content features, type labels, contextual information, and actual detonation effects of historical structural segments, constructing a detonation point configuration parameter prediction model. When processing the current structural segment in real time, the segment's content features, type labels, and corresponding contextual information are input into the trained detonation point configuration parameter prediction model. This model determines that the segment is suitable for using "price comparison type" as the detonation factor, with a medium detonation level, and the detonation expression method is "directly stating preferential information." It also suggests that the detonation point content be implanted in the middle to later time interval of the segment.

[0030] Step S105: Based on the generated short video script, load the customer's voiceprint model file for speech synthesis, generate a speech timestamp alignment vector based on the synthesized speech, and perform synchronous rendering and insert visual supplementary materials through a digital human model to generate short video content.

[0031] Based on the generated short video script, the speech synthesis module loads the client's voiceprint model file, sets the speech rate range, pitch characteristics, and emotion parameters, and performs text-to-speech synthesis. A speech timestamp alignment vector is generated from the synthesized speech, using the start and end times of each sentence as subsequent rendering annotation nodes. The client's digital human model is loaded, and the set character appearance, action style, and camera expression template information are read, bound to the speech synthesis output, and a synchronous rendering process is executed. Based on the structural tag matrix, predefined visual supplementary materials, including product shots, environmental transitions, emotional backgrounds, and dynamic subtitles, are inserted during each structural transition. The generated short video content undergoes multi-channel compression, image enhancement, and audio noise reduction post-processing according to structural tags, and is uniformly packaged into a short video publishing format supported by the target platform. The short video content is sent to the bound content distribution platform through the publishing interface, recording metadata at the time of publication, including structural information, short video identifier, and push status markers.

[0032] For example, a short video titled "How to Improve Work Efficiency" is being produced. The video is 30 seconds long and includes several structural segments: an opening introduction, a value demonstration, an emotional progression, and a call to action. The script for the short video has already been generated, and the next step is to synthesize the video content and begin post-production. In this process, the speech synthesis module is first invoked, and the voiceprint model file provided by the client is loaded to ensure that the speech synthesis accurately reflects the client's brand voice characteristics. To ensure naturalness and clarity, the speech rate is set to a range of 150-180 words per minute, ensuring it is neither too fast nor too slow for the audience. A slightly higher-than-normal pitch is also used to make the speech clearer and more motivating. Furthermore, to enhance emotional expression, the emotion parameter is set to 0.7, giving the speech content a certain emotional fluctuation and increasing its motivational effect. The speech synthesis module converts the text into speech based on these parameters and generates a speech timestamp alignment vector, preparing for subsequent rendering and video synchronization. By loading the client's digital human model, the system retrieves the character's appearance, movement style, and camera expression template information. The character's appearance is that of a mature and composed professional man; the movement style features natural gestures and timely facial expressions; and the camera expression template adjusts the camera angle according to different content segments, with a full-body shot at the beginning and close-up facial shots during the value demonstration segment. During subsequent rendering, the coordination between voice and digital human movements was simultaneously adjusted to ensure a perfect match between lip movements and speech content, and natural and coordinated body language. Visually, corresponding supplementary visual materials were inserted based on the content and emotional changes of each structural segment. For example, in the opening introduction segment, a simple office scene background is used, with dynamic subtitles "Welcome Everyone" gradually appearing from the bottom of the screen, and the camera smoothly focusing on the host's full body, helping the audience better understand the video's theme. Upon entering the value demonstration segment, the camera rapidly switches between shots, showcasing PPT content on "time management skills," while also incorporating product shots, such as displaying calendars and schedules, to help the audience more intuitively understand the presented content. To enhance the emotional progression of the video, an emotional background was used throughout the emotional development segment. The background color gradually changed from light gray to dark blue, matching the narrator's increasingly passionate tone and allowing the audience to feel the escalating emotion. After all content rendering was completed, post-processing began. First, the video underwent multi-channel compression to ensure smooth playback on multiple platforms. To improve video quality, image enhancement was performed, increasing color contrast and making the picture more vivid. Audio noise reduction was also applied, removing some background noise and making the speech clearer and easier to understand. Through these optimizations, the video was designed to provide the best viewing experience both visually and aurally. Finally, after the video content was completed, the generated short videos were uniformly packaged and, according to the requirements of the target platforms, packaged into MP4 format.Each structural segment and timestamp, each visual element, and audio information in the video is recorded in the video's metadata, including structural information, video identifiers, and push status markers. Through the publishing interface, the final video content is sent to the target content distribution platform, and the video's publishing status is recorded to ensure smooth promotion and display.

[0033] Step S106: Obtain short video behavior data by calling the data feedback interface of the content distribution platform, match structural segments and behavior events using a timeline alignment mechanism, calculate the response probability of structural segments, evaluate the contribution of content features to the behavior data, and adjust the video control vector.

[0034] By calling the data feedback interface of the content distribution platform, behavioral data of short videos within a preset time window is obtained, including play count, completion rate, interaction count, forwarding frequency, comment semantic vector, collection rate, and content popularity gradient. The acquired behavioral data is categorized by short video ID and platform identifier using a data aggregation module, establishing a unified data structure table containing timestamps, behavioral event types, behavioral intensity values, and user interaction paths. A timeline alignment mechanism is used to match the time range of each structural segment in the structural tag matrix with the occurrence time of behavioral events, constructing a mapping relationship between structural segments and behavioral events. An event trigger probability model is used to calculate the response probability of each structural segment on different types of behavioral events, forming a structural segment behavioral response matrix. This determines the correlation between each structural segment and the behavioral event; structural segments with a correlation higher than a preset correlation threshold are marked as behavior-driven structures, while those with a correlation lower than the preset correlation threshold are marked as low-impact structures. Based on the type labels, content features, and response metrics of each structural segment, a multiple linear regression algorithm is used for model training. The contribution of each content feature to the behavioral data, along with response metrics such as like rate, comment trigger rate, collection rate, and completion rate, is evaluated based on the R² and P-values ​​of the model fit. If the correlation between a content feature and a behavioral metric is lower than a preset correlation threshold, that content feature is marked as pending update. Based on the marking results, a gradient correction model is constructed using backpropagation to adjust the video control vector input to the large language model, and its mapping relationship is stored in the short video generation database.

[0035] For example, a short video is published to a content distribution platform, and audience behavior data is acquired within a preset time window, such as 48 hours, after the video's publication. This behavior data includes video views, completion rate, number of interactions, frequency of reposts, comment semantic vectors, collection rate, and content popularity gradient. By calling the content distribution platform's data feedback interface, these key data points can be obtained. For instance, within 48 hours, the video had 1 million views, an 85% completion rate, 5,000 comments, and was reposted 1,000 times. Comment analysis shows that the comment semantic vectors indicate a predominantly positive viewer sentiment, the collection rate is 10%, and the content popularity gradient shows a slight decrease in popularity in the subsequent period. The acquired behavior data is categorized using a data aggregation module, recording a timestamp, behavior event type, behavior intensity value, and user interaction path for each video event. The data shows that views rise rapidly in the first minute of the video, the completion rate peaks at 30 seconds, and comment activity increases in the final segment. Based on this data, a unified format data structure table is established. A timeline-based alignment mechanism is employed to match the time range of each structural segment with the occurrence time of behavioral events. If the video is divided into four structural segments—the opening segment, the value demonstration segment, the emotional progression segment, and the call to action segment—these segments are compared with the event times of audience behavior. For example, the value demonstration segment (seconds 5-15) is aligned with the comment surge period (seconds 10-15), establishing a mapping relationship between structural segments and behavioral events. Based on this, an event trigger probability model is used to calculate the response probability of each structural segment for different types of behavioral events. For instance, the value demonstration segment has a higher number of comments and likes, resulting in a behavioral response probability of 0.8, indicating that the content of this segment is more likely to stimulate audience participation. In contrast, the opening segment has a lower response probability of only 0.3, indicating less audience interaction in this segment. By analyzing the behavioral response matrix of each structural segment, the correlation between each segment and behavioral events was determined. For example, the comment trigger rate of the emotional progression segment was high, and the forwarding frequency of the behavioral call-to-action segment was high. The model calculated a correlation of 0.9, which is very high, indicating that these structural segments have a strong driving effect on user behavior. In contrast, the correlation of the opening introduction segment was only 0.4, indicating that its impact on user behavior was relatively small. To further optimize the video content, the type labels and content features of each structural segment, such as emotional fluctuations and visual jumps, were combined with behavioral response indicators. The model was trained using a multiple linear regression algorithm. If the model's R² value is 0.85, it indicates that the model can well explain the relationship between audience behavior data and video content features. The P-value is 0.01, indicating that the variables in the model, such as emotional fluctuations and visual jumps, have a significant impact on audience behavior. During the regression analysis, it was found that some content features, such as the emotional fluctuation value in the opening introduction segment, had a low correlation with the like rate and comment trigger rate, which was below the preset correlation threshold of 0.5.Therefore, these content features are marked as pending updates, meaning that the content of this segment needs adjustment, such as increasing the emotional fluctuation value or enhancing interactivity. Based on these marking results, a gradient correction model is constructed using a backpropagation mechanism to adjust the video control vector input to the large language model. These adjustments include enhancing emotional fluctuation and increasing keyword density, making the video more in line with user preferences and interaction needs. Finally, the adjusted video control vector is input into the large language model to generate a new video script. This script includes a more efficient structural arrangement and stronger emotional incentives, aiming to improve audience interaction and sharing rates. All these adjustment results are stored in the short video generation database to provide data support for future video content optimization.

[0036] like Figure 3 This embodiment of a customer mining system based on short videos may specifically include: The short video content acquisition module is used to obtain the public content index data and raw data stream of short videos through the short video platform interface.

[0037] The video structure segmentation module is used to generate structure segmentation segments based on the structure transition points in the short video time-series text, the shot change points in the image, and the audio energy jump points.

[0038] The structural segment type identification module is used to construct a structural segment type identification model based on the structural features of the structural segmentation sections, identify the type label of each structural segment, and generate a structural label matrix.

[0039] The script copy generation module is used to assign weights to each feature of each structural segment based on the content features of the structural segments and the context information of the short video, determine the video control vector, and generate short video script copy by combining the viral factor prompts.

[0040] The short video generation module is used to perform speech synthesis based on the generated short video script, load the customer's voiceprint model file, generate a speech timestamp alignment vector based on the synthesized speech, and perform synchronous rendering and insertion of visual supplementary materials through a digital human model to generate short video content.

[0041] The control vector adjustment module is used to obtain short video behavior data by calling the data feedback interface of the content distribution platform, match structural segments and behavior events using a timeline alignment mechanism, calculate the response probability of structural segments, evaluate the contribution of content features to the behavior data, and adjust the video control vector.

[0042] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the concept of this application. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A customer mining method based on short videos, characterized in that, The method includes: Obtain the public content index data and raw data stream of short videos through the short video platform interface; Based on the structural transition points in the time-series text, the abrupt change points in the image, and the energy jump points in the audio, structural segmentation segments are generated. The time-series text is obtained by extracting speech feature information from the short video through the audio parsing module and transcribing it. Through sentiment analysis and text grammar parsing tools, structural transition points in the time-series text are identified, the structural features of the structural segmentation segments are obtained, and stored in the structural segmentation segment database. Based on the structural features of the segmented sections, a structural segment type identification model is constructed to identify the type label of each structural segment and generate a structural label matrix. Based on the content features of each structural segment and the contextual information of the short video, weights are assigned to each feature of each structural segment to determine the video control vector. This vector is then combined with viral factor prompts to generate the short video script, including: By leveraging a structural segment database, historical data on the content features of structural segments and their type labels are obtained. Combined with the contextual information of the short videos, a recurrent neural network is used for model training to construct a content feature weight allocation model. Contextual information includes the video's industry category, video content style tags, and target audience profiles. Based on the structural segment content features provided by the user, the content feature weight allocation model assigns weights to each feature of each structural segment, encoding the content features and weights of each segment into sparse or dense vector representations, which are then concatenated into a video control vector. Based on the structural segment content features, the potential for each segment to become viral is assessed, and detonation point configuration parameters are determined. These parameters include detonation factor type selection, placement constraints, and detonation expression methods. By inputting the video control vector and preset viral factor prompts into a large language model, a short video script tailored to the client's needs is generated. Preset viral factors include visual hooks, stylistic parameters, expression tendency factors, colloquialism control variables, and detonation point configuration parameters. Based on the generated short video script, the client's voiceprint model file is loaded for speech synthesis, and a voice timestamp alignment vector is generated based on the synthesized speech. The digital human model is then used for synchronous rendering and insertion of visual supplementary materials to generate short video content. Short video behavior data is obtained by calling the data feedback interface of the content distribution platform. The time axis alignment mechanism is used to match structural segments and behavior events, calculate the response probability of structural segments, evaluate the contribution of content features to the behavior data, and adjust the video control vector. The response probability of structural segments is the response probability of each structural segment on different types of behavior events.

2. The method according to claim 1, wherein, The step of obtaining the public content index data and raw data stream of short videos through the short video platform interface includes: Based on the industry keywords and content tag groups set by the user, the system obtains the public content index data and raw data stream of short videos through the short video platform interface within a preset time window, and sorts the popularity of short videos within the preset time window. The public content index data includes short video cover features, short video content links, title text, comment summary, publication timestamp, number of views, interaction data and author identifier. Interaction data includes the number of likes, comments and reposts. The raw data stream includes audio stream, short video frame data and subtitle information.

3. The method according to claim 1, wherein, The process of generating structural segmentation segments based on structural transition points in temporal text, abrupt changes in images, and audio energy jump points includes: Frame-level image slicing is performed on the short video image content to obtain a sequence of visual information including character poses, shot transition frequency, background music (BGM) change nodes, and subtitle appearance positions. Image and audio information are time-axis aligned, and a spatiotemporal sequence graph is established based on frame-level positional mapping. Based on the short video frame data, the SSIM algorithm is used to compare the similarity between adjacent frames to detect shot transition points in the short video. Short-time Fourier transform is used to extract the audio's spectral information, and energy jump points in the audio are identified based on the root mean square energy (RMS). Based on structural transition points in the temporal text, combined with shot transition points in the image and audio energy jump points, structural segmentation segments are generated. The structural features of these segments are obtained and stored in a structural segmentation database. These structural features include keyword density, emotional fluctuation values, and visual jump degrees.

4. The method according to claim 1, wherein, The process involves constructing a structural segment type identification model based on the structural features of the segmented sections, identifying the type label of each structural segment, and generating a structural label matrix, including: Historical data on the structural features of segmented sections are obtained from the structural segment database. Each segment is labeled with a type tag, and a random forest algorithm is used to train the model to build a structural segment type recognition model. The type tags include introductory segments, value display segments, emotional progression segments, and behavioral call-to-action segments. Based on the real-time acquired structural features of segmented sections, the structural segment type recognition model is used to identify the type tag of each segment, generate a structural tag matrix, extract the content features of each segment in the structural tag matrix, and store them in the structural segment database. The content features include emotional fluctuation curves, visual transitions, keyword intent, and semantic context.

5. The method according to claim 1, wherein, The process of determining the detonability of each structural segment based on its content characteristics and identifying detonation point configuration parameters includes: Based on the content characteristics of structural segments, the detonability determination formula is used. Calculate the detonability determination value of the structural segment. ,in, The continuous semantic length of the structural segment represents the temporal length of the structural segment. To preset the minimum threshold for the semantic span that can be triggered, This represents the extreme value of emotion within the structural segment. The effective threshold for emotional fluctuation. This represents the number of visual changes within a structural segment. The minimum visual disturbance threshold, For symbolic functions, The function is a floor function; if the detonability judgment value is less than or equal to the preset judgment threshold, the generation of detonation point configuration parameters in the corresponding structural segment is prohibited; if the detonability judgment value is greater than the preset judgment threshold, the structural segment is determined to be detonable; through the structural segment database, historical data of structural segment content features and structural segment type labels are obtained, and combined with the context information of the structural segment, a recurrent neural network is used to train the model and construct a detonation point configuration parameter prediction model; based on the real-time obtained structural segment content features, type labels and context information, the detonation point configuration parameters of each structural segment are determined.

6. The method according to claim 1, wherein, The process involves loading a client's voiceprint model file for speech synthesis based on the generated short video script, generating a speech timestamp alignment vector based on the synthesized speech, and then simultaneously rendering and inserting visual supplementary materials using a digital human model to generate short video content, including: Based on the generated short video script, the speech synthesis module is called to load the customer's voiceprint model file, set the speech rate range, pitch characteristics and emotion parameter values, and perform text-to-speech synthesis; the synthesized speech generates a speech timestamp alignment vector, and the start and end time positions corresponding to each sentence of speech content are used as subsequent rendering annotation nodes; By loading the customer's digital human model, reading the set character appearance, action style and shot expression template information, binding the speech synthesis output and executing the synchronous rendering process; according to the structural tag matrix, inserting predefined visual supplementary materials, including product shots, environmental transitions, emotional backgrounds and dynamic subtitles, when switching between each structural segment; The generated short video content undergoes multi-channel compression, image enhancement, and audio noise reduction post-processing based on structural tags, and is uniformly packaged into a short video publishing format supported by the target platform. The short video content is then sent to the bound content distribution platform through the publishing interface, recording the metadata at the time of publication, including structural information, short video identifier, and push status flag.

7. The method according to claim 1, wherein, The process involves obtaining short video behavior data through the data feedback interface of the content distribution platform, matching structural segments with behavior events using a timeline alignment mechanism, calculating the response probability of structural segments, evaluating the contribution of content features to the behavior data, and adjusting the video control vector, including: By calling the data feedback interface of the content distribution platform, behavioral data of short videos within a preset time window is obtained, including play count, completion rate, number of interactions, repost frequency, comment semantic vector, collection rate, and content popularity gradient. The acquired behavioral data is categorized by short video ID and platform identifier using a data aggregation module, establishing a unified data structure table containing timestamps, behavioral event types, behavioral intensity values, and user interaction paths. A timeline alignment mechanism is used to match the time range of each structural segment in the structural tag matrix with the occurrence time of behavioral events, constructing a mapping relationship between structural segments and behavioral events. An event trigger probability model is used to calculate the response probability of each structural segment on different types of behavioral events, forming a structural segment behavioral response matrix to determine the relationship between each structural segment and the behavior. The relevance between events is assessed, with structural segments having a relevance higher than a preset relevance threshold labeled as behavior-driven structures and those lower than the threshold labeled as low-impact structures. Based on the type labels, content features, and response metrics of each structural segment, a multiple linear regression algorithm is used for model training. The contribution of each content feature to the behavioral data is evaluated based on the R² and P-values ​​of the model fit, along with response metrics such as like rate, comment trigger rate, collection rate, and completion rate. If the relevance between a content feature and a behavioral metric is lower than the preset relevance threshold, that content feature is marked as pending update. Based on the marking results, a gradient correction model is constructed using backpropagation to adjust the video control vector input to the large language model, and its mapping relationship is stored in the short video generation database.

8. A customer mining system based on short videos, implemented based on a customer mining method based on short videos as described in any one of claims 1-7, characterized in that, The system includes the following modules: The short video content acquisition module is used to acquire the public content index data and raw data stream of short videos through the short video platform interface; The video structure segmentation module is used to generate structure segmentation segments based on structure transition points in time-series text, shot change points in images, and audio energy jump points. The structural segment type identification module is used to construct a structural segment type identification model based on the structural features of the structural segmentation sections, identify the type label of each structural segment, and generate a structural label matrix. The script copy generation module is used to assign weights to each feature of each structural segment based on the content features of the structural segments and the context information of the short video, determine the video control vector, and generate short video script copy by combining the viral factor prompt words. The short video generation module is used to load the customer's voiceprint model file for speech synthesis based on the generated short video script, generate a speech timestamp alignment vector based on the synthesized speech, and generate short video content through synchronous rendering and insertion of visual supplementary materials using a digital human model. The control vector adjustment module is used to obtain short video behavior data by calling the data feedback interface of the content distribution platform, match structural segments and behavior events using a timeline alignment mechanism, calculate the response probability of structural segments, evaluate the contribution of content features to the behavior data, and adjust the video control vector.

Images