A method and system for family audio-video imprint aggregation and biographical film generation based on bloodline narration

Abstract

The application discloses a kind of family audio-video imprint automatic aggregation and personal biography film generation method and system based on genealogy figure. First, structured family knowledge base including family member node, blood relationship edge and associated multimedia data is constructed. In response to biography generation request, start multi-modal figure imprint aggregation: fusion face recognition and blood relationship auxiliary matching, automatically identify and extract all image fragments of target figure appearance from audio-video library, form cross-modal material set. Further, blood narrative guided intelligent editing is carried out: based on age estimation, material time sequence segmentation is carried out, narrative structure is automatically generated in combination with blood event node, and material enhancement technology is used to output into film. The system automatically generates the "family imprint" column in personal data card, and aggregates and displays related audio-video materials. Through integrated path, scattered family memories are gathered into personal image biography, which greatly reduces the production threshold, realizes the automatic aggregation and generation of family memories.

Description

Technical Field

[0001] This invention relates to the fields of digital humanities, multimedia information management, and artificial intelligence, and in particular to a system and method for automatically aggregating audio-visual traces centered on family tree figures and automatically generating biographical films by utilizing facial recognition, bloodline reasoning, intelligent editing, and multimodal content generation technologies. Background Technology

[0002] Genealogies are important carriers of family lineage and cultural memory. With the development of digital technology, traditional paper genealogies are gradually evolving towards electronic and multimedia formats. However, existing digital genealogy products still have the following technical shortcomings: First, the event- or timeline-centric organizational approach lacks the ability to deeply aggregate information at the individual level. Existing digital genealogy software primarily uses a family tree structure to display member relationships or a timeline to show event sequences. When a user wants to access all video materials of a particular family member, they must manually browse through numerous albums and search one by one, which is inefficient and prone to omissions. The system lacks the ability to automatically aggregate cross-modal materials with "people" as the core index, and it cannot automatically present all related audio-visual traces in a person's personal profile card.

[0003] Secondly, there is a lack of automated generation capabilities from scattered materials into complete narrative films. Current technology, even when able to aggregate photos and video clips of a particular person, only displays a list of materials and cannot automatically generate a biographical film with a narrative logic. Users who wish to create commemorative films still need to manually import content into editing software, manually arrange the time sequence, and manually add subtitles and background music—a process with high barriers to entry and significant time consumption.

[0004] Third, facial recognition technology lacks accuracy across age groups in family settings. General facial recognition models face unique challenges in family video scenarios: people in historical photos and videos may span decades, with significant changes in appearance; old materials may have blurry image quality and poor angles; and family members may share similar appearances, easily leading to misidentification. Existing technologies lack optimization methods specifically for family scenarios.

[0005] Fourth, the narrative logic of biographical films is disconnected from family bloodline events. Existing intelligent video editing tools' "one-click production" function relies on visual aesthetics or simple scene categorization, lacking the semantic understanding of family bloodline events (such as birth, marriage, and the birth of children). While the generated content may be visually appealing, its narrative logic is detached from the actual life milestones of family members, making it difficult to convey the emotional depth of family memories.

[0006] In summary, constructing a system and method that can automatically aggregate family audio-visual traces using family genealogy figures as an index and generate personal biographical films based on bloodline events has become a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0007] I. Purpose of the Invention The purpose of this invention is to provide a method and system for automatically aggregating family audio-visual imprints based on family genealogy figures and generating personal biographical videos, in order to solve the technical problems in the prior art such as the lack of audio-visual aggregation capabilities of digital genealogies based on the individual dimension, the high threshold for producing personal biographical videos, the insufficient accuracy of cross-age facial recognition, and the disconnect between the editing narrative logic and family bloodline events.

[0008] II. Technical Solution In a first aspect, embodiments of the present invention provide a method for automatically aggregating family audio-visual imprints based on genealogical figures and generating personal biographical videos, including the following steps: Step S1: Build a deeply structured family knowledge base.

[0009] A structured family knowledge base is constructed, comprising family member nodes, bloodline edges, timeline information, and associated multimedia data. This knowledge base further achieves deep structured memory modeling by constructing a four-dimensional family knowledge graph. Identity feature dimension: Stores personalized information such as facial feature vectors, voiceprint features, and personality tags of family members. Among them, facial feature vectors are stored separately according to age group to form multi-age feature trajectories.

[0010] Blood relation dimension: Records the type of blood relationship between members (such as father and son, mother and daughter, grandparent and grandchild, etc.) and the weight of intimacy.

[0011] Audiovisual spatiotemporal dimension: Using timeline and location as indexes, it stores metadata of multimedia assets, including shooting time, location, character tags, scene tags, and emotion tags.

[0012] Memory event dimension: Record major family events (birth, marriage, migration, celebration, etc.) and their participating members, emotional tags, and related multimedia.

[0013] Step S2: Respond to the biography generation request and initiate multimodal character imprint aggregation.

[0014] In response to a user's request to generate a biographical video for a target person, the system executes a multimodal person imprint aggregation process: 2.1 Target Person Feature Extraction: Based on the target person's identity information, extract their facial feature vectors (multiple age groups) and a list of blood relatives from the family knowledge graph.

[0015] 2.2 Cross-modal material recognition and extraction: A face recognition model is used to detect faces and compare features in keyframes of photos and videos from a family video library, identifying all image materials containing the target person. For video materials, segment-level segmentation is performed by combining face temporal detection and scene transition detection to extract continuous video segments containing the target person.

[0016] The audio segments of a target person's speech are detected using voiceprint recognition technology and then associated with the transcribed text.

[0017] 2.3 Bloodline-Assisted Matching: When the confidence level of the target person in historical materials is lower than the preset threshold due to factors such as a large age range or blurry image quality, the bloodline-assisted matching mechanism is activated. Identify other family members whose identities have been confirmed in low-confidence material; Obtain the type of blood relationship between the family members and the target person, and the expected facial similarity weights; Adjust the confidence threshold for determining the identity of the target person based on the weight of blood relationship; When the adjusted overall confidence level exceeds the preset threshold, the material is automatically labeled as containing the target person.

[0018] 2.4 Material Aggregation and Deduplication: The identified photos, video clips, and audio clips are sorted and deduplicated according to the timeline to form a cross-modal material collection indexed by the target person.

[0019] Step S3: Intelligent editing guided by bloodline narrative.

[0020] Intelligent editing that guides the aggregated video clips through a bloodline narrative, generating an editing timeline: 3.1 Age estimation and time segmentation: Based on the birth date of the target person and the shooting time of the material, the aggregated material is divided into life stages such as childhood, adolescence, youth, middle age, and old age according to age group.

[0021] 3.2 Bloodline event extraction: Retrieve bloodline event nodes related to the target person from the family knowledge graph, including nodal events such as birth, birthday, school enrollment, graduation, marriage, birth of children, birth of grandchildren, and important anniversaries.

[0022] 3.3 Narrative Structure Generation: Taking chronological segmentation as the main line, blood-related events are embedded as narrative anchors to automatically generate a narrative structure that includes life experiences, and a corresponding editing timeline is generated for each narrative segment.

[0023] 3.4 Selection of Best Footage: When multiple candidate footage exists within the same narrative segment, a weighted score is calculated based on multiple dimensions such as the emotional tags, clarity, proportion of characters, and importance of events, and the best footage is selected to fill the editing timeline.

[0024] Step S4: Material enhancement and video compositing.

[0025] Perform enhancement processing and compositing rendering on the footage on the editing timeline: 4.1 Restoration and colorization of old materials: Intelligent colorization processing based on deep learning for black and white or faded photos; resolution enhancement, noise reduction, jitter reduction, and frame interpolation smoothing for old videos; noise reduction enhancement and volume equalization for old audio.

[0026] 4.2 Automatic Subtitle Generation: The speech of characters in the film is transcribed using speech recognition to generate synchronized subtitles; the identities of characters in the film are identified, and the speaker's name is marked in the subtitles.

[0027] 4.3 Background audio synthesis: Based on the emotional tone of the film's narrative segments, matching background music is selected from the music library and seamlessly spliced ​​and volume gradually adjusted.

[0028] 4.4 Rendering Output: Render the biographical video of the target person.

[0029] Step S5: Manual verification and model feedback.

[0030] The system provides a visual interface for users to review the character recognition results and adjust the editing timeline, and records user operations as training data for model feedback, which is used to optimize the face recognition model and the material selection model.

[0031] In addition, this invention also supports generating family memoirs according to the time dimension or scene type in response to the user's selected generation mode, so as to meet the diverse needs of family memory presentation.

[0032] Step S6: Automatic generation and display of family imprints.

[0033] The system automatically generates a "Family Imprint" section in each family member's personal profile card. This section is implemented in the following way: 6.1 Automatic Aggregation: Based on the multimodal person imprint aggregation results of step S2, all photos, video clips, and audio recordings related to the member are automatically arranged in chronological order.

[0034] 6.2 Categorized Display: Users can filter by material type (photos / videos / audio) or browse by timeline.

[0035] 6.3 One-click generation entry: Users can directly click the "Generate Biographical Video" button in the Family Imprint section. The system will start the intelligent editing and finalization process of steps S3 to S4 based on the currently aggregated materials.

[0036] 6.4 Dynamic Updates: When new materials are added to the family video library, the system automatically identifies and updates the family imprint of that member in real time, without requiring manual operation by the user.

[0037] With this feature, users can view all relevant audio and video materials of a target person in one place in their personal profile card without having to search through the family audio and video library one by one, achieving a "see it as soon as you open" dimension of memory presentation.

[0038] Secondly, embodiments of the present invention provide a system for automatically aggregating family audio-visual imprints based on genealogical figures and generating personal biographical videos, including: The family knowledge base management module is used to build and maintain a structured family knowledge base; The Person Imprint Aggregation Engine is used to integrate facial recognition technology with blood relationship-assisted matching to automatically identify and extract all video clips of the target person from the family video library. The bloodline narrative editing module is used to segment aggregated materials chronologically and generate a narrative structure by combining bloodline event nodes in the family knowledge graph, and output the editing timeline. The video compositing and rendering module is used to enhance the footage on the editing timeline and render and output a personal biographical video. The manual proofreading interaction module provides a visual interface for users to review the recognition results, adjust the editing timeline, and record user operations as training data for the model. A multi-mode generation module is used to support switching between biographical mode and time-based memoir mode for generation.

[0039] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0040] III. Beneficial Technical Effects Compared with the prior art, the present invention has the following beneficial technical effects: For the first time, it has achieved automatic aggregation of cross-modal audio-visual traces centered on family tree figures: photos, video clips, and audio materials scattered in the family audio-visual library are automatically gathered into a complete trace set indexed by a certain person, solving the pain point of users manually searching for all images of a certain member in a large amount of material.

[0041] Achieve a one-stop presentation of "family imprints": The family imprint section is automatically generated in the personal profile card of each family member, allowing users to view all relevant audio and video materials of that member without having to search through them, improving search efficiency by more than 90%.

[0042] Significantly lowers the barrier to entry for creating personal biographical videos: Users do not need professional editing skills. The system automatically completes the entire process of material aggregation, narrative arrangement, repair and enhancement, and subtitle and music synthesis, reducing the video production work that originally took hours or even days to just minutes.

[0043] Significantly improves the accuracy of cross-age facial recognition in family scenarios: By using a blood relationship-assisted reasoning mechanism and combining kinship information in the family knowledge graph, the accuracy of facial recognition across age groups and in blurry image quality materials is improved by 25-35%.

[0044] Achieving intelligent editing with bloodline narrative logic: Unlike existing editing tools that are based on visual aesthetics, the film generated by this invention takes life stages as the main line and bloodline events as the narrative anchor, giving the film a complete life narrative structure and profound emotional connotation.

[0045] Activate dormant family audiovisual assets: Through an automated generation mechanism, transform a large number of old photos and videos lying dormant on hard drives and in albums into personal biographical films that are easy to spread, share and pass on.

[0046] Establish a continuously optimized learning loop: Through human proofreading and feedback training mechanisms, the system can gradually adapt to users' aesthetic preferences and narrative habits, and the automation effect continues to improve with the number of uses.

[0047] Supports multiple generation modes: In addition to the biographical mode, it also supports generating family memoirs by time dimension or scene type, meeting users' diverse needs for presenting family memories. Attached Figure Description

[0048] Figure 1 The system architecture diagram provided for embodiments of the present invention; Figure 2 A flowchart of multimodal person imprint aggregation provided in an embodiment of the present invention; Figure 3 A flowchart of blood relationship-assisted face recognition matching provided in an embodiment of the present invention; Figure 4 A flowchart of intelligent editing guided by bloodline narrative provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the manual proofreading and feedback training interactive interface provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the user interface for the personal data card and family imprint function provided in an embodiment of the present invention; Figure 7 An example image showing the result of generating a personal biographical video as provided in an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] Example 1: Construction of Family Knowledge Base and Initialization of Four-Dimensional Graph This embodiment describes in detail the process of constructing a family knowledge base, taking a family with three generations of members as an example.

[0051] 1.1 Structured Collection of Family Basic Data Users enter basic family information through the graphical interface provided by the system: Member information entry: Grandfather: Zhang Deshan, born March 12, 1925, in Xinxiang City, Henan Province, passed away July 20, 1998. Grandmother: Li Xiuying, born on August 5, 1928, in Xinxiang City, Henan Province, passed away on November 3, 2015. Father: Zhang Jianguo, born on June 18, 1955, in Xinxiang City, Henan Province. Mother: Wang Shufang, born on February 10, 1958, in Luoyang City, Henan Province. Son: Zhang Ming (the user), born on April 22, 1985, in Zhengzhou, Henan Province. Daughter: Zhang Li, born on November 5, 1990, in Zhengzhou, Henan Province. Establishment of blood relations: Zhang Deshan and Li Xiuying: Husband and Wife Zhang Deshan and Zhang Jianguo: Father and Son Li Xiuying and Zhang Jianguo: Mother and Son Zhang Jianguo and Wang Shufang: Husband and Wife Zhang Jianguo and Zhang Ming: Father and Son Zhang Jianguo and Zhang Li: Father and Daughter Wang Shufang and Zhang Ming: Mother and Son Wang Shufang and Zhang Li: Mother and daughter Timeline framework creation: The system automatically creates a personal timeline for each member, marking important time nodes (birth, enrollment, graduation, marriage, employment, retirement, etc.), which users can supplement and improve according to their actual situation.

[0052] 1.2 Intelligent processing of historical multimedia data Users uploaded approximately 1,500 files containing family history multimedia assets, including old black and white photos, color photos, videos transcribed from family videotapes, videos shot with mobile phones, and audio files.

[0053] The system initiates a multimodal parallel processing pipeline: Visual content analysis: The MTCNN face detection model is used to locate face regions in photos and video frames; the ArcFace face recognition model is used to extract 512-dimensional facial feature vectors; the scene classification model is used to identify the scene type of the photo; and the sentiment analysis model is used to estimate the emotional tone of the photo.

[0054] Time information processing: Extract the shooting time from the EXIF ​​information of the photo; for old photos without time information, estimate the time interval by combining the age of the people, clothing characteristics, and scene characteristics.

[0055] Audio content processing: The WeNet speech recognition model is used to transcribe spoken recordings into text; the ECAPA-TDNN voiceprint model is used to extract speaker voiceprint features; and a sentiment analysis model is used to identify sentiment tags in the audio.

[0056] 1.3 Construction of a Four-Dimensional Family Knowledge Graph Based on the above processing results, the system constructs a four-dimensional family knowledge graph: Identity Feature Dimension: Create an identity node for each family member, storing facial feature vectors, voiceprint feature vectors, personality tags, language styles, etc., stored separately for each age group.

[0057] Bloodline relationship dimension: Establish bloodline relationship edges between member nodes and store relationship type, intimacy weight, and relationship description.

[0058] Audiovisual spatiotemporal dimension: Create resource nodes for each multimedia asset, storing shooting time, shooting location, character tags, scene tags, emotional tags, and resource path.

[0059] Event Dimension: Create event nodes to store event type, event time, event location, participating members, emotional tone, and associated multimedia.

[0060] 1.4 Establishment of Atlas Index The system establishes multi-level indexes to support efficient retrieval: FAISS indexes are created for facial feature vectors and scene feature vectors; B+ tree indexes are created by time dimension; a bloodline adjacency table is created with member ID as the key; and composite indexes are created by event type, time, and participating members.

[0061] Example 2: Multimodal Person Imprint Aggregation (Core Example) This embodiment demonstrates in detail the multimodal person imprint aggregation process using the target person "grandfather Zhang Deshan" as an example, and focuses on explaining the face recognition matching mechanism based on blood relationship assistance.

[0062] 2.1 Starting the Aggregation Task Users select the target person "Zhang Deshan (grandfather)" through the system interface, click the "Generate Personal Biographical Video" button, and the system starts the multimodal person imprint aggregation engine.

[0063] 2.2 Target Character Feature Extraction The system extracts the identity characteristics of grandfather Zhang Deshan from the family knowledge base: Facial feature vectors: extract features for three age groups (youth 20-40 years old, middle age 40-60 years old, and old age 60-73 years old). Voiceprint feature vector: extracted from oral recordings left by grandfather List of blood relatives: Obtain the grandfather's direct relatives (wife Li Xiuying, son Zhang Jianguo, grandson Zhang Ming, granddaughter Zhang Li, etc.). 2.3 Parallel Recognition of Cross-Modal Materials The system performs face detection and recognition on keyframes of photos and videos in the family's audio-visual library. For video footage that identifies the grandfather, the system combines face temporal detection and scene transition detection to perform segment-level segmentation and extract independent video clips containing the grandfather.

[0064] Voiceprint recognition technology was used to detect segments of the grandfather's voice in the recordings and to transcribe each audio segment.

[0065] 2.4 Blood Relationship-Assisted Matching (Core Innovative Process) When processing a batch of black-and-white family photos taken in the 1970s, the system encountered a recognition problem: A group photo taken in 1975 includes the grandfather Zhang Deshan (50 years old), the grandmother Li Xiuying (47 years old), the father Zhang Jianguo (20 years old), and others. Due to the age of the photo, its blurry quality, and the shadow obscuring the grandfather's face, the facial recognition model's confidence level for recognizing the grandfather was only 0.52 (below the high confidence threshold of 0.8).

[0066] The system activates a blood relation-assisted matching mechanism: Identifying other members: The system successfully identified the grandmother Li Xiuying (confidence 0.89) and the father Zhang Jianguo (confidence 0.91) in the photo, both of whom were identified with high confidence.

[0067] Obtaining blood relations: By querying the family knowledge graph, it was found that Li Xiuying's relationship with her grandfather is "husband and wife", and Zhang Jianguo's relationship with his grandfather is "father and son".

[0068] Calculate auxiliary weights: Marital relationship: Expected weight of facial similarity 0.85 Father-son relationship: Expected weight of facial similarity 0.90 Overall auxiliary weight = 1 - (1-0.85)×(1-0.90) = 0.985 Confidence adjustment: Original confidence level: 0.52 Adjusted confidence level = 0.52 + (1-0.52) × 0.6 × 0.985 ≈ 0.80 (Note: 0.6 is the auxiliary weight influence factor, which can be configured.) Automatic labeling: After adjustment, the overall confidence level is 0.80, which exceeds the preset threshold of 0.75. The system automatically labels the photo as "contains grandfather Zhang Deshan" and records the supporting reasoning evidence.

[0069] Through this mechanism, 17 out of the original 20 low-confidence black and white old photos were successfully labeled automatically, reducing the workload of manual review by 85%.

[0070] 2.5 Material Aggregation and Deduplication The system aggregates and removes duplicates from all identified materials, sorts them by timeline, and forms a complete timeline of the grandfather Zhang Deshan's life, covering approximately 50 years of his life from his youth to his later years.

[0071] Example 3: Intelligent Editing and Film Generation Guided by Bloodline Narrative This embodiment demonstrates in detail the process of intelligent editing and personal biographical video generation based on aggregated materials.

[0072] 3.1 Age estimation and time series segmentation Based on the grandfather Zhang Deshan's birth date (March 12, 1925) and the filming time, the system categorizes the footage into life stages: childhood (0-12 years old), adolescence (13-18 years old), youth (19-35 years old), middle age (36-55 years old), and old age (56-73 years old). The system selects representative footage for each stage, prioritizing footage with high clarity, natural facial expressions, and typical characteristics of the era.

[0073] 3.2 Bloodline Event Extraction The system retrieves bloodline events related to the grandfather Zhang Deshan from the family knowledge graph, including marriage (1950), birth of children (1955), starting work (1952), children's marriage (1980), birth of grandchildren (1985, 1990), and death (1998).

[0074] 3.3 Narrative Structure Generation The system automatically generates the narrative structure for the biographical film about Zhang Deshan, the grandfather: Prologue: Tracing Family Lineage (Ages 0-18): Family Origins, Childhood, and Education Growth Chapter: Youth (Ages 19-35): Starting Work, Meeting and Falling in Love with Grandmother, Getting Married and Starting a Family Establishing a Career: The Prime of Life (Ages 36-55): Participating in major infrastructure projects, career development, and family life. Inheritance Chapter: Continuing Family Traditions (Ages 56-73): Children's weddings, grandchildren's births, enjoying grandchildren's company The Later Years: Echoes of Time (Old Age): Retirement Life, Interactions with Grandchildren, Oral History Epilogue: Eternal Remembrance: News of Passing, Remembrance by Descendants, Family Legacy 3.4 Material Selection and Timeline Filling The system assigns a weighted score to the materials within each narrative segment and selects the best for inclusion. Weighted scoring model configuration ("Warm Remembrance" style): Emotional tag weight: 0.35 (prioritize heartwarming and joyful content) Sharpness weight: 0.25 (prioritize high-resolution footage) Character weighting: 0.15 (prioritizing footage with clear images of the grandfather's face) Event importance weight: 0.15 (bonus points for materials directly related to the bloodline event) Weight of blood relation: 0.10 (bonus points for footage featuring the subject in the same frame as immediate family members) 3.5 Material Enhancement and Video Compositing Restoration and colorization of old materials: intelligent colorization of old black and white photos; resolution enhancement and shake reduction of old home videos; noise reduction and enhancement of old dictation recordings.

[0075] Automatic subtitle generation: Generate synchronized subtitles for dialogue and narration in the film, identify the speaker in the subtitles, and generate descriptive subtitles for important events.

[0076] Background audio synthesis: Based on the emotional tone of each narrative segment, matching background music is selected from the music library and seamlessly spliced ​​together with volume gradient processing.

[0077] 3.6 Rendering Output The system renders and outputs a biographical video of Zhang Deshan's grandfather, approximately 13 minutes long, in 1080p resolution, in MP4 format. The video includes an opening title, visual separation of narrative chapters, and an ending credits. Users can preview the video through the system interface and can also enter a manual proofreading mode for fine-tuning.

[0078] Example 4: Manual Proofreading and Continuous Learning Loop This example demonstrates the manual proofreading mechanism and how the system is continuously optimized based on user feedback.

[0079] 4.1 Person Recognition Proofreading The system generates a list of materials to be confirmed, displaying materials with low recognition confidence for user review. Users can confirm, correct, or remove the recognition results. The system records the user's corrections, removes the material from the original person's imprint, and adds it to the imprint of the correct person.

[0080] 4.2 Editing Timeline Proofreading Users can access the visual editing interface and adjust the generated editing timeline, such as adjusting photo display duration, changing the order of footage, and changing background music. The system records user adjustments, including preferences for photo duration, footage order, and music.

[0081] 4.3 Feedback Training Data Generation The system converts the user's proofreading operations into training data: the user-confirmed recognition results are used as positive samples, and the corrected results are used as negative samples; the user's adjusted editing preferences are used as preference labels.

[0082] 4.4 Incremental Model Update After accumulating a certain amount of character recognition, proofreading, and editing adjustments, the system employs efficient parameter fine-tuning technology to perform lightweight updates to the recognition and optimization models. The updated models are more aligned with the characteristics of the family and the user's aesthetic preferences in subsequent material recognition. Through a continuous learning loop, the system's automation accuracy steadily improves with repeated use.

[0083] Example 5: Demonstration of Personal Data Card and Family Imprint Functions

[0084] This embodiment demonstrates the specific presentation effect of the present invention at the user interaction level, focusing on the implementation method of the "family imprint" function.

[0085] 5.1 Personal Profile Card Interface When a user logs into the system to view a family member, the system displays that member's profile card, which includes the following information: Column Content Basic Information: Name, Date of Birth and Death, Place of Birth, Blood Relationship A character profile automatically generated by the system (based on aggregating bloodline events). Key life milestones on the timeline (birth, marriage, birth of children, etc.) Automatically aggregated list of all audio and video materials related to the family traces and this member Biographical video generated / pending personal biographical video 5.2 Implementation flow of the family imprint function The family imprint function is implemented based on the multimodal person imprint aggregation results in step S2: [New material added to the family video library] ↓ [Multimodal people footprint aggregation engine automatically recognizes] ↓ [Associate the recognition results with the corresponding member nodes] ↓ [This member's family trace list is updated in real time] ↓ [Viewed by the user in their profile card] 5.3 Family Imprint Display Interface The Family Footprints section is organized and displayed in the following manner: Timeline view: All footage is arranged chronologically by shooting time, forming the member's life timeline. Category View: Users can filter by category, such as video, audio, or clips. Thumbnail preview: Hover your mouse over the image to preview the content. One-click generation entry: Click the "Generate Biographical Video" button to directly enter the intelligent editing process. 5.4 Technical Effects With the "Family Imprint" feature, users can view all relevant audio-visual materials of a target person in one place on their personal profile card without having to search through the family's audio-visual library one by one. Compared with existing digital family tree products (which require manually browsing albums or using search functions), this invention improves search efficiency by more than 90%, achieving a "see-it-instant" memory presentation of the person's information.

[0086] 5.5 Dynamic Update Mechanism When new material is added to the family video library, the system automatically triggers the following process: Perform multimodal person imprint aggregation and recognition on the new material. Associate the recognition results with the corresponding member nodes. The member's family traces list is updated in real time. Users can see the newly added materials by refreshing the page. This mechanism ensures the real-time nature and accuracy of family imprints, eliminating the need for manual maintenance by users.

[0087] Example 6: Sample of Personal Biographical Video Generation Results

[0088] This example demonstrates the final generated biographical video effect.

[0089] 6.1 Film Structure Taking Zhang Deshan, the grandfather, as an example, the generated biographical film contains the following structure: Chapter content duration and core materials Prologue: Tracing Family Lineage (1 minute 30 seconds) - Childhood photos, footage of the family's old house The coming-of-age segment, spanning 2 minutes and 30 seconds, includes photos of young people, work IDs, and audio recordings. Establishing a Career: A 3-Minute Journey of Struggle - Construction Site Photos, Certificates of Honor, and Interview Videos Passing on Family Traditions: 3-minute family portrait, wedding video, and photos of grandchildren. The Twilight Years: A 2-minute video recording of life in old age and an audio recording of oral history. The epilogue, a one-minute remembrance, includes a portrait, a family tree, and commemorative captions. 6.2 Video Effects Image quality: Old photos undergo intelligent color enhancement, and videos are enhanced in resolution, resulting in consistent overall image quality. Subtitles: Automatically generate synchronized subtitles and identify the speaker. Background music: The background music is automatically matched according to the emotional tone of each chapter and seamlessly blended together. Total duration: Approximately 13 minutes, 1080p resolution 6.3 User Interaction Users can view the video after it is generated: Online preview playback Enter manual proofreading mode to adjust the order of materials, trimming time, and change music. Export as MP4 format and share to social media platforms or save to your local device.

Claims

1. A method for automatically aggregating family audio-visual imprints based on genealogical figures and generating personal biographical videos, characterized in that, include: Construct and maintain a structured family knowledge base containing family member nodes, blood relationship edges, and associated multimedia data; In response to a request to generate a biographical video for a target person, multimodal character imprint aggregation is performed: by integrating facial recognition technology with blood relationship-assisted matching, all video clips of the target person appearing in the family video library are automatically identified and extracted, forming a cross-modal material set indexed by the person. Intelligent editing guided by bloodline narrative for aggregated video clips: Based on the age estimation model of the target person, the material is segmented chronologically, and the narrative structure is automatically generated by combining bloodline event nodes related to the target person in the family knowledge graph, and the editing timeline is output. Enhance the footage on the editing timeline and render it to output a biographical video of the target person.

2. The method according to claim 1, characterized in that, The structured family knowledge base achieves deep structured memory modeling by constructing a family knowledge graph in four dimensions: identity features, blood relations, audio-visual time and space, and memory events.

3. The method according to claim 1, characterized in that, The multimodal person imprint aggregation includes: Extract facial feature vectors and a list of blood relatives of the target person based on their identity information; A facial recognition model is used to identify all image materials and video clips of the target person appearing in the family's audio-visual library; Detecting audio segments of a target person's speech using voiceprint recognition technology; Based on a blood relationship-assisted matching mechanism, identity completion and confirmation are performed on materials whose identification confidence level is below the threshold. All identified image segments are time-aligned and deduplicated to form a cross-modal material collection.

4. The method according to claim 3, characterized in that, The blood relation-assisted matching mechanism includes: Identify other family members whose identities have been confirmed in low-confidence material; Obtain the blood relationship type between the family members and the target person; Adjust the confidence threshold for identifying the target person based on the expected facial similarity weight in blood relations; When the adjusted overall confidence level exceeds the preset threshold, the material is automatically labeled as containing the target person.

5. The method according to claim 1, characterized in that, The bloodline-narrative-guided intelligent editing includes: Based on the target person's birth date and the estimated shooting time of the footage, the aggregated footage is divided into multiple life stages according to age groups; Retrieve bloodline event nodes related to the target person from the family knowledge graph and use them as narrative anchors; Using chronological segmentation as the main thread, bloodline events are embedded to generate a narrative structure that includes life experiences, and a corresponding editing timeline is generated for each narrative segment. When multiple candidate clips exist within the same narrative segment, the optimal clip is selected based on a weighted scoring model to fill the editing timeline.

6. The method according to claim 1, characterized in that, The enhancement process includes restoring and colorizing old materials, automatically generating subtitles, and synthesizing background audio. Before rendering output, the system also includes: providing a visual interface for users to review character recognition results and adjust the editing timeline, and recording user operations as training data for model feedback.

7. The method according to claim 1, characterized in that, Also includes: Responding to the user's selected generation mode, it supports generating family memoirs by time dimension or scene type.

8. A system for automatically aggregating family audio-visual imprints based on genealogical figures and generating personal biographical videos, characterized in that, include: The family knowledge base management module is used to build and maintain a structured family knowledge base; The Person Imprint Aggregation Engine is used to integrate facial recognition technology with blood relationship-assisted matching to automatically identify and extract all video clips of the target person from the family video library. The bloodline narrative editing module is used to segment aggregated materials chronologically and generate a narrative structure by combining bloodline event nodes in the family knowledge graph, and output the editing timeline. The video compositing and rendering module is used to enhance the footage on the editing timeline and render and output a personal biographical video.

9. The system according to claim 8, characterized in that, Also includes: The manual proofreading interaction module provides a visual interface for users to review the recognition results, adjust the editing timeline, and record user operations as training data for the model. A multi-mode generation module is used to support switching between biographical mode and time-based memoir mode for generation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images