A multi-modal feature quantization extraction system for all ages of human beings

Abstract

This invention discloses a multimodal feature quantification and extraction system for all age groups of humans, relating to the fields of artificial intelligence and digital life modeling. It includes a multimodal feature extraction module for all age groups, a high-fidelity extraction module for small samples, a lightweight feature data compression module, a digital personality data format definition module, and a data collection boundary control module. It collects multimodal features such as facial expressions, micro-expressions, voice, and body language in a tiered manner according to the dimensions of elderly, middle-aged, youth, and children, with specific optimizations for the extraction logic of the elderly and children groups; lightweight compression is achieved based on edge computing power, without dependence on cloud computing power; a custom digital personality basic data format is used to achieve data ownership confirmation and cross-platform interoperability; and human-specific feature collection boundaries are defined, with lossless stripping of auxiliary fields throughout the entire process. This invention achieves multimodal feature extraction for all age groups, lightweight edge processing, standardized packaging, and compliant control, adapting to the needs of civilian application of digital life and full life cycle modeling.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and digital life modeling, and in particular to a multimodal feature quantification extraction system for all ages of humans. Background Technology

[0002] Digital human feature extraction is a core pre-processing technology for digital life modeling and virtual human interaction. It is mainly used to collect multimodal features of human faces, voices, limbs, behaviors, etc., to provide basic data for subsequent digital personality modeling, rendering, and interaction. It is a key link to achieve the retention and accurate restoration of human digital life throughout its entire life cycle.

[0003] Current digital human feature extraction technologies suffer from several common technical shortcomings: large data collection requirements, cumbersome collection processes, and long pre-modeling preparation cycles; low personality feature fidelity, lack of age-stratified feature extraction mechanisms, absence of specific optimization logic for special age groups such as the elderly and children, and uneven distribution of feature recognition across all age groups; lack of high-fidelity extraction capability for small samples, resulting in high collection thresholds and hindering civilian application; poor compatibility with edge devices, lack of lightweight compression solutions for feature data, heavy reliance on cloud computing power, and low efficiency and high latency in edge reading and processing; and the absence of a unified digital personality data format standard, resulting in poor cross-platform compatibility, significant adaptation losses with subsequent AIGC modeling systems, and an inability to meet the core data collection needs for preserving the entire lifecycle of human digital life.

[0004] The existing publicly available patent technologies have not yet provided an effective solution to the above problems: Patent CN113886778A discloses a method for generating a dynamic self-sovereign identity that can be supervised: it does not construct a hierarchical feature extraction system for all age groups of humans, does not optimize feature extraction logic for the elderly and children, and has extremely low feature recognition in special age groups; it adopts a massive labeled data collection mode, lacks the ability to extract high-fidelity small samples, and has high data collection costs and low efficiency; it does not design a lightweight compression module on the edge, and feature data processing relies entirely on cloud computing power, which cannot be adapted to low-end and mid-range edge devices, resulting in significant data processing delays.

[0005] The published patent CN116051278A discloses a method for the creation, authentication, use, and transaction of digital humans based on blockchain: it does not define a standardized basic data format for digital personality, has poor cross-platform data interoperability, and suffers significant adaptation loss when interfacing with the AIGC lightweight modeling system; it does not clarify the boundaries for collecting human-specific features, and non-human features are prone to interfering with core extraction calculations, and there is no intelligent stripping mechanism for auxiliary fields; the data preprocessing efficiency on the edge is low, and it cannot meet the needs of civilian and lightweight implementation.

[0006] In summary, existing digital human feature extraction technologies and comparison files all suffer from shortcomings such as insufficient adaptability across all age groups, lack of small sample capacity, poor lightweight design on mobile devices, and lack of format standards. Therefore, a multimodal feature quantification extraction system for all age groups of humans is needed to support the civilian, full life cycle, and mobile application of human digital life. Summary of the Invention

[0007] Purpose of the invention: This invention provides a multimodal feature quantification extraction system for all age groups of humans. It solves the technical problems of insufficient all-age group adaptation, high-fidelity extraction of small samples, lightweight compression on the device side, standardized data format definition, and precise control of the collection boundary in the existing technology through five major technical means: hierarchical collection of all age groups, high collection threshold, reliance on the cloud on the device side, lack of unified data standards, and ambiguity of collection boundaries.

[0008] Technical solution: To solve the above-mentioned technical problems, according to one aspect of the present invention, more specifically, a multimodal feature quantification and extraction system for all age groups of humans, including a multimodal feature extraction module for all age groups, a high-fidelity extraction module for small samples, a lightweight feature data compression module, a digital personality data format definition module, and a data acquisition boundary control module.

[0009] All-age-group hierarchical multimodal feature extraction module This module constructs a hierarchical multimodal feature extraction system for all age groups of humans. It collects features in layers according to four age dimensions: elderly, middle-aged, youth, and children. The multimodal features collected simultaneously include eight categories: facial features, micro-expressions, eye movement patterns, voice tone, speech rate and rhythm, dialect / expression habits, body posture, and lifestyle preferences. The feature extraction logic is specifically optimized for the feature distribution characteristics of the elderly and children, improving the recognition of features in special age groups. The module's output data format is standardized and adapted to the digital personality modeling process, and the output data format is compatible with the ONNX and PyTorch frameworks, providing core data support for the accurate reconstruction of human digital life.

[0010] Small sample high-fidelity extraction module This module optimizes the high-fidelity extraction algorithm for small human samples. Through feature dimension reduction and key feature enhancement training, it enables high-fidelity extraction of human full-dimensional and multimodal features with only a small amount of voice data and multi-scene image data. This effectively lowers the data collection threshold and meets the needs of civilian lightweight application. The algorithm has no sample adaptation bias during operation and is compatible with people of all ages and dialects.

[0011] Feature data lightweight compression module This module designs a lightweight compression logic for human feature data. Based on the computing power characteristics of various end-device devices such as mobile phones, tablets, and smart screens, it completes targeted quantization and compression processing while ensuring that the accuracy of feature restoration is not lost. The compressed data can be quickly read and processed on all types of consumer-grade end-device devices such as mobile phones, tablets, and smart screens. The module does not rely on cloud computing power during operation, and at the same time optimizes the end-device reading speed of compressed data to reduce data processing latency.

[0012] Digital Personality Data Format Definition Module This module defines and adopts a custom digital personality basic data format standard, specifically DPBDFV1.0. This standard includes metadata fields, encryption headers, and unique identification bits. Data encryption protection is achieved through the encryption header, data ownership is confirmed through the unique identification bits, and data standardization and encapsulation are completed in combination with metadata fields. The data format is compatible with the ONNX framework and the PyTorch framework, enabling interoperable transmission of feature data between different platforms.

[0013] Data Acquisition Boundary Control Module This module clearly defines the boundaries of human characteristic data collection, limiting all collected data to human-specific characteristics. The module has no independent logic for collecting non-human characteristics. Non-human related content exists only as ancillary fields of human characteristics, accounting for ≤1.5%, and these ancillary fields do not participate in the core feature extraction calculation. The module is equipped with a full-link intelligent stripping interface for ancillary fields. This interface is a fully integrated interface that can perform lossless stripping of ancillary fields at any stage of data collection, processing, and storage. It can achieve lossless one-click deletion of ancillary fields without changing the parameters, accuracy, or integrity of the core human characteristic data, and without affecting the integrity and accuracy of the human characteristic data.

[0014] Beneficial effects: A hierarchical feature extraction system covering all age groups was constructed, and the feature extraction logic was specifically optimized for the elderly and children to achieve balanced collection and accurate identification of multimodal features across all age groups. It achieves high-fidelity extraction with small samples, requiring only a small amount of voice and images to complete full-dimensional feature collection, greatly reducing the threshold for civilian collection and adapting to mass consumer scenarios; It adopts lightweight compression technology on the device side, eliminating the dependence on cloud computing power and adapting to all types of consumer devices such as mobile phones, tablets, and smart screens, thereby improving the efficiency of data processing on the device side. Defines the DPBDFV1.0 standardized data format, with data encryption and ownership verification capabilities, enabling cross-platform data interoperability and seamless integration with the AIGC modeling system; It accurately defines the boundaries of data collection, has no independent non-human feature collection logic, and allows for lossless stripping of auxiliary fields throughout the entire process, ensuring the purity of core features and data compliance. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of the present invention. Detailed Implementation

[0016] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Reference Figure 1 As shown, a multimodal feature quantification and extraction system for all ages of humans includes a multimodal feature extraction module for all ages, a high-fidelity extraction module for small samples, a lightweight feature data compression module, a digital personality data format definition module, and a data acquisition boundary control module.

[0018] Example 1

[0019] A multimodal feature quantification and extraction system for all age groups of humans, using civilian mobile devices as the operating platform, completes the entire feature extraction process for young users. The specific implementation steps are as follows: Step 1: Run the multimodal feature extraction module for all age groups After a user launches the system on their mobile device, the system first automatically identifies the user's age as youth and loads the youth-specific feature extraction parameters. Simultaneously, feature extraction logic for the elderly and children remains in the background, ready to adapt to switching needs across all age groups. The system then synchronously initiates a multimodal feature acquisition process, which specifically collects the following: Facial features: facial contours, proportions of facial features, and skin texture characteristics; Micro-expressions: subtle facial features such as blinking, raising eyebrows, and the curvature of the corners of the mouth; Eye movement patterns: eye movement trajectory, focal point of fixation, and saccade speed characteristics; Speech intonation: the characteristics of voice timbre, pitch, and intonation. Speech rate and rhythm: the speed of speech, pauses, and rhythmic characteristics of sentence transitions; Dialect / Expression Habits: Local language pronunciation, colloquial vocabulary, and logical expression characteristics; Body posture: head posture, shoulder posture, trunk posture characteristics; Lifestyle behavioral preferences: Interaction habits and operational behavior preferences.

[0020] After data collection, the module standardizes the characteristics of the youth dimension, and the output data is compatible with the ONNX and PyTorch frameworks, completing the standardized adaptation with the digital personality modeling process and providing basic data for subsequent digital life modeling.

[0021] Step 2: Running the small sample high-fidelity extraction module Following system prompts, users record 5 minutes of everyday conversational voice data (natural dialogue, without deliberate reading) and upload 10 images from different scenes, including 6 static facial images and 4 dynamic expression images, covering multiple angles such as front, left, right, head up, and head down. The system then activates a small-sample high-fidelity extraction algorithm. First, it performs feature dimension reduction processing on the voice and image data, eliminating redundant and invalid data. Next, it performs key feature enhancement training on core personality traits, improving the extraction accuracy of core features such as facial features, voice, and behavior. The algorithm operates without sample adaptation bias throughout, seamlessly adapting to all age groups and dialect groups. Finally, based on the aforementioned small sample size, it completes high-fidelity extraction of the user's full-dimensional multimodal features, meeting the low-threshold data collection needs for civilian-use, lightweight deployment.

[0022] Step 3: The feature data lightweight compression module runs. After acquiring high-fidelity feature data, the module performs targeted quantization and compression processing based on the computing power and hardware characteristics of the mobile device, ensuring no loss of feature restoration accuracy. The compressed data can be quickly read and processed on all types of consumer-grade mobile devices, including mobile phones, tablets, and smart screens. The module operates without cloud computing power dependence; all data processing is completed locally on the device. It also optimizes data reading speed, reduces on-device data processing latency, and is compatible with mid-to-low-end devices.

[0023] Step 4: Run the digital personality data format definition module The module encapsulates the compressed feature data according to the DPBDFV1.0 digital personality basic data format standard. The encapsulated content includes, in sequence: Metadata fields: Record basic information such as user age, collection time, device model, and feature type; Encryption header: Encrypts the entire data to prevent data leakage and tampering; Unique identifier: Marks data ownership and clarifies the data owner.

[0024] The encapsulated data is compatible with both the ONNX and PyTorch frameworks, enabling seamless interoperability between different platforms and devices, and lossless integration with the digital personality modeling process.

[0025] Step 5: Data Acquisition Boundary Control Module Operation The system performs boundary verification on the packaged data, strictly limiting the retention to human-specific features and eliminating any independent non-human feature collection logic. During verification, a small number of non-human related auxiliary fields (such as background environment and irrelevant object information) are identified. The system automatically controls the proportion of auxiliary fields to ≤1.5% and prohibits them from participating in core feature extraction calculations. The module uses an intelligent end-to-end auxiliary field stripping interface to perform lossless one-click deletion of auxiliary fields during data storage. After deletion, the parameters, accuracy, and integrity of the user's core human feature data remain unchanged, ensuring the accuracy of subsequent digital life modeling and guaranteeing the compliance and purity of data collection.

[0026] Example 2

[0027] The difference between this embodiment and Embodiment 1 is that the operating platform is a smart large-screen device, the user age dimension is children, the system switches to children's dimension feature extraction parameters, and loads a customized optimization strategy specifically for children, making special optimizations for features such as children's facial proportions, voice rhythm, and expression habits to improve the recognition of children's features. The remaining collection process, small sample input, lightweight compression, DPBDFV1.0 standard encapsulation, and auxiliary field stripping are completely consistent with Embodiment 1. The system can stably adapt to all age dimensions, including the elderly, middle-aged, young adults, and children, meeting the feature extraction needs of all age groups.

[0028] Example 3

[0029] The difference between this embodiment and Embodiment 1 is that the user's age dimension is elderly. The system loads a feature extraction optimization strategy specifically for the elderly, making special optimizations for features such as skin texture, voice tone, and body posture to improve the recognition accuracy of elderly features. When users input dialect voice data, the system has no sample adaptation bias and can complete the feature extraction of dialect groups normally. The entire process is processed on the client side without cloud dependence. After the lossless stripping of auxiliary fields, the data can be directly synchronized to the AIGC modeling system, achieving standardized, lightweight, and compliant operation throughout the entire process.

[0030] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A multimodal feature quantification and extraction system for all age groups of humans, comprising a multimodal feature extraction module for all age groups, a high-fidelity extraction module for small samples, a lightweight feature data compression module, a digital personality data format definition module, and a data acquisition boundary control module, characterized in that: The full-age-range hierarchical multimodal feature extraction module is used to construct a human full-age-range hierarchical multimodal feature extraction system. It collects features in layers according to four age dimensions: elderly, middle-aged, youth, and children. It simultaneously collects multimodal features such as facial features, micro-expressions, eye movement patterns, voice tone, speech rate and rhythm, dialect / expression habits, body posture, and lifestyle preferences. It performs special optimization of feature extraction logic for the feature distribution characteristics of the elderly and children. It achieves accurate feature matching and positioning through skeletal topology detection and timestamp anchor alignment, improving the feature recognition of special age groups. The module's output data format is standardized and adapted to the digital personality modeling process, and the output data format is compatible with the ONNX framework and PyTorch framework, providing core data support for the accurate restoration of human digital life. The small sample high-fidelity extraction module is used to optimize the human small sample high-fidelity extraction algorithm. Through feature dimension reduction processing, residual fitting transfer learning, and key feature reinforcement training, it achieves high-fidelity extraction of full-dimensional multimodal features under low sample input, effectively reducing the data collection threshold and adapting to the needs of civilian lightweight implementation. The algorithm has no sample adaptation bias during operation and can be compatible with people of all ages and people speaking different dialects. The feature data lightweight compression module is used to design lightweight compression logic for human feature data. Based on the computing power characteristics of the edge device, it completes targeted quantization compression processing while ensuring that the feature restoration accuracy is not lost. It optimizes data reading and transmission efficiency through edge quantization encoding and hierarchical feature quantization control. The compressed data supports fast reading and processing on multiple types of edge devices. The module does not rely on cloud computing power during operation, reducing data processing latency. The digital personality data format definition module is used to define and adopt a custom digital personality basic data format standard to realize data ownership and cross-platform interoperability. The data format is compatible with the ONNX framework and PyTorch framework, enabling the interoperability and transmission of feature data between different platforms. The data acquisition boundary control module is used to define the boundaries of human feature data acquisition, limiting all acquired data to human-specific features. The module has no independent non-human feature acquisition logic. Non-human related content exists only as ancillary fields of human features, and the ancillary fields do not participate in the core feature extraction calculation. The module is equipped with an intelligent stripping interface for the entire chain of ancillary fields, which supports lossless one-click deletion of ancillary fields. Deleting ancillary fields does not affect the integrity and accuracy of human feature data.

2. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, In the full-age-group hierarchical multimodal feature extraction module, the four age dimensions of elderly, middle-aged, youth and children are fixed hierarchical dimensions. The module optimizes the feature extraction logic for the elderly and children groups, and is a customized optimization strategy to improve the feature recognition of the elderly and children groups.

3. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, The multimodal feature extraction module for all age groups synchronously collects multimodal features, which are a complete combination of multiple features such as facial features, micro-expressions, eye movement patterns, voice tone, speech rate and rhythm, dialect / expression habits, body posture, and lifestyle preferences, with no missing features.

4. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, The feature dimension reduction, residual fitting transfer learning, and key feature enhancement training of the small sample high-fidelity extraction module are the core processing steps of the algorithm, which are used to achieve high-fidelity extraction of full-dimensional features under small sample input.

5. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, The targeted quantization and compression processing of the feature data lightweight compression module is adapted based on the computing power characteristics of the end device, and the compression process does not lose the feature restoration accuracy.

6. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, The feature data lightweight compression module is compatible with various types of end-side devices, including mobile phones, tablets, and smart large-screen consumer-grade end-side devices.

7. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, The digital personality data format definition module uses DPBDFV1.0 as its basic data format standard. It achieves data encryption protection through an encryption header, identifies the data subject through a unique identity verification identifier, and encapsulates the data format by combining metadata fields.

8. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, In the data acquisition boundary control module, the intelligent stripping interface for the ancillary fields is a fully integrated interface that can perform lossless stripping of ancillary fields at any stage of data acquisition, processing, and storage.

9. The multimodal feature quantization extraction system for all age groups of humans according to claim 1, characterized in that, In the data acquisition boundary control module, the lossless one-click deletion operation of auxiliary fields does not change the parameters, accuracy, or integrity of the core human characteristic data.

Images