A multimodal fusion strategy adaptive multi-user automatic identification method and device

Abstract

The application discloses a multi-modal fusion strategy adaptive multi-user automatic identification method and device, and belongs to the technical field of human-computer interaction and biometric recognition. User voice streams, facial image frames and interactive behaviors containing environmental physical quantities are collected, voiceprint, facial and behavior rhythm features are extracted, local multi-user templates are matched, and each mode candidate user and confidence are output. According to the environmental physical quantity, the fusion weight is dynamically adjusted, the comprehensive score is obtained by weighted summation, the target user is determined by the highest score, and the identity mark is output in combination with the threshold value. In response to the mark, personalized strategies are called, and terminal parameters are switched atomically to execute subsequent responses. The application solves the problems that a single mode is easy to be disturbed and a fixed strategy is poor in adaptability, and realizes collaborative improvement of recognition accuracy and personalization.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and biometric recognition technology, and in particular to a multimodal fusion strategy adaptive multi-user automatic recognition method, apparatus, device and storage medium. Background Technology

[0002] Currently, multi-user automatic identification technology mainly relies on single-modal biometric identification methods, with voiceprint recognition being the most common. Existing voiceprint recognition systems collect users' speech stream data, extract basic acoustic parameters such as wavelength, frequency, intensity, and rhythm, and compare them with pre-stored templates in a registration database to determine identity. This type of method is simple to implement and has low computational overhead, thus it is widely used in many terminal devices. However, its inherent limitations are also significant, making it difficult to meet the requirements for recognition accuracy and robustness in complex application scenarios.

[0003] First, single-modal recognition schemes are highly dependent on voiceprint quality. When environmental noise is high or channel interference exists, the signal-to-noise ratio of the speech signal drops sharply, leading to voiceprint feature distortion, a significant decrease in recognition rate, and insufficient system reliability. Second, existing voiceprint feature extraction methods mostly remain at the shallow physical parameter level, failing to fully utilize advanced technologies such as deep neural networks to mine more discriminative high-order semantic features. This results in limited ability to distinguish between similar users and a high risk of misjudgment. Third, the definition of "usage mode" in existing systems is too narrow, usually involving only basic operations such as loading contacts or switching desktop applications. It does not correlate the identity recognition results with the systematic adaptive adjustment of the overall system interaction strategy, lacking a personalized response mechanism for different user habits. In addition, existing solutions lack effective fallback recognition strategies. When a user's voiceprint becomes unavailable due to changes in voice, a quiet environment, or device malfunction, the system completely loses its ability to determine identity and cannot provide a backup recognition channel, severely impacting user experience and system availability.

[0004] In view of this, there is an urgent need for a multi-user identification method that integrates multimodal information, has dynamic adaptive adjustment capabilities, and includes a redundant identification mechanism, in order to overcome the shortcomings of the existing technologies and improve the accuracy of identity recognition and the level of personalized adaptation of interaction strategies in complex environments. Summary of the Invention

[0005] The present invention aims to at least partially solve one of the technical problems in the related art.

[0006] To address this issue, this invention proposes an adaptive multi-user automatic identification method using a multimodal fusion strategy. This method collects user voice streams, facial image frames, and interactive behavior data containing environmental physical quantities. Voiceprint, facial, and behavioral prosodic features are extracted and matched in parallel with a local registration database, outputting candidate users for each modality and their confidence scores. The fusion weights are dynamically adjusted based on environmental physical quantities, and a weighted sum is obtained to obtain a comprehensive score. The highest score determines the target user, and an identity identifier is output based on a threshold. In response to the identifier, a personalized strategy is invoked, and terminal parameters are atomically switched. This invention solves the problems of susceptibility to interference in single-modal scenarios and poor adaptability of fixed fusion strategies, achieving a synergistic improvement in accurate identification and personalized interaction.

[0007] Another objective of this invention is to propose a multimodal fusion strategy adaptive multi-user automatic identification device.

[0008] The third objective of this invention is to provide a computer device.

[0009] The fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0010] To achieve the above objectives, this invention proposes a multimodal fusion strategy adaptive multi-user automatic identification method, comprising: Collect the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities. Extract voiceprint features, facial features, and behavioral prosodic features from the three types of data respectively. Match each feature with the multi-user template in the local registration library and output the candidate users and the confidence scores of each modality in parallel. The fusion weights of each modality are dynamically adjusted based on environmental physical quantities. The adjusted weights are then used to perform a weighted summation of the confidence scores of each candidate user to calculate the comprehensive fusion score. The candidate user with the highest comprehensive fusion score is identified as the target user, and the user identity identifier is output based on the comparison result of the comprehensive fusion score and the preset threshold. In response to the user's identity identifier, the corresponding personalized interaction strategy data is retrieved from the strategy configuration library, and the terminal's interaction parameters are atomically switched to the target parameters corresponding to the personalized interaction strategy data in order to execute subsequent interaction responses.

[0011] The multimodal fusion strategy adaptive multi-user automatic identification method of this invention may also have the following additional technical features: In one embodiment of the present invention, the acquisition of the current user's voice stream data, facial image frame data, and interactive behavior data including environmental physical quantities includes: User voice is collected by a microphone array consisting of at least two microphones, analog-to-digital conversion is performed at a preset sampling rate to generate voice stream data, and ambient sound pressure level data is collected simultaneously as part of the environmental physical quantity. The system simultaneously captures user facial images using an RGB camera and an infrared camera, and uses a time-of-flight sensor to obtain distance data between the user and the terminal, generating facial image frame data. The system logs record the user interaction times and the types of functions called, generating time logs and content logs in the interaction behavior data, and outputting ambient sound pressure level data and distance data as components of environmental physical quantities.

[0012] In one embodiment of the present invention, voiceprint features, facial features, and behavioral prosodic features are extracted from speech stream data, facial image frame data, and interactive behavior data, respectively, including: Pre-emphasis, framing, and windowing are performed on the speech stream data. The FBank acoustic feature matrix of each frame of speech is extracted. The FBank acoustic feature matrix is ​​input into the voiceprint embedding model based on the ECAPA-TDNN architecture. After multi-scale feature aggregation and channel attention weighting, a 192-dimensional voiceprint embedding vector is output as the voiceprint feature. Face detection is performed on facial image frame data, and alignment is performed based on the detected facial key points. The aligned face region is then cropped and scaled to a preset pixel size, input into a face embedding model based on the MobileFaceNet architecture, and outputs a 128-dimensional face embedding vector as facial features. The time feature of the current interaction is extracted from the time log of the interaction behavior data, the function category feature of the current call is extracted from the content log, the estimated height of the current user is extracted from the distance data, and the speech rate and fundamental frequency value of the current interaction are extracted from the speech stream data. Together, they form a behavioral prosodic statistical vector as behavioral prosodic features.

[0013] In one embodiment of the present invention, each feature is matched with a multi-user template in a local registry, and candidate users corresponding to each modality and their corresponding confidence scores are output in parallel, including: Using voiceprint features as query vectors, the first cosine similarity between the corresponding query vector and the voiceprint template vector of each registered user stored in the local registration database is calculated. Registered users whose maximum first cosine similarity exceeds the first preset threshold are identified as voiceprint candidate users, and the maximum first cosine similarity is normalized and used as the voiceprint confidence score output. Using facial features as query vectors, the second cosine similarity between the corresponding query vector and the facial template vectors of each registered user stored in the local registration database is calculated. Registered users whose maximum second cosine similarity exceeds the second preset threshold are identified as face candidate users, and the maximum second cosine similarity is normalized and used as the face confidence score output. Using behavioral prosodic statistical vectors as query data, the time-segment similarity, content similarity, height-distance similarity, and prosodic similarity between the corresponding query data and the historical behavioral baselines of each registered user stored in the local registry are calculated. After weighted fusion of the similarities in each dimension, the behavioral candidate users and the corresponding behavioral confidence scores of the candidate users are determined and output.

[0014] In one embodiment of the present invention, dynamically adjusting the fusion weights of each mode based on environmental physical quantities includes: The environmental sound pressure level data in the environmental physical quantities are analyzed. When the environmental noise exceeds the first decibel threshold, the basic fusion weight of the voiceprint modality is reduced from the first weight value to the second weight value, while the basic fusion weight of the face modality is increased from the third weight value to the fourth weight value. The illumination data in the environmental physical quantities are analyzed. When the ambient illumination is lower than the first illuminance threshold or when it is determined from the facial image frame data that the user is not facing the camera, the basic fusion weight of the face modality is lowered to the fifth weight value, while the basic fusion weight of the voiceprint modality is raised to the sixth weight value. The voiceprint modality and face modality are detected separately to determine whether they are available. When both the voiceprint modality and face modality are determined to be unavailable, the fusion weight of the behavioral modality is configured as the seventh weight value, and the behavioral candidate user and the behavioral confidence of the behavioral candidate user are used as the sole criterion for judgment.

[0015] In one embodiment of the present invention, a user identity identifier is output based on a comparison between the comprehensive fusion score and a preset threshold, including: When the overall fusion score is greater than or equal to the high confidence threshold, the target user is determined to be a known user, and the corresponding registered identity identifier of the target user and the corresponding interaction strategy index are output. When the overall fusion score is greater than or equal to the low confidence threshold and less than the high confidence threshold, the corresponding target user is identified as a user to be confirmed, and a tentative identity identifier and a general interaction strategy index are output. When the overall fusion score is less than the low confidence threshold, the current user is judged as an unknown user, and a new user registration guidance instruction is output as the user's identity identifier.

[0016] In one embodiment of the present invention, atomically switching the terminal's interaction parameters to the target parameters corresponding to the personalized interaction strategy data includes: Before switching execution, the dialogue context data and unfinished task status of the previous user are saved to a temporary cache, and the safe switching boundary point of the current interaction output is obtained; When the safe switching boundary point is reached, the personalized interaction policy data of the target user is read from the policy configuration library. Based on the speech synthesis parameters, interface display parameters and function permission parameters in the personalized interaction policy data, the speech rate and timbre configuration of the terminal speech synthesis engine, the font and theme configuration of the interface rendering engine and the access permission list of the function scheduling module are updated respectively. After the switch is completed, based on the task inheritance identifier in the personalized interaction strategy data, it is decided to resume the execution of the previous user's unfinished tasks, transfer the unfinished tasks to the target user's account, or permanently suspend the unfinished tasks.

[0017] To achieve the above objectives, another aspect of the present invention proposes a multimodal fusion strategy adaptive multi-user automatic identification device, comprising: The perception and recognition module is used to collect the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities. It extracts voiceprint features, facial features, and behavioral prosodic features from the three types of data, matches each feature with multi-user templates in the local registration library, and outputs candidate users and their confidence scores in parallel for each modality. The fusion scoring module is used to dynamically adjust the fusion weights of each modality based on environmental physical quantities, and to calculate the comprehensive fusion score of each candidate user by weighted summation of the confidence scores using the adjusted weights. The adjudication output module is used to identify the candidate user with the highest comprehensive fusion score as the target user, and output the user identity identifier based on the comparison result of the comprehensive fusion score and the preset threshold. The strategy switching module is used to retrieve the corresponding personalized interaction strategy data from the strategy configuration library in response to the user's identity identifier, and atomically switch the terminal's interaction parameters to the target parameters corresponding to the personalized interaction strategy data in order to execute subsequent interaction responses.

[0018] This invention discloses a multimodal fusion strategy-based adaptive multi-user automatic identification method and apparatus. By fusing three modal features—voiceprint, face, and behavioral prosody—and dynamically adjusting the fusion weights of each modality in conjunction with environmental physical quantities, it effectively solves the problems of poor anti-interference capability of single-modal recognition in complex environments and insufficient adaptability of fixed fusion strategies. Through hierarchical comparison of the comprehensive fusion score with preset thresholds, it outputs multi-level identity identifiers for known users, users awaiting confirmation, or unknown users, and atomically switches personalized interaction strategies and task inheritance processing, realizing an integrated closed-loop process from multimodal perception, dynamic fusion, hierarchical adjudication to strategy linkage. This invention significantly improves the robustness and accuracy of identity recognition in multi-user scenarios, enhances the method's adaptability in dynamic environments such as lighting and noise, and has significant engineering practical value in terms of personalized switching of interaction strategies and system security.

[0019] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing a multimodal fusion strategy adaptive multi-user automatic identification method as described in the first aspect embodiment.

[0020] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a multimodal fusion strategy adaptive multi-user automatic identification method as described in the first aspect embodiment.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a multimodal fusion strategy adaptive multi-user automatic identification method according to an embodiment of the present invention; Figure 2 This is a black and white block diagram of the overall system architecture according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a multimodal fusion strategy adaptive multi-user automatic identification device according to an embodiment of the present invention; Figure 4 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] The following description, with reference to the accompanying drawings, describes an adaptive multi-user automatic identification method, apparatus, device, and storage medium based on a multimodal fusion strategy according to embodiments of the present invention.

[0026] Example 1 like Figure 1 As shown, the multimodal fusion strategy adaptive multi-user automatic identification method of the present invention includes the following steps: S1 collects the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities. It extracts voiceprint features, facial features, and behavioral prosodic features from the three types of data, matches each feature with the multi-user templates in the local registration library, and outputs the candidate users corresponding to each modality and the confidence scores of the candidate users in parallel.

[0027] In one embodiment of the present invention, the ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation in Time Delay Neural Network) architecture is used to extract voiceprint features. Key advantages include: (1) Multi-scale feature aggregation – extracting speech features at different time scales through the SE-Res2Net module; (2) Channel attention mechanism – automatically learning the importance weights of different feature channels; (3) Statistical pooling – combining mean and standard deviation statistical information to preserve global speech features.

[0028] It should be noted that during the registration phase: 30-60 seconds of voice data are collected at a sampling rate of 16kHz, with pre-emphasis (coefficient 0.97), framing (frame length 25ms, frame shift 10ms), and windowing (Hamming window). 80-dimensional FBank features are extracted and input into the ECAPA-TDNN model, resulting in a 192-dimensional voiceprint embedding vector. The mean of the embedding vectors from multiple collections is calculated, and a voiceprint template is generated and stored in a local database. Liveness detection distinguishes between live speech and recorded attacks by analyzing the high-frequency energy distribution of the speech (recordings typically show significant energy attenuation in the >8kHz frequency band) and temporal dynamic features.

[0029] It should be noted that in the voiceprint registration interaction process: When a user uses the system for the first time, the system will initiate the "Add User" process via screen or voice prompts. First, the guardian, administrator, or user must confirm the user name and user type. Then, a random number sequence, short phrases, and free-response questions will be played via voice prompts. Voice samples are ideally collected in 6-10 segments, each 5-8 seconds long, for a total effective voice duration of 30-60 seconds. After each segment is collected, voice activity detection, signal-to-noise ratio (SNR) detection, and duplicate content detection are performed. Only qualified segments are written to the registration cache. If the SNR of a sample is below 10dB, the effective voice is less than 3 seconds, or it does not match the prompt text, the user is prompted to reread. If three consecutive attempts fail, the system saves the qualified segments and enters guest mode or prompts the user to continue registration later.

[0030] It should be noted that the criteria for voiceprint liveness detection are as follows: Under 16kHz sampling conditions, the system calculates the ratio (Rhf) of high-frequency energy (6-8kHz) to total energy (0-8kHz) and compares it with the liveness speech baseline (Rhf_base) from the registration phase. If Rhf / Rhf_base is below 0.55, and the short-time spectral flatness is more than 30% lower than the liveness baseline, or the inter-frame dynamic range coefficient is more than 40% lower than the liveness baseline, it is considered a suspected recording attack. If Rhf / Rhf_base is between 0.55 and 0.70, a random number or short phrase challenge is triggered, requiring the user to repeat it within 5 seconds. The above thresholds can be calibrated according to microphone frequency response and scene noise. Under normal indoor testing conditions, the recommended threshold corresponds to a false rejection rate of approximately 2% to 5%, which can be reduced by increasing the number of challenges.

[0031] It is worth mentioning that in the recognition stage: real-time acquisition of speech segments during user interaction (the shortest effective speech is 3 seconds), extraction of 80-dimensional FBank features, input into the model to obtain a 192-dimensional voiceprint embedding vector. The cosine similarity between the current embedding vector and the voiceprint templates of all registered users is calculated, and the user with the highest similarity exceeding the threshold of 0.35 is determined to be a matching user.

[0032] S2, dynamically adjust the fusion weights of each modality according to the environmental physical quantities, and use the adjusted weights to perform a weighted summation of the confidence scores of each candidate user to calculate the comprehensive fusion score of each candidate user.

[0033] In one embodiment of the present invention, the MobileFaceNet lightweight face recognition model is first employed, and the corresponding process includes: It should be noted that during the registration phase, users are guided to face the camera and capture five facial images from different angles (front view, 30° to the left, 30° to the right, 15° upward, and 15° downward). A RetinaFace detector is used for face detection, alignment is performed based on five key points, and the images are cropped to 112×112 pixels. These images are then input into the MobileFaceNet model to extract a 128-dimensional facial feature vector. During the recognition phase, video frames are captured in real-time for face detection and alignment. After extracting the facial feature vector, its similarity to a registered user template is calculated; a match is considered successful if the similarity exceeds a threshold of 0.6. Liveness detection uses infrared images or depth information (ToF sensor) to determine if the face is genuine, preventing photo-based attacks.

[0034] It should be understood that in the face registration interaction process: after voiceprint registration is completed, the system guides the user to complete the face capture sequentially through screen outlines, voice prompts, or indicator lights, focusing on frontal, left, right, upward, and downward views. Before each capture, the system checks the integrity, clarity, exposure, and liveness status of the face frame; if all conditions are met, features are extracted and stored. If no face is detected, the face is occluded by more than 30%, the image is blurry, or infrared / depth liveness detection fails, no template is generated, and the user is prompted to adjust their position or lighting. If three consecutive attempts fail, voiceprint registration is allowed to be completed only, but the user's profile will be marked as "face template pending."

[0035] It should be noted that the user registration limit and management mechanism are as follows: The terminal supports 20 local registrations by default, which can be expanded to 100 users depending on storage space and processor capabilities. Each user profile includes a user ID, user type, voiceprint template, face template, behavioral baseline, and interaction policy configuration. When the registration limit is reached, the system only allows administrators to delete, disable, or overwrite low-frequency users via management passwords, bound apps, or the local management interface, or to export historical user profiles before registering new users; ordinary users cannot directly overwrite existing templates.

[0036] Then, a behavioral feature-assisted recognition module is used, as shown in Table 1: Table 1

[0037] Furthermore, the process of behavioral feature-assisted identification includes: It should be understood that during the baseline establishment phase: the system continuously collects user behavior data and establishes a behavior baseline model within the first 7-14 days after user registration. During the recognition phase: when the confidence levels of voiceprint and facial recognition are insufficient, the system collects behavioral feature data of the current interaction, calculates the similarity between the current behavioral features and the behavioral baseline models of each user (KL divergence is used for distributional similarity, and Euclidean distance is used for numerical features), and outputs the behavioral feature-assisted recognition results and confidence levels.

[0038] It should be noted that the fallback classification strategy is as follows: When an unregistered user uses the system for the first time, the system will make a rough classification based on behavioral characteristics: elderly: speech rate <140 words / minute and fundamental frequency <150Hz and height >155cm; children: speech rate >150 words / minute and fundamental frequency >200Hz and height <140cm; adults: speech rate 140~200 words / minute and fundamental frequency 120~250Hz and height 150~185cm.

[0039] It's worth noting that the fallback thresholds are derived from several sources: the speech rate, fundamental frequency, and height thresholds for the elderly, children, and adults are initial empirical thresholds determined based on experimental samples of home smart terminals, publicly available voice statistics intervals, and experience with device installation height. These thresholds can be statistically corrected using local calibration data during actual deployment. For example, the system can collect the speech rate and fundamental frequency distribution of the household users for the first 7 days and fine-tune the thresholds according to the P25 / P75 quantiles; the height threshold can also be corrected by combining camera installation height, ToF distance, and viewing angle calibration parameters. These thresholds can be adjusted according to the actual application scenario and are not a single fixed limitation; they can be adjusted based on the specific application scenario.

[0040] It should be understood that the behavioral baseline data structure and storage method include: behavioral baselines are preferably stored in a local SQLite database, but can also be stored using encrypted JSON files. The behavioral baseline fields for a single user include user_id, time_histogram_24h, weekday_histogram, function_category_vector, distance_height_stats, speech_rate_stats, pitch_stats, last_update_time, and sample_count. For example: {user_id:"U001", time_histogram_24h:[0.02,0.01,...,0.08], function_category_vector:{call:0.32,weather:0.21,story:0.04,health:0.18}, distance_height_stats:{mean_height_cm:165,std_height_cm:4,mean_distance_m:1.8}, prosody:{speech_rate_mean:128,pitch_mean:145}, sample_count:356}. The time period distribution is stored as a 24-dimensional normalized vector, high-frequency functions are stored as category probability vectors, and height / distance and prosodic features are stored as mean, standard deviation, and sample number.

[0041] It should be further explained that the confidence score of behavioral features is calculated as follows: For candidate user Ui, the current behavioral features and the baseline model are first converted into similarities of 0 to 1. The time-period similarity is St = exp(-KL(Pcur_time||Pi_time) / τt), the content similarity is Sc = cos(Vcur_content,Vi_content), the height / distance similarity is Sh = exp(-||Xcur-Xi||2 / (2σh^2)), and the prosodic similarity is Sp = exp(-||Rcur-Ri||2 / (2σp^2)). Then, the features are fused according to weights: Cb(Ui) = clip(0.30×St+0.25×Sc+0.20×Sh+0.25×Sp,0,1). When a dimension is missing, its weight is proportionally allocated to the remaining available dimensions; τt, σh, and σp are obtained from baseline samples collected 7-14 days after registration, or the factory default values ​​can be used.

[0042] S3, determine the candidate user with the highest comprehensive fusion score as the target user, and output the user identity identifier based on the comparison result of the comprehensive fusion score and the preset threshold.

[0043] In one embodiment of the present invention, the decision fusion module employs a weighted voting mechanism to synthesize the outputs of the three modalities. Basic weights: voiceprint Wv=0.5, face Wf=0.3, behavior Wb=0.2. Dynamic adjustment rules: If ambient noise > 60dB, the voiceprint weight decreases to 0.2, and the face weight increases to 0.5; if the user is facing away from the camera or the light intensity is < 50 lux, the face weight decreases to 0.1, the voiceprint weight increases to 0.6, and the behavior weight increases to 0.3; if both voiceprint and face are unavailable, the behavior weight is 1.0, entering a pure behavior feature fallback mode.

[0044] Then, a fusion calculation is performed: for each candidate user Ui, Si = Wv×Cv(Ui) + Wf×Cf(Ui) + Wb×Cb(Ui), and the user with the highest score is selected. Confidence threshold determination: if Sbest≥0.7, the user is determined to be a known user, and the corresponding interaction strategy is loaded; if 0.4≤Sbest<0.7, an exploratory interaction strategy is adopted; if Sbest<0.4, the user is determined to be an unknown user, and the registration guidance process is initiated.

[0045] It is worth mentioning that the environmental parameter detection method is as follows: ambient noise is continuously monitored by the microphone array before and after wake-up, and the dB value is obtained by conversion through A-weighted sound pressure level or calibrated RMS energy, with a 1-second sliding average value taken within the recognition window; light intensity is preferably obtained by an ambient light sensor. When no ambient light sensor is configured, the lux value can be estimated from the camera exposure time, gain, average image brightness, and infrared illumination status. The above environmental parameters are continuously monitored at a low frequency during low-power standby. The sampling frequency is increased after the user wakes up or an interaction intent is detected, and the current environmental state is frozen at the beginning of each round of recognition for the weight calculation of this round.

[0046] It should be understood that the criteria for determining modality unavailability include: Voiceprint unavailability includes no detected valid speech, valid speech less than 3 seconds, signal-to-noise ratio below 8dB, highest voiceprint similarity below 0.35, failed voiceprint liveness detection, or voiceprint recognition module timeout; Face unavailability includes the camera not detecting a face, face bounding box confidence below 0.8, keypoint alignment failure, image brightness or clarity below a threshold, failed infrared / depth liveness detection, highest face similarity below 0.6, or face recognition module timeout. Unavailable modalities do not participate in this round of fusion; their weights are dynamically allocated to available modalities according to rules.

[0047] It should be noted that the dynamic weight adjustment takes effect in the following ways: weight adjustment takes effect according to the "interaction round", that is, at the beginning of each round of user input after a wake-up, the system collects environmental noise, illumination and modal availability, calculates the weight for this round, and keeps it unchanged during the recognition and response of this round. If the interaction duration exceeds 10 seconds or the environmental state crosses the threshold and lasts for more than 2 seconds, the weight is recalculated in the next round; weights are not frequently switched frame by frame within the same round to avoid jitter in the fusion result.

[0048] S4. In response to the user's identity identifier, retrieve the corresponding personalized interaction strategy data from the strategy configuration library, and atomically switch the terminal's interaction parameters to the target parameters corresponding to the personalized interaction strategy data in order to execute subsequent interaction responses.

[0049] In one embodiment of the present invention, the dimensions and modes of adaptive switching of the interaction strategy are shown in Table 2: Table 2

[0050] It should be noted that when a user identity switch is detected, the current user's dialogue context is saved, the target user's full interaction strategy configuration is loaded, the target user's voice is used to play transition prompts, and the user decides whether to restore the previous user's unfinished interactions based on the new user's preferences.

[0051] It should be understood that the interaction strategy configuration data structure includes: the strategy configuration can be stored in an SQLite table or in JSON format. An example JSON is: {user_id:"U001", profile_type:"elder", tts:{speed_wpm:130,volume_gain_db:3,voice:"clear_female"}, asr:{dialect_enabled:true,languages:["zh-CN","local"]}, ui:{font_pt:18,button_scale:1.3,theme:"high_contrast"},function_priority:["call","safety","health","weather"], safety:{fall_detection:true,content_filter:false}, dialog:{max_turns:null,repeat_enabled:true}}. The database implementation can be split into tables such as user_profile, voice_policy, ui_policy, function_policy, and safety_policy, and linked by user_id.

[0052] It should be noted that the implementation of loading interactive policy configurations includes: the policy switching engine prioritizes reading policy configurations from a local encrypted SQLite database; the 3-5 most recently used user policies can be cached in memory to shorten switching latency; if the cache is not hit, the target user configuration is loaded on demand, and the version number and integrity are verified. Configuration loading is performed asynchronously, without interruption of the current TTS broadcast or ongoing security tasks; if a policy field is missing, the default policy for the user type is used to fill in the missing field.

[0053] It should be noted that the hot-switching trigger conditions and execution sequence include: a hot-switching is triggered when the fusion score Sbest ≥ 0.7 and the target user is the same for two consecutive recognition windows, or when the Sbest of a single recognition window is ≥ 0.85. The execution steps are as follows: 1) Freeze the current user's dialogue context and unfinished tasks; 2) Determine if the target user is the same as the current user; 3) If they are not the same, asynchronously load the target user's policy and warm up the TTS tone, interface parameters, and function permissions; 4) Wait for the end of the current voice broadcast sentence, the screen refresh boundary, or the task safety checkpoint; 5) Atomically replace the policy handle and broadcast the transition prompt; 6) Determine whether to resume, transfer, or suspend the previous user's task based on the target user's preferences. During the switching process, the recognition thread is not blocked, and the interactive output only pauses briefly at the boundary points, usually less than 300ms.

[0054] It is worth mentioning that the processing logic when the target user is the same as the current user includes: if the identification results are the same user in two consecutive identifications, the system does not reload the strategy configuration, but only refreshes the user's active time, behavior samples and confidence records; when the identification result is in the uncertainty range of 0.4 to 0.7, historical preference prior P(Ui|time,device,context) can be introduced, such as the preferences of frequently used users, the most recently active users and the current requested content in the same period, but the weight of the historical prior does not exceed 0.15 of the total fusion score, and cannot cover the results of liveness detection failure or restricted permissions.

[0055] It is worth mentioning that the physical form and hardware deployment of the terminal include: the terminal can be a standalone smart speaker, smart screen, or conference room terminal, or it can be deployed as an embedded software module in a home gateway, set-top box, or robot device. For standalone devices, an RGB+IR camera and a ToF sensor are preferably arranged along the upper edge of the front, with 2-4 microphones arranged in an array in the ring or front area of ​​the device, and the speaker positioned below or on the side to reduce acoustic feedback. The processor can be a quad-core ARM Cortex-A55 or equivalent processor, preferably with an NPU of 0.5 TOPS or higher; a minimum of 2GB of RAM and 8GB of local storage are recommended, with user templates and policy configurations typically occupying tens of MB after encryption.

[0056] It should be noted that the hardware modules work collaboratively as follows: In standby mode, the terminal is triggered by a low-power wake-up word engine, buttons, touch, or proximity sensors; after wake-up, the microphone array, camera module, and behavior sensors simultaneously collect data, with voiceprint recognition, face recognition, and behavior analysis performed in parallel. In a typical indoor environment, the latency from wake-up to completion of user identification is 500~1500ms; if the camera does not capture a usable face, the voiceprint and behavior results are fused first, and the camera results can be used as a supplement for the next round.

[0057] It's worth noting the module data interfaces and communication protocols: each recognition module can be deployed as a function interface within the same process, or communicate via local gRPC, REST, or message queues. The unified output structure is {module:"voice|face|behavior", user_id:"U001", confidence:0.0~1.0 floating-point number, available:true / false, reason_code:"OK|NO_SIGNAL|LOW_QUALITY|LIVENESS_FAIL|TIMEOUT", feature_version:"v1"}. The decision fusion engine calls the three modules in parallel, with default timeouts of 600ms for voiceprint, 800ms for face, and 300ms for behavior; when a module times out, available=false is marked and it is removed according to dynamic weight rules, without blocking other modality results.

[0058] Overall, Figure 2 The system architecture is shown in a black-and-white block diagram. The input layer includes a microphone array, camera module, and behavior sensor; the recognition layer includes a voiceprint recognition module, a face recognition module, and a behavior feature analysis module; the decision layer outputs the final user's identity through a decision fusion engine; the output layer loads interaction strategies through a policy switching engine and completes personalized interaction through TTS voice response, screen display, and function response. See Table 3 for details. Table 3

[0059] The method of this invention realizes an integrated closed-loop process from multi-modal perception, dynamic fusion, hierarchical adjudication to policy linkage, which significantly improves the robustness and accuracy of identity recognition in multi-user scenarios, enhances the method's adaptability in dynamic environments, and has significant engineering practical value in terms of personalized switching of interaction strategies and system security.

[0060] Example 2 For example, the application scenarios of the method in the embodiments of the present invention include: Scenario 1: Automatic multi-user identification of home smart speakers.

[0061] Step 1 (User Registration): A family of three (grandfather 70, mother 35, daughter 8) shares a smart speaker. When the grandfather uses it for the first time, the speaker guides him to say "Hello, I am grandpa," extracting and storing a 192-dimensional voiceprint template. He is then guided to face the camera to capture five facial images, extracting and storing 128-dimensional facial features. After registration, he is bound to "User A (elderly)." The mother and daughter then register sequentially, binding themselves as "User B (adult)" and "User C (child)," respectively. The system gradually builds a behavioral baseline model for the three individuals over the next seven days.

[0062] Step 2 (Automatic Recognition and Strategy Switching): Grandpa says "How's the weather today?" at 6:30 AM. Voiceprint recognition cosine similarity: 0.62 (normalized confidence: 0.72); face recognition similarity: 0.85 (confidence: 0.85); behavioral characteristics (6:30 AM time, speech rate 120 words / minute, height 165cm) confidence: 0.78. Weighted fusion score SA = 0.5 × 0.72 + 0.3 × 0.85 + 0.2 × 0.78 = 0.771 > 0.7, thus identifying the user as known user A. Load elderly mode: speech rate 130 words / minute, volume +3dB, dialect recognition enabled, large font interface, security features prioritized.

[0063] Step 3 (Catch-Up Identification): After school, the daughter comes home and says, "Tell me a story," with her back to the speaker. Ambient noise is 65dB, voiceprint confidence is only 0.25 (no match with any user); the daughter's back is to the speaker, so facial recognition is unavailable. Enter the behavioral feature catch-up mode (Wb=1.0). Behavioral analysis: Interaction time 17:30 (match with user C 0.88), speech rate 175 words / minute, base frequency 230Hz (match 0.82), request for "tell a story" (match 0.91). Determined to be user C, load children's mode.

[0064] Scenario 2: Multi-user identification terminal in an office setting.

[0065] The smart terminal in the conference room automatically identifies different participants and switches meeting modes accordingly. The terminal has pre-registered the voiceprints and facial features of five senior executives. When Mr. Zhang enters the conference room, the terminal automatically recognizes him: voiceprint similarity 0.58, facial similarity 0.92, and behavioral characteristics matching the 9:00 AM work time. The strategy then switches: loading Mr. Zhang's meeting preferences—displaying recent meeting agendas, prioritizing connection to frequently used video conferencing systems, and using a formal business voice style. If Mr. Li subsequently joins, the terminal hot-switches its strategy: saving Mr. Zhang's context, loading Mr. Li's preferences, and using Mr. Li's preferred voice to deliver a welcome message.

[0066] Scenario 3: Adaptive learning for behavioral feature-assisted recognition.

[0067] After three months of system operation, a change in User A's (grandfather's) behavioral pattern was observed—high-frequency usage occurred between 6:00 AM and 12:00 PM, as per the original baseline, and more frequent usage began between 2:00 PM and 6:00 PM in new observations. The system automatically updated the behavioral baseline model using an exponential moving average (EMA) algorithm: baseline_new = 0.9 × baseline_old + 0.1 × new_data, retaining 90% weight from the historical baseline and incorporating 10% of the new observation data. The updated behavioral model more accurately identified User A, avoiding misjudgments.

[0068] Scenario 4: Multi-user identification in edge computing or low-power scenarios.

[0069] In low-power smart speakers or battery-powered terminals, the system employs a tiered recognition strategy to ensure real-time performance. During standby, only a low-power wake word model and environmental noise monitoring are run, while the camera remains in sleep or low frame rate mode. Upon wake-up, lightweight VAD and voiceprint coarse screening are performed first, and face recognition is only initiated when the voiceprint confidence score is below 0.7 or when multiple people are detected. ECAPA-TDNN and MobileFaceNet can utilize INT8 quantization, model pruning, and NPU delegated deployment, with behavioral feature analysis incrementally updated based on the most recent interaction cache. If the processor load exceeds 80% or the battery level is below 20%, the system reduces the face detection frame rate from 15fps to 5fps, adjusts the behavioral baseline update cycle from real-time updates to batch updates every 10 minutes, and prioritizes voiceprint recognition and security alarm functions. Through these strategies, single-round recognition latency can still be controlled within approximately 1.5 seconds on ARM Cortex-A55 level processors, while simultaneously reducing continuous power consumption.

[0070] The application scenarios of the method in this invention, by fusing three modal features—voiceprint, face, and behavioral prosody—and dynamically adjusting the fusion weights based on environmental physical quantities, effectively solve the problems of poor anti-interference capability of single-modal recognition in complex environments such as noise and lighting, and insufficient adaptability of fixed fusion strategies. Through a fallback recognition mechanism, identity determination can still be completed based on behavioral features when both voiceprint and face are unavailable, ensuring recognition continuity and system availability. Through atomic strategy hot-switching and task inheritance mechanisms, lossless connection of interaction contexts in multi-user continuous use scenarios is achieved, avoiding data loss or experience disruption due to user switching. Incremental adaptive updates of the behavioral baseline model continuously optimize recognition accuracy over usage time. Furthermore, the combination of hierarchical recognition strategies and lightweight model deployment maintains real-time performance even on low-power edge devices. This invention significantly improves the robustness and accuracy of multi-user recognition in dynamic environments and has broad engineering practical value. The following beneficial effects are achieved: 1. Significantly improved recognition accuracy: Multimodal fusion recognition (voiceprint + face) achieves an accuracy of over 95% on the standard test set, which is 5 to 8 percentage points higher than that of single voiceprint modality (approximately 87-90%), and its advantages are more pronounced in noisy or low-light environments.

[0071] 2. Last resort recognition capability: When voiceprint and face are unavailable, behavioral feature-assisted recognition can still provide 70-80% coarse classification accuracy, greatly reducing the probability of "unrecognizable".

[0072] 3. Zero-operation automatic switching: Eliminate manual selection steps, simplifying the interaction steps from "wake up → speak → select user → interact" to "wake up → speak → direct interaction".

[0073] 4. Personalized interactive experience: Different user groups receive optimized interaction parameters, especially special groups such as the elderly and children who experience a significant improvement in experience.

[0074] 5. Security Guarantee: Dual protection with voiceprint liveness detection (spectral analysis) and face liveness detection (infrared / depth) effectively prevents recording and photo attacks.

[0075] 6. Low computational overhead: It adopts a lightweight model (ECAPA-TDNN has about 6M parameters, MobileFaceNet has about 1M parameters), with a total computation of about 1.5 GFLOPS, and can run in real time on ordinary ARM processors.

[0076] 7. Continuous Adaptation: The behavioral baseline model is continuously updated using the EMA algorithm, and the accuracy of behavioral feature recognition gradually improves with the increase of usage time.

[0077] Example 3 To achieve the above invention, such as Figure 3 As shown, this embodiment also provides a multi-modal fusion strategy adaptive multi-user automatic identification device 10, which includes: The perception and recognition module 100 is used to collect the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities. It extracts voiceprint features, facial features, and behavioral prosodic features from the three types of data, matches each feature with multi-user templates in the local registration library, and outputs candidate users and their confidence scores in parallel for each modality.

[0078] The fusion scoring module 200 is used to dynamically adjust the fusion weights of each modality based on environmental physical quantities, and to calculate the comprehensive fusion score of each candidate user by weighted summation of the confidence scores using the adjusted weights.

[0079] The adjudication output module 300 is used to identify the candidate user with the highest comprehensive fusion score as the target user, and output the user identity identifier based on the comparison result of the comprehensive fusion score and the preset threshold.

[0080] The strategy switching module 400 is used to retrieve the corresponding personalized interaction strategy data from the strategy configuration library in response to the user identity identifier, and atomically switch the terminal's interaction parameters to the target parameters corresponding to the personalized interaction strategy data in order to execute subsequent interaction responses.

[0081] The device in this invention realizes an integrated closed-loop process from multi-modal perception, dynamic fusion, hierarchical adjudication to policy linkage, which significantly improves the robustness and accuracy of multi-user identification, enhances the adaptability in dynamic environments, and has significant engineering practical value.

[0082] Example 4 To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 4 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads the executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the multimodal fusion strategy adaptive multi-user automatic identification method described above.

[0083] Example 5 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a multimodal fusion strategy adaptive multi-user automatic identification method as described in the foregoing embodiments.

[0084] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0085] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A multimodal fusion strategy adaptive multi-user automatic identification method, characterized in that, include: Collect the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities. Extract voiceprint features, facial features, and behavioral prosodic features from the three types of data respectively. Match each feature with the multi-user template in the local registration library and output the candidate users and the confidence scores of each modality in parallel. The fusion weights of each modality are dynamically adjusted based on environmental physical quantities. The adjusted weights are then used to perform a weighted summation of the confidence scores of each candidate user to calculate the overall fusion score. The candidate user with the highest comprehensive fusion score is identified as the target user, and the user identity identifier is output based on the comparison result of the comprehensive fusion score and the preset threshold. In response to the user's identity identifier, the corresponding personalized interaction strategy data is retrieved from the strategy configuration library, and the terminal's interaction parameters are atomically switched to the target parameters corresponding to the personalized interaction strategy data in order to execute subsequent interaction responses.

2. The method as described in claim 1, characterized in that, The collection of the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities includes: User voice is collected by a microphone array consisting of at least two microphones, analog-to-digital conversion is performed at a preset sampling rate to generate voice stream data, and ambient sound pressure level data is collected simultaneously as part of the environmental physical quantity. The system simultaneously captures user facial images using an RGB camera and an infrared camera, and uses a time-of-flight sensor to obtain distance data between the user and the terminal, generating facial image frame data. The system logs record the user interaction times and the types of functions called, generating time logs and content logs in the interaction behavior data, and outputting ambient sound pressure level data and distance data as components of environmental physical quantities.

3. The method as described in claim 1, characterized in that, Voiceprint features, facial features, and behavioral prosodic features were extracted from speech stream data, facial image frame data, and interaction behavior data, respectively, including: Pre-emphasis, framing, and windowing are performed on the speech stream data. The FBank acoustic feature matrix of each frame of speech is extracted. The FBank acoustic feature matrix is ​​input into the voiceprint embedding model based on the ECAPA-TDNN architecture. After multi-scale feature aggregation and channel attention weighting, a 192-dimensional voiceprint embedding vector is output as the voiceprint feature. Face detection is performed on facial image frame data, and alignment is performed based on the detected facial key points. The aligned face region is then cropped and scaled to a preset pixel size, input into a face embedding model based on the MobileFaceNet architecture, and outputs a 128-dimensional face embedding vector as facial features. The time feature of the current interaction is extracted from the time log of the interaction behavior data, the function category feature of the current call is extracted from the content log, the estimated height of the current user is extracted from the distance data, and the speech rate and fundamental frequency value of the current interaction are extracted from the speech stream data. Together, they form a behavioral prosodic statistical vector as behavioral prosodic features.

4. The method as described in claim 1, characterized in that, Each feature is matched against multi-user templates in the local registry, and candidate users and their confidence scores are output in parallel for each modality, including: Using voiceprint features as query vectors, the first cosine similarity between the corresponding query vector and the voiceprint template vector of each registered user stored in the local registration database is calculated. Registered users whose maximum first cosine similarity exceeds the first preset threshold are identified as voiceprint candidate users, and the maximum first cosine similarity is normalized and used as the voiceprint confidence score output. Using facial features as query vectors, the second cosine similarity between the corresponding query vector and the facial template vectors of each registered user stored in the local registration database is calculated. Registered users whose maximum second cosine similarity exceeds the second preset threshold are identified as face candidate users, and the maximum second cosine similarity is normalized and used as the face confidence score output. Using behavioral prosodic statistical vectors as query data, the time-segment similarity, content similarity, height-distance similarity, and prosodic similarity between the corresponding query data and the historical behavioral baselines of each registered user stored in the local registry are calculated. After weighted fusion of the similarities in each dimension, the behavioral candidate users and the corresponding behavioral confidence scores of the candidate users are determined and output.

5. The method as described in claim 1, characterized in that, The fusion weights of each mode are dynamically adjusted based on environmental physical quantities, including: The environmental sound pressure level data in the environmental physical quantities are analyzed. When the environmental noise exceeds the first decibel threshold, the basic fusion weight of the voiceprint modality is reduced from the first weight value to the second weight value, while the basic fusion weight of the face modality is increased from the third weight value to the fourth weight value. The illumination data in the environmental physical quantities are analyzed. When the ambient illumination is lower than the first illuminance threshold or when it is determined from the facial image frame data that the user is not facing the camera, the basic fusion weight of the face modality is lowered to the fifth weight value, while the basic fusion weight of the voiceprint modality is raised to the sixth weight value. The voiceprint modality and face modality are detected separately to determine whether they are available. When both the voiceprint modality and face modality are determined to be unavailable, the fusion weight of the behavioral modality is configured as the seventh weight value, and the behavioral candidate user and the behavioral confidence of the behavioral candidate user are used as the sole criterion for judgment.

6. The method as described in claim 1, characterized in that, Based on the comparison between the comprehensive fusion score and the preset threshold, the user identity identifier is output, including: When the overall fusion score is greater than or equal to the high confidence threshold, the target user is identified as a known user, and the corresponding registered identity identifier of the target user and the corresponding interaction strategy index are output. When the overall fusion score is greater than or equal to the low confidence threshold and less than the high confidence threshold, the corresponding target user is identified as a user to be confirmed, and a tentative identity identifier and a general interaction strategy index are output. When the overall fusion score is less than the low confidence threshold, the current user is judged as an unknown user, and a new user registration guidance instruction is output as the user's identity identifier.

7. The method as described in claim 1, characterized in that, Atomically switching the terminal's interaction parameters to the target parameters corresponding to the personalized interaction strategy data includes: Before switching execution, the dialogue context data and unfinished task status of the previous user are saved to a temporary cache, and the safe switching boundary point of the current interaction output is obtained; When the safe switching boundary point is reached, the personalized interaction policy data of the target user is read from the policy configuration library. Based on the speech synthesis parameters, interface display parameters and function permission parameters in the personalized interaction policy data, the speech rate and timbre configuration of the terminal speech synthesis engine, the font and theme configuration of the interface rendering engine and the access permission list of the function scheduling module are updated respectively. After the switch is completed, based on the task inheritance identifier in the personalized interaction strategy data, it is decided to resume the execution of the previous user's unfinished tasks, transfer the unfinished tasks to the target user's account, or permanently suspend the unfinished tasks.

8. A multimodal fusion strategy adaptive multi-user automatic identification device, characterized in that, include: The perception and recognition module is used to collect the current user's voice stream data, facial image frame data, and interactive behavior data containing environmental physical quantities. It extracts voiceprint features, facial features, and behavioral prosodic features from the three types of data, matches each feature with multi-user templates in the local registration library, and outputs candidate users and their confidence scores in parallel for each modality. The fusion scoring module is used to dynamically adjust the fusion weights of each modality based on environmental physical quantities, and to calculate the comprehensive fusion score of each candidate user by weighted summation of the confidence scores using the adjusted weights. The adjudication output module is used to identify the candidate user with the highest comprehensive fusion score as the target user, and output the user identity identifier based on the comparison result of the comprehensive fusion score and the preset threshold. The strategy switching module is used to retrieve the corresponding personalized interaction strategy data from the strategy configuration library in response to the user's identity identifier, and atomically switch the terminal's interaction parameters to the target parameters corresponding to the personalized interaction strategy data in order to execute subsequent interaction responses.

9. An electronic device, characterized in that, include: processor; The memory stores executable instructions; when the processor executes the instructions, it implements the multimodal fusion strategy adaptive multi-user automatic identification method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The system contains a computer program that, when executed by a processor, implements a multimodal fusion strategy adaptive multi-user automatic identification method as described in any one of claims 1-7.

Images