Recording content processing methods, devices, storage media, and electronic devices

Abstract

This application discloses a method, apparatus, storage medium, and electronic device for processing audio recording content. The method includes: extracting text semantic features based on the target text content of the audio recording data to be classified using a large-scale audio recording content classification model; extracting acoustic environment features based on the audio recording data to be classified; and extracting user context features based on the acquisition behavior data of the audio recording data to be classified, so as to realize the identification of the target knowledge base to which the audio recording data to be classified belongs through multimodal features, thereby improving the accuracy of knowledge base allocation for audio recording content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and electronic device for processing audio recording content. Background Technology

[0002] With the popularization of mobile internet and smart terminal technology, voice recorders, smartphones and various wearable devices have become the main tools for people to record voice information such as meetings, interviews, classes and inspirations. The massive amount of voice data generated is gradually becoming an important part of personal knowledge assets.

[0003] However, recording equipment is limited to simple audio acquisition and storage. Post-recording content processing, information extraction, and knowledge archiving still rely on manual operation. Users often need to spend a significant amount of time repeatedly listening to recordings, manually organizing written notes, judging content themes based on personal experience, and categorizing each item into corresponding folders or knowledge bases. This manual processing method is not only inefficient but also prone to inconsistencies in classification standards, omissions of key information, or archiving errors due to subjective differences in perception. This prevents a large amount of high-value audio information from being linked and reused. Therefore, accurately identifying the category of the recorded content is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, computer storage medium, and electronic device for processing audio recording content. The technical solution is as follows: In a first aspect, embodiments of this application provide a method for processing audio recording content, the method comprising: Determine the target text content of the audio recording data to be classified; The recording content classification model is used to extract text semantic features based on the target text content, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. Based on the text semantic features, the acoustic environment features, and the user context features, the text content is archived and classified using a knowledge base to obtain the target knowledge base corresponding to the recording data to be classified.

[0005] In one possible implementation, the step of performing knowledge base archiving and classification processing on the target text content based on the text semantic features, the acoustic environment features, and the user context features to obtain the target knowledge base corresponding to the recording data to be classified includes: Based on the semantic features of the text, the probability distribution data of the audio recording data to be classified belonging to each candidate knowledge base is determined; The semantic classification ambiguity detection result is determined based on the probability distribution data; If the semantic classification ambiguity detection result indicates that there is an ambiguous type, then the text semantic features, the acoustic environment features, and the user context features are subjected to multimodal weighted fusion to obtain a multimodal comprehensive feature vector, and the target knowledge base corresponding to the recording data to be classified is identified based on the multimodal comprehensive feature vector; If the semantic classification ambiguity detection result is that there is no ambiguous type, then the target knowledge base corresponding to the recording data to be classified is determined based on the probability distribution data.

[0006] In one possible implementation, determining the semantic classification ambiguity detection result based on the probability distribution data includes: Based on the probability distribution data, a semantic uncertainty index is determined; If the semantic uncertainty index is greater than or equal to a preset threshold, a semantic classification ambiguity detection result with semantic ambiguity type is generated; If the semantic uncertainty index is less than a preset threshold, a semantic classification ambiguity detection result without semantic ambiguity type is generated.

[0007] In one possible implementation, the multimodal weighted fusion of the text semantic features, the acoustic environment features, and the user context features to obtain a multimodal comprehensive feature vector includes: Linear projection of the acoustic environment features yields acoustic query key vectors and acoustic value vectors, and linear projection of the user context features yields context query key vectors and context value vectors. An acoustic correction term vector is determined based on the text semantic features, the acoustic query key vector, and the acoustic value vector; a context correction term vector is determined based on the text semantic features, the context query key vector, and the context value vector. A multimodal comprehensive feature vector is obtained by weighted fusion of the text semantic features, the semantic uncertainty index, the acoustic correction term vector, and the context correction term vector.

[0008] In one possible implementation, determining the acoustic correction term vector based on the text semantic features, the acoustic query key vector, and the acoustic value vector, and determining the context correction term vector based on the text semantic features, the context query key vector, and the context value vector, includes: The acoustic correction term weights are obtained by performing an inner product operation based on the text semantic features and the acoustic query key vector, and the acoustic correction term vector is calculated based on the acoustic correction term weights and the acoustic value vector. The context correction term weight is obtained by performing an inner product operation based on the text semantic features and the context query key vector, and the context correction term vector is calculated based on the context correction term weight and the context value vector.

[0009] In one possible implementation, the extraction of acoustic environment features based on the recording data to be classified includes: Based on the recording data to be classified, the characteristics of the speaker are identified, and acoustic parameters are extracted based on the recording data to be classified. Identify physical space environment fingerprints based on the audio recording data to be classified; Acoustic environment features are generated based on the interlocutor features, the acoustic parameters, and the physical space environment fingerprint.

[0010] In one possible implementation, determining the target text content of the recording data to be classified includes: The recorded audio data to be classified is processed into speech-to-text to obtain the original text content, and the original text content is then subjected to structured semantic extraction to obtain the target text content.

[0011] Secondly, embodiments of this application provide a recording content processing apparatus, the apparatus comprising: The data acquisition module is used to determine the target text content of the audio recording data to be classified; The feature extraction module is used to extract text semantic features based on the target text content using a large recording content classification model, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. The data classification module is used to perform knowledge base archiving and classification processing on the text content based on the text semantic features, the acoustic environment features, and the user context features to obtain the target knowledge base corresponding to the recording data to be classified.

[0012] Optionally, the data classification module includes: The first classification processing unit is used to determine the probability distribution data of the recording data to be classified belonging to each candidate knowledge base based on the semantic features of the text. The second classification processing unit is used to determine the semantic classification ambiguity detection result based on the probability distribution data; The third classification processing unit is used to perform multimodal weighted fusion of the text semantic features, the acoustic environment features and the user context features to obtain a multimodal comprehensive feature vector if the semantic classification ambiguity detection result is that there is an ambiguity type, and to identify the target knowledge base corresponding to the recording data to be classified based on the multimodal comprehensive feature vector; The fourth classification processing unit is used to determine the target knowledge base corresponding to the recording data to be classified based on the probability distribution data if the semantic classification ambiguity detection result is that there is no ambiguity type.

[0013] Optionally, the second classification processing unit is specifically used for: Based on the probability distribution data, a semantic uncertainty index is determined; If the semantic uncertainty index is greater than or equal to a preset threshold, a semantic classification ambiguity detection result with semantic ambiguity type is generated; If the semantic uncertainty index is less than a preset threshold, a semantic classification ambiguity detection result without semantic ambiguity type is generated.

[0014] Optionally, the third classification processing unit includes: The first fusion processing subunit is used to linearly project the acoustic environment features to obtain an acoustic query key vector and an acoustic value vector, and linearly project the user context features to obtain a context query key vector and a context value vector. The second fusion processing subunit is used to determine an acoustic correction term vector based on the text semantic features, the acoustic query key vector, and the acoustic value vector, and to determine a context correction term vector based on the text semantic features, the context query key vector, and the context value vector; The third fusion processing subunit is used to perform weighted fusion based on the text semantic features, the semantic uncertainty index, the acoustic correction term vector and the context correction term vector to obtain a multimodal comprehensive feature vector.

[0015] Optionally, the second fusion processing subunit is specifically used for: The acoustic correction term weights are obtained by performing an inner product operation based on the text semantic features and the acoustic query key vector, and the acoustic correction term vector is calculated based on the acoustic correction term weights and the acoustic value vector. The context correction term weight is obtained by performing an inner product operation based on the text semantic features and the context query key vector, and the context correction term vector is calculated based on the context correction term weight and the context value vector.

[0016] Optionally, the feature extraction module is specifically used for: Based on the recording data to be classified, the characteristics of the speaker are identified, and acoustic parameters are extracted based on the recording data to be classified. Identify physical space environment fingerprints based on the audio recording data to be classified; Acoustic environment features are generated based on the interlocutor features, the acoustic parameters, and the physical space environment fingerprint.

[0017] Optionally, the feature extraction module is specifically used for: The recorded audio data to be classified is processed into speech-to-text to obtain the original text content, and the original text content is then subjected to structured semantic extraction to obtain the target text content.

[0018] The beneficial effects of the technical solutions provided in this application include at least the following: In the recording content processing method provided in this application embodiment, text semantic features are extracted based on the target text content of the recording data to be classified by a large recording content classification model, acoustic environment features are extracted based on the recording data to be classified, and user context features are extracted based on the collection behavior data of the recording data to be classified, so as to realize the identification of the target knowledge base to which the recording data to be classified belongs through multimodal features, thereby improving the accuracy of the knowledge base assigned to the recording content. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a method for processing audio recording content provided in an embodiment of this application; Figure 2 This is a flowchart illustrating another method for processing audio recording content provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a method for determining multimodal integrated feature vectors provided in an embodiment of this application. Figure 4 This is a schematic diagram of the structure of a recording content processing device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the inventive objectives, features, and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this application, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0023] The present application will now be described in detail with reference to specific embodiments.

[0024] In one embodiment, such as Figure 1 As shown, a method for processing recorded audio content is proposed. This method can be implemented using a computer program and can run on a recorded audio content processing device based on the von Neumann architecture. This computer program can be integrated into applications or run as a standalone utility application.

[0025] Specifically, the methods for processing the recorded content include: S101, Determine the target text content of the audio recording data to be classified.

[0026] The audio recording data to be classified refers to the raw audio data stream or audio file obtained by recording equipment and digitized. It is usually stored and transmitted in pulse code modulation format, waveform audio file format, or advanced audio coding format. The recording data not only includes human voice signals, but also mixed signals of environmental background noise and multiple speakers' speech.

[0027] In this embodiment, the recording device is a hardware terminal for acquiring audio signals, including but not limited to smart voice recorders, smartphones, tablets, in-vehicle smart terminals, and wearable audio acquisition devices. The recording device can acquire recording data and can also serve as the execution subject of the recording content processing method in this embodiment. Alternatively, in this embodiment, the recording device may only be used to acquire recording data and upload it to a server, with the server serving as the execution subject for the recording content processing method in this embodiment.

[0028] In one embodiment, upon detecting audio data to be classified, the target text content of the audio data to be classified is determined. The target text content refers to the text information extracted from the audio data to be classified and used for subsequent classification processing.

[0029] In scenarios where the recording device is the primary executor of the method, when the user ends the recording operation, the recording data to be classified is detected. At this point, the recording data to be classified is the recording data collected during this recording operation. In scenarios where the server is the primary executor of the method, when the server receives the recording data uploaded by the recording device, the recording data to be classified is detected. At this point, the recording data to be classified is the recording data uploaded by the recording device.

[0030] Optionally, the target text content can be the text content directly obtained by performing speech recognition processing on the recording data to be classified. Specifically, firstly, speech activity detection is performed on the recording data to remove silent segments. Then, an end-to-end speech recognition model, such as an acoustic model based on a Transformer or Conformer architecture combined with a language model, decodes the audio segments frame by frame into text words, and finally concatenates them to form the target text content.

[0031] Alternatively, the target text content can also be a structured summary text content obtained by performing structured generalization on the text content obtained from speech recognition of the audio data to be classified.

[0032] S102 employs a large-scale recording content classification model to extract text semantic features based on the target text content, extract acoustic environment features based on the recording data, and extract user context features based on the collection behavior data of the recording data. Among them, textual semantic features refer to features extracted from the target text content that can characterize the deep semantics, knowledge domain orientation, and content structure attributes of the text. Acoustic environment features refer to features extracted from the audio signals of the recording data itself that can characterize the physical environment attributes, speaker size, and voice interaction patterns at the time of recording. User context features refer to features extracted from the collected behavioral data that characterize the specific project path and spatiotemporal environment information of the user at the time of recording.

[0033] The large-scale audio content classification model is pre-trained based on the basic multimodal large-scale model for audio content classification scenarios.

[0034] In one embodiment, within the large-scale recording content classification model, information from different modalities is processed through three parallel feature extraction branches. Specifically, textual semantic features are extracted through the text data processing branch of the large-scale recording content classification model, acoustic environment features are extracted through the acoustic data processing branch of the large-scale recording content classification model, and user context features are extracted through the behavioral data processing branch of the large-scale recording content classification model.

[0035] S103, based on text semantic features, acoustic environment features and user context features, perform knowledge base archiving and classification processing on the text content to obtain the target knowledge base corresponding to the recording data to be classified.

[0036] In one embodiment, a large-scale recording content classification model is used to perform multimodal fusion of text semantic features, acoustic environment features, and user context features, and the target knowledge base to which the recording data belongs is identified based on the fused multimodal feature vector.

[0037] In this embodiment of the application, a large-scale recording content classification model is used to extract text semantic features based on the target text content of the recording data to be classified, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. This enables the identification of the target knowledge base to which the recording data to be classified belongs through multimodal features, thereby improving the accuracy of knowledge base allocation for recording content.

[0038] Please see Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the audio recording content processing method proposed in this application.

[0039] Specifically, the methods for processing the recorded content include: S201, the audio recording data to be classified is processed into speech-to-text to obtain the original text content, and the original text content is processed into structured semantic extraction to obtain the target text content.

[0040] In one embodiment, after obtaining the original text content, noise removal is first performed on the original text content to obtain reference text content. Based on speaker segmentation identifiers, the reference text content is divided into multiple speaker turn segments, and these multiple speaker turn segments are aggregated into different semantic paragraphs, including opening greetings, topic presentation, discussion and debate, conclusion and resolution, and task assignment paragraphs. The noise removal mainly includes removing repetitive interjections and meaningless filler words. Then, the semantic paragraphs are clustered to obtain at least one topic cluster corresponding to a theme. Based on the semantic paragraphs within each topic cluster, a theme summary sentence and a set of topic keywords for each topic cluster are extracted. Based on the semantic paragraphs within each topic cluster, the semantic roles of the participants are identified. Structured elements are extracted for each topic cluster, including the participants' viewpoint records, decision conclusions, explicit tasks, and controversial unresolved matters. Next, each topic cluster and its corresponding structured elements are encapsulated as topic detail nodes. The first-level child nodes of each topic detail node include a topic summary sentence, semantic roles of participants, viewpoint records, decision conclusions, to-do items, and unresolved items. Multiple topic detail nodes are aggregated into a detail layer in the form of a meeting content tree according to timeline or logical dependencies. Key information is extracted from the topic summary sentences and decision conclusions of each topic detail node to generate a one-segment content summary as the summary layer, and a list of key topic points is generated. A global participant set is extracted from the semantic roles of participants in all topic detail nodes, and a global time is extracted to generate a meta-information layer. The meta-information layer, summary layer, and detail layer are assembled into a tree-like hierarchical structure through parent-child reference relationships to obtain the target text content.

[0041] S202 employs a large-scale recording content classification model to extract textual semantic features based on the target text content, acoustic environment features based on the recording data to be classified, and user context features based on the acquisition behavior data of the recording data to be classified.

[0042] In one embodiment, extracting text semantic features may include: first, using a large-scale audio content classification model, extracting features from the tree-like hierarchical structure of the target text content according to the meta-information layer, summary layer, and detail layer to obtain a set of meta-information features, summary features, and field semantic features.

[0043] In one embodiment, the process of extracting acoustic environment features based on the recording data to be classified may specifically include: identifying speaker features based on the recording data to be classified; extracting acoustic parameters based on the recording data to be classified; identifying physical space environment fingerprints based on the recording data to be classified; and generating acoustic environment features based on speaker features, acoustic parameters, and physical space environment fingerprints.

[0044] The characteristics of the speakers include the number of speakers identified from the recording data to be classified, the percentage of each speaker's speaking time, and the frequency of their turns. Specifically, a large-scale recording content classification model is used to detect speech activity in the recording data to be classified. Based on the results of the speech activity detection, silent segments are removed to obtain speech segments. The voiceprint feature vector of each speech segment is extracted, and the number of speakers is determined based on the voiceprint feature vector.

[0045] The acoustic parameters refer to structured metrics extracted from the recording data to be classified, representing the rhythm of the speaker's speech behavior and the intensity of dialogue interaction. These parameters may include average speech rate, interruption frequency, and pause interval duration. Specifically, a large-scale recording content classification model is used to detect speech activity in the recording data to be classified, obtaining a valid speech frame sequence. Syllable boundary detection is performed on the valid speech frame sequence to obtain the total number of valid syllables. The total number of valid syllables is divided by the total effective speech duration to obtain the average speech rate. Alternatively, the number of valid text characters is counted based on the target text content, and the number of valid text characters is divided by the total effective speech duration to obtain the average speech rate. The large-scale recording content classification model identifies the start and end time sequences of each independent speaker's speech, and determines adjacent speaker switching events based on the speech start and end time sequences. Adjacent speaker switching events are traversed, and it is detected whether the start time of the subsequent speaker's speech is earlier than the end time of the previous speaker's speech. If there is a time overlap and the overlap duration exceeds a preset threshold, it is determined as an interruption event. The total number of interruption events within the total recording duration is counted, and the total number of interruption events is divided by the total duration to obtain the interruption frequency. The recording content classification model extracts silent segments between adjacent valid speech segments based on speech activity detection results; the first silent segment with a duration less than a preset duration threshold is filtered out, and the second silent segment with a duration greater than the preset duration threshold is retained; the mean and maximum duration of all second silent segments are calculated, and the mean and maximum duration are determined as pause interval duration features.

[0046] The physical space environment fingerprint refers to the features characterizing the physical space attributes of the recording location. Specifically, the energy attenuation tail segments after each speaker's speech are located using a large-scale recording content classification model. After removing the background noise base, a clean attenuation signal is obtained. Subsequently, the clean attenuation signal is processed by frame segmentation, and the short-time logarithmic energy value of each frame is calculated to construct an attenuation curve of energy changing over time. The attenuation curve is linearly fitted in the logarithmic domain to determine the time required for the sound energy to attenuate by 60 dB from the initial peak, denoted as the RT60 value. Next, differential envelope detection is performed on the energy attenuation curve to identify local energy rebound peaks caused by reflections from interfaces such as walls, ceilings, and floors during the attenuation process. The number of reflection events per unit time, the average time interval between adjacent reflection peaks, and the attenuation ratio of each reflection energy relative to the direct sound energy are statistically analyzed to generate multiple reflection echo pattern features. A physical space environment fingerprint including the RT60 value and multiple reflection echo pattern features is generated.

[0047] Specifically, acoustic environment features are generated based on the characteristics of the speakers, acoustic parameters, and physical space environment fingerprints through a large-scale model for classifying recorded content.

[0048] Among them, collected behavioral data refers to user behavior logs and sensor data automatically recorded by the recording device at the moment of triggering recording or during the recording process. Collected behavioral data may include timestamps, geographic coordinates, location scene tags, and the project file path or working directory identifier in which the user is currently active in the operating system or business software. User context features are semantic feature representations used to characterize the temporal environment, spatial scene, and business working status of the user at the time of recording.

[0049] The process of extracting user context features from the collection behavior data of the recording data to be classified can specifically include: extracting time context sub-features, spatial context sub-features, and business context sub-features from the collection behavior data using the recording content classification model, and concatenating the time context sub-features, spatial context features, and business context features to obtain the user context features.

[0050] Specifically, the recording content classification model obtains the timestamp of the recording trigger time, and extracts the hour component, weekday component, and date component from the timestamp; based on the time period mapping rule, the hour component and weekday component are mapped to the target time period category, which is one of the working hours, non-working hours, or lunch break; based on the date mapping rule, the date component and weekday component are mapped to the target date category, which includes one of the working days, public holidays, and adjusted working days; and time context sub-features are generated based on the target time period category and the target date category.

[0051] The large-scale model for classifying audio content obtains latitude and longitude coordinates based on GPS positioning signals. A geofence matching algorithm then maps these coordinates to a target spatial scene category. This target spatial scene category is one of a predefined set of categories, which may include, but are not limited to, company campuses, customer sites, home offices, vehicles, or outdoor environments. If GPS positioning signals are unavailable or insufficiently accurate, location inference is performed based on base station positioning or Wi-Fi access point identifiers, and the inference result is mapped to the target spatial scene category. The target spatial scene is then identified as a spatial context sub-feature.

[0052] The recording content classification model, based on the operating system process monitoring interface or business software application programming interface, obtains the application identifier of the user's current active state and the directory path of the file opened in that application; it performs prefix matching between the file directory path and a preset project path mapping table to determine whether the user is under a specific project path; if the match is successful, it extracts the project identifier and business domain attribute corresponding to that project path; if the match fails or there is no project path, it generates a business status vacancy label; it generates business context sub-features based on the application identifier, project identifier, and business domain attribute; or, it generates business context sub-features based on the application identifier, project identifier, and business status vacancy label.

[0053] S203, determine the probability distribution data of the audio recording data to be classified belonging to each candidate knowledge base based on the semantic features of the text.

[0054] The probability distribution data includes the probability that the audio recordings to be classified belong to each candidate knowledge base, which are existing knowledge bases of different categories. For example, candidate knowledge bases may include, but are not limited to, sales visit knowledge bases, personal voice memo knowledge bases, classroom note knowledge bases, meeting summary knowledge bases, and inspiration notes knowledge bases.

[0055] In one embodiment, a large-scale audio content classification model is used to perform matrix multiplication between the text semantic vector and the classification weight matrix to obtain a classification score vector. An activation function is then used to normalize the classification score vector to obtain the probability distribution data of the audio data to be classified belonging to each candidate knowledge base. Specifically, the large-scale audio content classification model obtains meta-information semantic vectors, summary semantic vectors, document compression vectors, and topic distribution vectors based on text semantic features. Text semantic vectors are then generated based on these meta-information semantic vectors, summary semantic vectors, document compression vectors, and topic distribution vectors.

[0056] Specifically, the global participant set, global time range, collection timestamp, and number of participants in the metadata features are encoded through the large-scale audio content classification model to generate the initial metadata vector.

[0057] The audio content classification model encodes the segmented content summary from the summary features to generate a summary body vector, and encodes the list of key issues from the summary features to generate a list vector, thus generating an initial summary vector that includes the summary body vector and the list vector.

[0058] The large-scale model for classifying audio content decomposes each topic detail node in the semantic feature set into six structured fields: topic summary sentence, participants, viewpoint record, decision conclusion, to-do items, and unresolved items. Each structured field is used as an independent text fragment input field encoder, and word embedding and context encoding are used to generate an initial set of semantic vectors for the fields bound to the topic nodes.

[0059] Furthermore, using the large-scale recording content classification model, the vectors corresponding to the global participant set and the global event scope in the initial metadata vector are used as query vectors, and the vectors corresponding to the collection timestamp and the number of participants in the initial metadata vector are used as key vectors. The contribution weight of the key vectors to the query vector is calculated using a cross-attention mechanism. After weighted aggregation and fully connected projection, the metadata semantic vector is obtained.

[0060] By performing self-attention encoding on the main abstract vector in the initial abstract vector using a large-scale audio content classification model, the core semantic theme is extracted. Then, the key point list vector is used as an auxiliary prompt input cross-attention mechanism to calculate the supplementary weights of each key point list vector to the main abstract vector. After weighted fusion and pooling operations, the abstract semantic vector is obtained.

[0061] The weights of each field in the initial semantic vector set are determined by a large-scale model for classifying audio content. Based on the field weights, the initial semantic vectors of each field in the initial semantic vector set are weighted to obtain the field reference semantic vectors. The field reference semantic vectors are grouped according to their respective topic nodes. Through a topic-based self-attention aggregation mechanism, topic semantic vectors corresponding to each topic node are generated. All topic semantic vectors are then globally aggregated through a cross-topic attention mechanism to generate a chapter compression vector.

[0062] Furthermore, a topic distribution vector is obtained by performing topic distribution inference on the concatenated vector (combined from the compressed text vector and the summary semantic vector) using a large-scale audio content classification model. Each dimension of the topic distribution vector corresponds to an abstract business topic category, and the value of each dimension represents the semantic association strength between the target text content and the corresponding abstract business topic category. For example, if the topic distribution vector inferred from the concatenated vector corresponding to the target text content has a value of 0.65 in the technical review dimension, 0.25 in the project review dimension, and 0.1 in the requirements analysis dimension, it indicates that the text semantics of the target text content are mainly of the technical review type, but there is semantic overlap of the project review type.

[0063] Finally, using a large-scale audio content classification model, the metadata semantic vector, summary semantic vector, text compression vector, and topic distribution vector are concatenated according to a preset dimensional order to obtain a composite vector. A fully connected projection layer is then used to perform non-linear activation and dimensional compression on this composite vector to obtain the text semantic vector. It can be understood that the text semantic vector simultaneously carries the main idea of ​​the target text content, the distribution of topics, and global contextual information.

[0064] S204, Determine the semantic classification ambiguity detection result based on probability distribution data.

[0065] In one embodiment, determining the semantic classification ambiguity detection result may specifically include: determining a semantic uncertainty index based on probability distribution data; if the semantic uncertainty index is greater than or equal to a preset threshold, generating a semantic classification ambiguity detection result with a semantic ambiguity type; if the semantic uncertainty index is less than the preset threshold, generating a semantic classification ambiguity detection result without a semantic ambiguity type.

[0066] Understandably, when the semantic uncertainty index is greater than or equal to the preset threshold, it indicates that the classification recognition of the text semantic features is low and there is ambiguity. The large recording content classification model has difficulty determining the knowledge base category to which the recording data to be classified belongs based on the text semantic features. On the other hand, when the semantic uncertainty index is less than the preset threshold, it indicates that the classification recognition of the text semantic features is high and there is no ambiguity. The large recording content classification model can determine the knowledge base category to which the recording data to be classified belongs based on the text semantic features.

[0067] Optionally, one method for determining the semantic uncertainty index is as follows: multiply each reference probability by a first value corresponding to that reference probability to obtain a second value; sum all the second values ​​to obtain a third value; and take the negative of the third value to obtain the semantic uncertainty index. Here, the first value is the value obtained by performing a logarithmic operation on the reference probabilities with base 2, and the reference probabilities are the probabilities that the speech data to be classified belongs to the candidate knowledge base.

[0068] S205, if the semantic classification ambiguity detection result is that there is an ambiguous type, then the text semantic features, acoustic environment features and user context features are multimodally weighted and fused to obtain a multimodal comprehensive feature vector, and a target knowledge base corresponding to the target text content is identified based on the multimodal comprehensive feature vector.

[0069] It is understandable that when the semantic classification ambiguity detection result is that there is an ambiguous type, the large-scale recording content classification model identifies the knowledge base category to which the recording data to be classified belongs through text semantic features. The large-scale recording content classification model is difficult to determine the knowledge base category to which the recording data to be classified belongs based on text semantic features. Therefore, on the basis of text semantic features, acoustic environment features and user context features can be combined to obtain a multimodal comprehensive feature vector.

[0070] Alternatively, see Figure 3 The diagram illustrates a method for determining multimodal synthetic feature vectors. This method includes the following steps S301-S303: S301: Linear projection of acoustic environment features yields acoustic query key vector and acoustic value vector; linear projection of user context features yields context query key vector and context value vector. S302: Determine the acoustic correction term vector based on text semantic features, acoustic query key vector and acoustic value vector; determine the context correction term vector based on text semantic features, context query key vector and context value vector. S303: A multimodal comprehensive feature vector is obtained by weighted fusion of textual semantic features, semantic uncertainty index, acoustic correction term vector and context correction term vector.

[0071] In one embodiment, acoustic query key vectors and acoustic value vectors are obtained by linearly projecting acoustic environment features using a large recording content classification model, and context query key vectors and context value vectors are obtained by linearly projecting user context features using the same large recording content classification model.

[0072] In one embodiment, when performing step S302, it may include: performing an inner product operation based on text semantic features and acoustic query key vector to obtain acoustic correction term weights, and calculating acoustic correction term vectors based on acoustic correction term weights and acoustic value vectors; performing an inner product operation based on text semantic features and context query key vector to obtain context correction term weights, and calculating context correction term vectors based on context correction term weights and context value vectors.

[0073] Specifically, the large-scale recording content classification model uses matrix operations to calculate the first inner product of the text semantic vector corresponding to the text semantic features and the acoustic query key vector. The first inner product is normalized to obtain the acoustic correction term weight. The acoustic value vector is then weighted using the acoustic correction term weight to obtain the acoustic correction term vector. The large-scale recording content classification model also uses matrix operations to calculate the second inner product of the text semantic vector corresponding to the text semantic features and the context query key vector. The second inner product is normalized to obtain the context correction term weight. The context value vector is then weighted using the context correction term weight to obtain the context correction term vector.

[0074] In one embodiment, a nonlinear mapping function is used to calculate the semantic uncertainty index through a large-scale recording content classification model to obtain the target weight. The target weight is then used to perform a weighted summation operation on the acoustic correction term vector and the context correction term vector to obtain the reference fusion feature vector. Finally, the text semantic vector corresponding to the text semantic feature and the reference fusion feature vector are concatenated to obtain the multimodal comprehensive feature vector.

[0075] Optionally, the following describes the training process of a large-scale audio content classification model: Model creation: Obtain the basic multimodal large language model, create a first initial recording content classification scenario plugin model and a second initial recording content classification scenario plugin model for recording content classification scenarios, and form an initial recording content classification large model based on the basic multimodal large language model, the first initial recording content classification scenario plugin model and the second initial recording content classification scenario plugin model; the basic multimodal large language model (MLLM) includes, but is not limited to, the DeepSeek large model, the GPT series large models, etc.

[0076] Sample data acquisition: Acquire sample data in the recording content classification scenario. This sample data includes sample recording data from multiple recording scenarios, sample text content of the sample recording data, and sample collection behavior data of the sample recording data.

[0077] Sample data annotation: Based on the classification requirements of the recording content classification scenario, the text semantic feature labels, acoustic environment feature labels, user context feature labels, and probability distribution labels are annotated for the sample recording data. The probability distribution label includes the probability value of the sample recording data belonging to each candidate knowledge base.

[0078] Model training process: Input the sample data into the initial recording content classification model for at least one round of model training. During the forward training process, based on the sample data, the initial recording content classification model extracts predicted text semantic features, predicted acoustic environment features, and predicted user context features. The initial recording content classification model performs knowledge base classification processing based on the predicted text semantic features to obtain the first predicted probability distribution data. The initial recording content classification model performs multimodal weighted fusion based on the predicted text semantic features, predicted acoustic environment features, and predicted user context features to obtain a multimodal comprehensive feature vector. Based on the multimodal comprehensive feature vector, knowledge base classification processing is performed to obtain the second predicted probability distribution data. During the model reverse training process, the first model loss value is determined based on the predicted text semantic features and text semantic feature labels; the second model loss value is determined based on the first predicted probability distribution data and probability distribution labels; and the first target loss value is obtained based on the first and second model loss values. The third model loss value is determined based on the predicted acoustic environment features and acoustic environment feature labels; the fourth model loss value is determined based on the predicted user context features and user context feature labels; the fifth model loss value is determined based on the second predicted probability distribution data and probability distribution labels; and the second target loss value is obtained based on the third, fourth, and fifth model loss values. Based on the first target loss value, the model parameters of the first initial recording content classification scenario plugin model in the initial recording content classification large model are adjusted to obtain the first recording content classification scenario plugin model. Based on the second target loss value, the model parameters of the second initial recording content classification scenario plugin model in the initial recording content classification large model are adjusted to obtain the second recording content classification scenario plugin model. The trained recording content classification large model is composed of the basic multimodal large model, the first recording content classification scenario plugin model, and the second recording content classification scenario plugin model.

[0079] Indicatively, the first and second initial recording content classification scenario plugin models can be created based on machine learning models.

[0080] Optionally, the conditions for ending model training may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold. Specific conditions for ending model training can be determined based on the actual situation and are not specifically limited here.

[0081] It should be noted that the machine learning models involved in one or more embodiments of this specification include, but are not limited to, fitting of one or more of the following machine learning models: Convolutional Neural Network (CNN) model, Deep Neural Network (DNN) model, Recurrent Neural Networks (RNN) model, embedding model, Gradient Boosting Decision Tree (GBDT) model, Logistic Regression (LR) model, etc.

[0082] S206, if the semantic classification ambiguity detection result is that there is no ambiguous type, then the target knowledge base corresponding to the recording data to be classified is determined based on the probability distribution data.

[0083] Understandably, when the semantic classification ambiguity detection result indicates no ambiguity type, the large-scale model representing the recording content classification can identify the knowledge base category to which the recording data to be classified belongs through textual semantic features. Therefore, the candidate knowledge base corresponding to the maximum probability in the probability distribution data is determined, and this candidate knowledge base is identified as the target knowledge base corresponding to the recording data to be classified.

[0084] Optionally, after determining the target knowledge base to which the audio recording data to be classified belongs, the audio recording data to be classified and the target text content can be stored together in the target knowledge base to facilitate the reusability of the audio recording data to be classified and the target text content.

[0085] In this embodiment, the original text content is obtained by converting the audio recording data to be classified into speech and text. The original text content is then subjected to structured semantic extraction to obtain the target text content, ensuring the accuracy of the subsequently extracted text semantic features. The probability distribution data of the audio recording data to be classified belonging to each candidate knowledge base is determined by the text semantic features. The semantic classification ambiguity detection result is determined based on the probability distribution data. If the semantic classification ambiguity detection result indicates that there is an ambiguous type, the text semantic features, acoustic environment features, and user context features are multimodally weighted and fused to obtain a multimodal comprehensive feature vector. The target knowledge base corresponding to the target text content is then identified based on the multimodal comprehensive feature vector. If the semantic classification ambiguity detection result indicates that there is no ambiguous type, the target knowledge base corresponding to the audio recording data to be classified is determined based on the probability distribution data. Therefore, when the knowledge base category to which the recording data belongs can be identified based on text semantic features, there is no need to combine acoustic environment features and user context features for secondary identification of the knowledge base to which the recording data belongs, thereby improving the efficiency of recording data classification. However, when the knowledge base category to which the recording data belongs cannot be identified based on text semantic features, the knowledge base to which the recording data belongs is identified by combining acoustic environment features and user context features for secondary identification. The knowledge base with the highest category fit is then assigned to the recording data to be classified, ensuring the accuracy of recording data classification without requiring manual classification by the user.

[0086] The following will combine Figure 4 This application provides a detailed description of the audio recording processing apparatus provided in its embodiments. It should be noted that... Figure 4 The audio recording processing apparatus shown is used to execute this application. Figures 1-3 The methods shown in the embodiments are for illustrative purposes only, illustrating the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figures 1-3 The example shown.

[0087] Please see Figure 4 This diagram illustrates the structure of a recording content processing apparatus according to an embodiment of this application. The recording content processing apparatus 40 can be implemented as all or part of the apparatus through software, hardware, or a combination of both. According to some embodiments, the recording content processing apparatus 40 includes a data acquisition module 410, a feature extraction module 420, and a data classification module 430, specifically used for: Data acquisition module 410 is used to determine the target text content of the audio recording data to be classified; The feature extraction module 420 is used to extract text semantic features based on the target text content using a large recording content classification model, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. The data classification module 430 is used to perform knowledge base archiving and classification processing on the text content based on the text semantic features, the acoustic environment features and the user context features to obtain the target knowledge base corresponding to the recording data to be classified.

[0088] Optionally, the data classification module 430 includes: The first classification processing unit is used to determine the probability distribution data of the recording data to be classified belonging to each candidate knowledge base based on the semantic features of the text. The second classification processing unit is used to determine the semantic classification ambiguity detection result based on the probability distribution data; The third classification processing unit is used to perform multimodal weighted fusion of the text semantic features, the acoustic environment features and the user context features to obtain a multimodal comprehensive feature vector if the semantic classification ambiguity detection result is that there is an ambiguity type, and to identify the target knowledge base corresponding to the recording data to be classified based on the multimodal comprehensive feature vector; The fourth classification processing unit is used to determine the target knowledge base corresponding to the recording data to be classified based on the probability distribution data if the semantic classification ambiguity detection result is that there is no ambiguity type.

[0089] Optionally, the second classification processing unit is specifically used for: Based on the probability distribution data, a semantic uncertainty index is determined; If the semantic uncertainty index is greater than or equal to a preset threshold, a semantic classification ambiguity detection result with semantic ambiguity type is generated; If the semantic uncertainty index is less than a preset threshold, a semantic classification ambiguity detection result without semantic ambiguity type is generated.

[0090] Optionally, the third classification processing unit includes: The first fusion processing subunit is used to linearly project the acoustic environment features to obtain an acoustic query key vector and an acoustic value vector, and linearly project the user context features to obtain a context query key vector and a context value vector. The second fusion processing subunit is used to determine an acoustic correction term vector based on the text semantic features, the acoustic query key vector, and the acoustic value vector, and to determine a context correction term vector based on the text semantic features, the context query key vector, and the context value vector; The third fusion processing subunit is used to perform weighted fusion based on the text semantic features, the semantic uncertainty index, the acoustic correction term vector and the context correction term vector to obtain a multimodal comprehensive feature vector.

[0091] Optionally, the second fusion processing subunit is specifically used for: The acoustic correction term weights are obtained by performing an inner product operation based on the text semantic features and the acoustic query key vector, and the acoustic correction term vector is calculated based on the acoustic correction term weights and the acoustic value vector. The context correction term weight is obtained by performing an inner product operation based on the text semantic features and the context query key vector, and the context correction term vector is calculated based on the context correction term weight and the context value vector.

[0092] Optionally, the feature extraction module 420 is specifically used for: Based on the recording data to be classified, the characteristics of the speaker are identified, and acoustic parameters are extracted based on the recording data to be classified. Identify physical space environment fingerprints based on the audio recording data to be classified; Acoustic environment features are generated based on the interlocutor features, the acoustic parameters, and the physical space environment fingerprint.

[0093] Optionally, the feature extraction module 420 is specifically used for: The recorded audio data to be classified is processed into speech-to-text to obtain the original text content, and the original text content is then subjected to structured semantic extraction to obtain the target text content.

[0094] Please refer to Figure 5 This diagram illustrates the structure of an electronic device provided in an exemplary embodiment of this application. The electronic device in this embodiment may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0095] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of a central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0096] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems.

[0097] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0098] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 can be a touch display screen.

[0099] The touch display screen can be designed as a full-screen, curved screen, or irregularly shaped screen. It can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; however, this application does not limit the specific design of the touch display screen.

[0100] In addition, those skilled in the art will understand that the structure of the smart home device shown in the above figures does not constitute a limitation on the smart home device. The smart home device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the smart home device also includes radio frequency circuits, input units, sensors, audio circuits, Wireless Fidelity (Wi-Fi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0101] In some embodiments, Figure 5 The processor 110 in the illustrated electronic device can be used to call the program for processing the recorded content stored in the memory 120, and specifically perform the following operations: Determine the target text content of the audio recording data to be classified; The recording content classification model is used to extract text semantic features based on the target text content, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. Based on the text semantic features, the acoustic environment features, and the user context features, the text content is archived and classified using a knowledge base to obtain the target knowledge base corresponding to the recording data to be classified.

[0102] In one embodiment, when the processor 110 performs the knowledge base archiving and classification process on the target text content based on the text semantic features, the acoustic environment features, and the user context features to obtain the target knowledge base corresponding to the recording data to be classified, the processor 110 specifically performs the following operations: Based on the semantic features of the text, the probability distribution data of the audio recording data to be classified belonging to each candidate knowledge base is determined; The semantic classification ambiguity detection result is determined based on the probability distribution data; If the semantic classification ambiguity detection result indicates that there is an ambiguous type, then the text semantic features, the acoustic environment features, and the user context features are subjected to multimodal weighted fusion to obtain a multimodal comprehensive feature vector, and the target knowledge base corresponding to the recording data to be classified is identified based on the multimodal comprehensive feature vector; If the semantic classification ambiguity detection result is that there is no ambiguous type, then the target knowledge base corresponding to the recording data to be classified is determined based on the probability distribution data.

[0103] In one embodiment, when the processor 110 executes the process of determining the semantic classification ambiguity detection result based on the probability distribution data, it specifically performs the following operations: Based on the probability distribution data, a semantic uncertainty index is determined; If the semantic uncertainty index is greater than or equal to a preset threshold, a semantic classification ambiguity detection result with semantic ambiguity type is generated; If the semantic uncertainty index is less than a preset threshold, a semantic classification ambiguity detection result without semantic ambiguity type is generated.

[0104] In one embodiment, when the processor 110 performs multimodal weighted fusion of the text semantic features, the acoustic environment features, and the user context features to obtain a multimodal comprehensive feature vector, it specifically performs the following operations: Linear projection of the acoustic environment features yields acoustic query key vectors and acoustic value vectors, and linear projection of the user context features yields context query key vectors and context value vectors. An acoustic correction term vector is determined based on the text semantic features, the acoustic query key vector, and the acoustic value vector; a context correction term vector is determined based on the text semantic features, the context query key vector, and the context value vector. A multimodal comprehensive feature vector is obtained by weighted fusion of the text semantic features, the semantic uncertainty index, the acoustic correction term vector, and the context correction term vector.

[0105] In one embodiment, when the processor 110 executes the steps of determining the acoustic correction term vector based on the text semantic features, the acoustic query key vector, and the acoustic value vector, and determining the context correction term vector based on the text semantic features, the context query key vector, and the context value vector, it specifically performs the following operations: The acoustic correction term weights are obtained by performing an inner product operation based on the text semantic features and the acoustic query key vector, and the acoustic correction term vector is calculated based on the acoustic correction term weights and the acoustic value vector. The context correction term weight is obtained by performing an inner product operation based on the text semantic features and the context query key vector, and the context correction term vector is calculated based on the context correction term weight and the context value vector.

[0106] In one embodiment, when the processor 110 performs the extraction of acoustic environment features based on the recording data to be classified, it specifically performs the following operations: Based on the recording data to be classified, the characteristics of the speaker are identified, and acoustic parameters are extracted based on the recording data to be classified. Identify physical space environment fingerprints based on the audio recording data to be classified; Acoustic environment features are generated based on the interlocutor features, the acoustic parameters, and the physical space environment fingerprint.

[0107] In one embodiment, when the processor 110 executes the process of determining the target text content of the recording data to be classified, it specifically performs the following operations: The recorded audio data to be classified is processed into speech-to-text to obtain the original text content, and the original text content is then subjected to structured semantic extraction to obtain the target text content.

[0108] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor to implement the recording content processing method as described in the above embodiments.

[0109] This application also provides a computer program product that stores at least one instruction, which is loaded and executed by the processor to implement the recording content processing method described in the above embodiments.

[0110] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0111] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for processing audio recordings, characterized in that, The method includes: Determine the target text content of the audio recording data to be classified; The recording content classification model is used to extract text semantic features based on the target text content, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. Based on the text semantic features, the acoustic environment features, and the user context features, the text content is archived and classified using a knowledge base to obtain the target knowledge base corresponding to the recording data to be classified.

2. The method according to claim 1, characterized in that, The process of performing knowledge base archiving and classification on the target text content based on the text semantic features, the acoustic environment features, and the user context features to obtain the target knowledge base corresponding to the recording data to be classified includes: Based on the semantic features of the text, the probability distribution data of the audio recording data to be classified belonging to each candidate knowledge base is determined; The semantic classification ambiguity detection result is determined based on the probability distribution data; If the semantic classification ambiguity detection result indicates that there is an ambiguous type, then the text semantic features, the acoustic environment features, and the user context features are subjected to multimodal weighted fusion to obtain a multimodal comprehensive feature vector, and the target knowledge base corresponding to the recording data to be classified is identified based on the multimodal comprehensive feature vector; If the semantic classification ambiguity detection result is that there is no ambiguous type, then the target knowledge base corresponding to the recording data to be classified is determined based on the probability distribution data.

3. The method according to claim 2, characterized in that, The determination of semantic classification ambiguity detection results based on the probability distribution data includes: Based on the probability distribution data, a semantic uncertainty index is determined; If the semantic uncertainty index is greater than or equal to a preset threshold, a semantic classification ambiguity detection result with semantic ambiguity type is generated; If the semantic uncertainty index is less than a preset threshold, a semantic classification ambiguity detection result without semantic ambiguity type is generated.

4. The method according to claim 3, characterized in that, The process of multimodal weighted fusion of the text semantic features, the acoustic environment features, and the user context features to obtain a multimodal comprehensive feature vector includes: Linear projection of the acoustic environment features yields acoustic query key vectors and acoustic value vectors, and linear projection of the user context features yields context query key vectors and context value vectors. An acoustic correction term vector is determined based on the text semantic features, the acoustic query key vector, and the acoustic value vector; a context correction term vector is determined based on the text semantic features, the context query key vector, and the context value vector. A multimodal comprehensive feature vector is obtained by weighted fusion of the text semantic features, the semantic uncertainty index, the acoustic correction term vector, and the context correction term vector.

5. The method according to claim 4, characterized in that, The step of determining the acoustic correction term vector based on the text semantic features, the acoustic query key vector, and the acoustic value vector, and determining the context correction term vector based on the text semantic features, the context query key vector, and the context value vector, includes: The acoustic correction term weights are obtained by performing an inner product operation based on the text semantic features and the acoustic query key vector, and the acoustic correction term vector is calculated based on the acoustic correction term weights and the acoustic value vector. The context correction term weight is obtained by performing an inner product operation based on the text semantic features and the context query key vector, and the context correction term vector is calculated based on the context correction term weight and the context value vector.

6. The method according to claim 1, characterized in that, The extraction of acoustic environment features based on the recording data to be classified includes: Based on the recording data to be classified, the characteristics of the speaker are identified, and acoustic parameters are extracted based on the recording data to be classified. Identify physical space environment fingerprints based on the audio recording data to be classified; Acoustic environment features are generated based on the interlocutor features, the acoustic parameters, and the physical space environment fingerprint.

7. The method according to claim 1, characterized in that, The determination of the target text content of the audio recording data to be classified includes: The recorded audio data to be classified is processed into speech-to-text to obtain the original text content, and the original text content is then subjected to structured semantic extraction to obtain the target text content.

8. A recording content processing device, characterized in that, The device includes: The data acquisition module is used to determine the target text content of the audio recording data to be classified; The feature extraction module is used to extract text semantic features based on the target text content using a large recording content classification model, extract acoustic environment features based on the recording data to be classified, and extract user context features based on the acquisition behavior data of the recording data to be classified. The data classification module is used to perform knowledge base archiving and classification processing on the text content based on the text semantic features, the acoustic environment features, and the user context features to obtain the target knowledge base corresponding to the recording data to be classified.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, which are adapted to be loaded by a processor and executed as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1 to 7.

Images