A method and system for clustering and classifying audio data

By assisting users in selecting training samples of audio clustering classification models, using multimodal data recommendation tags and interactively determining the sample selection basis, the problem that audio data clustering classification results in the existing technology do not meet user expectations, and achieve more High user experience and system applicability.

CN118538235BActive Publication Date: 2024-11-15YANCHENG DONGYUE XINGXU NETWORK TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410600198.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-15
Publication Date
2024-11-15
Estimated Expiration
2044-05-15

AI Technical Summary

Technical Problem

The existing audio data clustering and classification methods cannot meet users' personalized needs, resulting in the clustering and classification results that do not meet user expectations and reduce user experience.

Method used

Provide an audio data clustering and classification method, which helps users to personalize the training samples of the audio clustering and classification model, uses multimodal data to recommend clustering and classification tags, and interactively assists users to determine the basis for sample selection, train neural network models. Implement personalized clustering and classification of audio data.

Benefits of technology

It improves the accuracy of audio data clustering and classification results, meets user expectations, and greatly improves the user experience and system applicability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118538235B_ABST
    Figure CN118538235B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for clustering and classifying audio data, wherein the method includes: assisting users in personalized selection of audio clustering and classification model training samples; when the selection is completed, using the audio clustering and classification model training samples to train a neural network model to obtain an audio clustering and classification model; obtaining audio data to be clustered and classified; using the audio clustering and classification model to cluster and classify the audio data, and outputting the clustering and classification results. The method and system for clustering and classifying audio data of the present invention provide a way for users to personalized select audio clustering and classification model training samples for the audio clustering and classification model, and users can implement it in combination with their own audio clustering and classification expectations, so that the clustering and classification results of clustering and classifying audio data using the audio clustering and classification model meet the user's audio clustering and classification expectations, greatly improving the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a clustering classification method and system for audio data. Background Art

[0002] At present, there are more and more demands and usage scenarios for clustering and classification of audio data. For example, music platforms need to classify the newly released song audios every month, and radio stations need to classify the newly recorded program audios every month.

[0003] However, in general, when clustering and classifying audio data, training samples (audio data marked with audio category labels) are mostly used to train audio clustering classification models, and the trained audio clustering classification model is directly used to cluster and classify the audio data input by the user. For example: the invention patent with application number CN202111594903.3 discloses an audio data clustering method, device, equipment and storage medium, wherein the audio data clustering method includes: obtaining multiple audio clips to be processed; extracting features for each audio clip to obtain corresponding voiceprint features; inputting the voiceprint features corresponding to each audio clip into a graph neural network to obtain a feature vector representation of each audio clip; clustering each audio clip based on the feature vector representation of each audio clip. This method directly uses a pre-trained graph neural network to cluster audio clips according to the voiceprint features of the audio clips.

[0004] However, doing so may result in the clustering classification results of the audio data not meeting the user's audio clustering classification expectations. The user cannot make personalized selections of training samples for training the audio clustering classification model based on their own audio clustering classification expectations, resulting in an audio clustering classification model that is not trained to meet the user's expectations, which reduces the user experience.

[0005] Therefore, a solution is urgently needed. Summary of the invention

[0006] One of the purposes of the present invention is to provide a clustering classification method for audio data, providing a way for users to personalize the selection of audio clustering classification model training samples for the audio clustering classification model, and the user can implement it in combination with his or her own audio clustering classification expectations, so that the clustering classification results of clustering and classifying audio data using the audio clustering classification model meet the user's audio clustering classification expectations, thereby greatly improving the user experience.

[0007] An embodiment of the present invention provides a clustering classification method for audio data, comprising:

[0008] Assist users to select audio clustering classification model training samples in a personalized way;

[0009] When the selection is completed, the audio clustering classification model training samples are used to train the neural network model to obtain the audio clustering classification model;

[0010] Obtain audio data to be clustered and classified;

[0011] Use the audio clustering classification model to cluster and classify the audio data, and output the clustering classification results.

[0012] Preferably, assisting the user to individually select audio clustering classification model training samples includes:

[0013] Obtain multimodal data of users;

[0014] Based on the multimodal data, a recommended cluster classification label sequence is determined; the recommended cluster classification label sequence includes a first recommended cluster classification label sorted from large to small according to the recommendation degree;

[0015] Displaying a first information window to the user; the first information window scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in descending order according to the recommendation degree;

[0016] Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window;

[0017] Based on the sample selection criteria, the audio clustering classification model training samples are determined.

[0018] Preferably, the interactively assisting user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window includes:

[0019] Continuously obtain the user's field of view;

[0020] Constructing a first direction vector, a second direction vector and a third direction vector of the first information window; wherein the first direction vector is constructed based on the window center position of the first information window and the scrolling display direction of the first information window; the second direction vector is constructed based on the window center position of the first information window and the opposite direction of the scrolling display direction of the first information window; the third direction vector is constructed based on the window center position of the first information window and a first vertical direction perpendicular to the scrolling display direction of the first information window;

[0021] When there are both overlapping areas and non-overlapping areas in the sight area and the first information window, determine that the non-overlapping area is greater than the area difference of the overlapping area;

[0022] When the area difference is greater than or equal to the area difference threshold, determining a fourth direction vector based on the region center position of the overlapping region and the direction of a line from the region center position of the overlapping region to the region center position of the non-overlapping region;

[0023] When the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, controlling the first information window to scroll forward to display the first recommended cluster classification label;

[0024] When the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, controlling the first information window to scroll in reverse to display the first recommended cluster classification label;

[0025] When the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0026] Preferably, the clustering classification method for audio data further includes:

[0027] Constructing a fifth direction vector of the first information window; wherein the fifth direction vector is constructed based on a window center position of the first information window and a second vertical direction perpendicular to a scrolling display direction of the first information window;

[0028] When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for the extended recommendation;

[0029] Based on the extended recommendation basis, determine the second recommended cluster classification label;

[0030] Displaying a second information window to the user; the second information window partially overlaps with the non-overlapping area, and all second recommended cluster classification labels are statically displayed in the second information window;

[0031] When the user's new visual area includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification labels in sequence;

[0032] When the vector angle between the new fourth direction vector and the third direction vector is smaller than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0033] Preferably, determining the second recommended cluster classification label based on the extended recommendation basis includes:

[0034] Determine the label cluster to which the extended recommendation basis belongs from the recommended cluster classification label sequence; the first recommended cluster classification labels in the same label cluster are arranged adjacent to each other in sequence and belong to the same parent label;

[0035] When the first information window has historically scrolled to display each first recommended cluster classification label in the label cluster and then reversely scrolled to display at least one first recommended cluster classification label in the label cluster, the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label.

[0036] An embodiment of the present invention provides a clustering classification system for audio data, including:

[0037] A first user assistance module, used to assist users in selecting audio clustering classification model training samples in a personalized manner;

[0038] A model training module is used to train the neural network model using the audio clustering classification model training samples to obtain the audio clustering classification model when the selection is completed;

[0039] A data acquisition module, used to acquire audio data to be clustered and classified;

[0040] The clustering classification module is used to cluster and classify audio data using the audio clustering classification model and output the clustering classification results.

[0041] Preferably, the first user assistance module assists the user in selecting the audio clustering classification model training sample in a personalized manner, including:

[0042] Obtain multimodal data of users;

[0043] Based on the multimodal data, a recommended cluster classification label sequence is determined; the recommended cluster classification label sequence includes a first recommended cluster classification label sorted from large to small according to the recommendation degree;

[0044] Displaying a first information window to the user; the first information window scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in descending order according to the recommendation degree;

[0045] Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window;

[0046] Based on the sample selection criteria, the audio clustering classification model training samples are determined.

[0047] Preferably, the first user assistance module interactively assists the user in determining the sample selection basis from the first recommended cluster classification label by viewing the first information window, including:

[0048] Continuously obtain the user's field of view;

[0049] Constructing a first direction vector, a second direction vector and a third direction vector of the first information window; wherein the first direction vector is constructed based on the window center position of the first information window and the scrolling display direction of the first information window; the second direction vector is constructed based on the window center position of the first information window and the opposite direction of the scrolling display direction of the first information window; the third direction vector is constructed based on the window center position of the first information window and a first vertical direction perpendicular to the scrolling display direction of the first information window;

[0050] When there are both overlapping areas and non-overlapping areas in the sight area and the first information window, determine that the non-overlapping area is greater than the area difference of the overlapping area;

[0051] When the area difference is greater than or equal to the area difference threshold, determining a fourth direction vector based on the region center position of the overlapping region and the direction of a line from the region center position of the overlapping region to the region center position of the non-overlapping region;

[0052] When the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, controlling the first information window to scroll forward to display the first recommended cluster classification label;

[0053] When the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, controlling the first information window to scroll in reverse to display the first recommended cluster classification label;

[0054] When the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0055] Preferably, the clustering classification system for audio data further includes:

[0056] The second user assistance module is used to include:

[0057] Constructing a fifth direction vector of the first information window; wherein the fifth direction vector is constructed based on a window center position of the first information window and a second vertical direction perpendicular to a scrolling display direction of the first information window;

[0058] When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for the extended recommendation;

[0059] Based on the extended recommendation basis, determine the second recommended cluster classification label;

[0060] Displaying a second information window to the user; the second information window partially overlaps with the non-overlapping area, and all second recommended cluster classification labels are statically displayed in the second information window;

[0061] When the user's new visual area includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification labels in sequence;

[0062] When the vector angle between the new fourth direction vector and the third direction vector is smaller than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0063] Preferably, the second user assistance module determines the second recommended cluster classification label based on the extended recommendation basis, including:

[0064] Determine the label cluster to which the extended recommendation basis belongs from the recommended cluster classification label sequence; the first recommended cluster classification labels in the same label cluster are arranged adjacent to each other in sequence and belong to the same parent label;

[0065] When the first information window has historically scrolled to display each first recommended cluster classification label in the label cluster and then reversely scrolled to display at least one first recommended cluster classification label in the label cluster, the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label.

[0066] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0067] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0069] Figure 1 A schematic diagram of a clustering classification method for audio data in an embodiment of the present invention;

[0070] Figure 2 Schematic diagram of a clustering classification system for audio data in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0072] The embodiment of the present invention provides a clustering classification method for audio data, such as Figure 1As shown, including:

[0073] S1, assisting users to select audio clustering classification model training samples in a personalized manner;

[0074] S2. When the selection is completed, the audio clustering classification model training samples are used to train the neural network model to obtain the audio clustering classification model;

[0075] S3, obtaining audio data to be clustered and classified;

[0076] S4. Use the audio clustering classification model to cluster and classify the audio data, and output the clustering classification results.

[0077] The user can make personalized selections of audio clustering classification model training samples for training the audio clustering classification model based on his or her own expected audio clustering classification requirements. For example, if the user wants to classify songs according to the emotion categories expressed by the songs, then when selecting the audio clustering classification model training samples, the user can select audio data marked with the emotion categories expressed by the songs as the audio clustering classification model training samples. In this way, the audio clustering classification model trained using the audio clustering classification model training samples can classify songs according to the emotion categories expressed by the songs in a targeted manner. When the user personally selects the audio clustering classification model training samples, the system will provide assistance to improve the efficiency of the personalized selection. Using training samples to train the neural network model belongs to the scope of the prior art and will not be elaborated on. Using the audio clustering classification model, the audio data is clustered and classified, and the clustering classification results are output. The audio data can be input by the user, that is, the original audio data that needs to be clustered and classified is input.

[0078] The present application provides a way for users to personalize the selection of audio clustering classification model training samples for the audio clustering classification model. Users can implement it in combination with their own audio clustering classification expectations, so that the clustering classification results of clustering and classifying audio data using the audio clustering classification model meet the user's audio clustering classification expectations, thereby greatly improving the user experience.

[0079] In one embodiment, assisting a user to individually select an audio clustering classification model training sample includes:

[0080] Obtain multimodal data of users;

[0081] Based on the multimodal data, a recommended cluster classification label sequence is determined; the recommended cluster classification label sequence includes a first recommended cluster classification label sorted from large to small according to the recommendation degree;

[0082] Displaying a first information window to the user; the first information window scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in descending order according to the recommendation degree;

[0083] Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window;

[0084] Based on the sample selection criteria, the audio clustering classification model training samples are determined.

[0085] Multimodal data includes: the industry to which the user belongs (for example, music platform administrator), the user's historical cluster classification labels (for example, sad music labels and cheerful music labels that have been selected in the past), etc.; based on the multimodal data, a first recommended cluster classification label is recommended to the user, and the recommendation degree of the first recommended cluster classification label is determined. For example, if the user's industry is a music platform administrator, and the user has selected the sad music label 10 times and the cheerful music label 7 times in the past, then the first recommended cluster classification label is determined to be a sad music label with a recommendation degree of 9, and it can also be a cheerful music label with a recommendation degree of 8; the recommendation mechanism and the method for determining the recommendation degree can be set in advance by technical personnel according to actual needs; the first information The window is an information display window, which scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in order from large to small according to the recommendation degree; when the user views the first information window, the user is interactively assisted to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window; the sample selection basis is the first recommended cluster classification label that the user wants to adopt; when the user adopts the first recommended cluster classification label, that is, as the sample selection basis, the audio cluster classification model training samples related to the sample selection basis can be retrieved to complete the determination, for example: the sample selection basis is classified according to the sad music category, then the audio cluster classification model training samples are a large amount of audio data marked with the sad music category. The embodiment of the present invention interactively assists the user in personalizing the selection of audio cluster classification model training samples, improves the efficiency of determining the audio cluster classification model training samples, improves the user experience, and improves the applicability of the system.

[0086] In one embodiment, the interactive assisting user determines the sample selection basis from the first recommended cluster classification label by viewing the first information window, including:

[0087] Continuously obtain the user's field of view;

[0088] Constructing a first direction vector, a second direction vector and a third direction vector of the first information window; wherein the first direction vector is constructed based on the window center position of the first information window and the scrolling display direction of the first information window; the second direction vector is constructed based on the window center position of the first information window and the opposite direction of the scrolling display direction of the first information window; the third direction vector is constructed based on the window center position of the first information window and a first vertical direction perpendicular to the scrolling display direction of the first information window;

[0089] When there are both overlapping areas and non-overlapping areas in the sight area and the first information window, determine that the non-overlapping area is greater than the area difference of the overlapping area;

[0090] When the area difference is greater than or equal to the area difference threshold, determining a fourth direction vector based on the region center position of the overlapping region and the direction of a line from the region center position of the overlapping region to the region center position of the non-overlapping region;

[0091] When the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, controlling the first information window to scroll forward to display the first recommended cluster classification label;

[0092] When the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, controlling the first information window to scroll in reverse to display the first recommended cluster classification label;

[0093] When the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0094] The user uses the display terminal to view the first information window, and the first information window is not displayed in full screen on the display terminal. Therefore, when the user views the first information window, there are both overlapping areas and non-overlapping areas in the user's sight area with the first information window. When the area difference between the non-overlapping area and the overlapping area is greater than or equal to an area difference threshold (for example, 5 square centimeters), it indicates that the user wants to achieve information interaction with the first information window based on the sight line; construct the first direction vector, the second direction vector, and the third direction vector of the first information window respectively; the scrolling display direction of the first information window will have two vertical directions, and one vertical direction is selected for constructing the third direction vector; when the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, control The first information window scrolls forward to display the first recommended cluster classification label, that is, it continues to scroll in the original scroll direction (according to the recommended value from large to small); when the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, the first information window is controlled to scroll in the reverse direction to display the first recommended cluster classification label, that is, the first information window is controlled to scroll in the direction of the original scroll direction, and the first recommended cluster classification label that has just been scrolled and displayed will be scrolled again; when the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the sample selection basis, indicating that the user indicates that the first recommended cluster classification label currently scrolled and displayed is adopted. Specifically, for example: when the scrolling display direction of the first information window is left, the user only needs to look towards the left area in the first information box, and the first information window will continue to scroll and display. If the first information window needs to scroll in the opposite direction, it is viewed towards the right area in the first information box. The first vertical direction can be upward. The user looks towards the upper area in the first information box, and the system will confirm that the first recommended cluster classification label currently scrolled and displayed is used as the sample selection basis. It has greatly improved the convenience of determining the basis for sample selection and further enhanced the user experience.

[0095] In one embodiment, the method for clustering and classifying audio data further includes:

[0096] Constructing a fifth direction vector of the first information window; wherein the fifth direction vector is constructed based on a window center position of the first information window and a second vertical direction perpendicular to a scrolling display direction of the first information window;

[0097] When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for the extended recommendation;

[0098] Based on the extended recommendation basis, determine the second recommended cluster classification label;

[0099] Displaying a second information window to the user; the second information window partially overlaps with the non-overlapping area, and all second recommended cluster classification labels are statically displayed in the second information window;

[0100] When the user's new visual area includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification labels in sequence;

[0101] When the vector angle between the new fourth direction vector and the third direction vector is smaller than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0102] When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for extended recommendation. For example, if the second vertical direction is downward, the user only needs to look towards the lower area in the first information frame, and the system will automatically use the first recommended cluster classification label currently scrolled and displayed in the first information window as the basis for extended recommendation. Based on the extended recommendation basis, the recommendation extension is performed to determine the second recommended cluster classification label. The second information window is an information display window. It is ensured that the second information window partially overlaps with the non-overlapping area, and all the second recommended cluster classification labels are statically displayed in the second information window, so that the second information window can be close to the user's line of sight and will not block the first information window. When the user newly views the second information window, if the user's new line of sight includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification label in sequence, that is, the first information window is controlled to temporarily jump in line and scroll and display the second recommended cluster classification label in sequence. At this time, the fourth direction vector will also be updated. When the vector angle between the new fourth direction vector and the third direction vector is less than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for sample selection. The embodiment of the present invention can perform recommendation expansion to assist the user in selecting the expanded second recommended cluster classification label, thereby further improving the user experience.

[0103] In one embodiment, determining the second recommended cluster classification label based on the extended recommendation basis includes:

[0104] Determine the label cluster to which the extended recommendation basis belongs from the recommended cluster classification label sequence; the first recommended cluster classification labels in the same label cluster are arranged adjacent to each other in sequence and belong to the same parent label;

[0105] When the first information window has historically scrolled to display each first recommended cluster classification label in the label cluster and then reversely scrolled to display at least one first recommended cluster classification label in the label cluster, the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label.

[0106] The first recommended cluster classification label in the same tag cluster is the subordinate cluster classification label of the parent label. For example, if the parent label is classified according to sentimental emotions, the first recommended cluster classification labels in the corresponding tag cluster are loneliness, anger, etc.; when the first information window scrolls and displays each first recommended cluster classification label in the tag cluster and then reversely scrolls and displays at least one first recommended cluster classification label in the tag cluster, it means that the user wants to select the first recommended cluster classification label in the tag cluster for category refinement, then the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label; there are subordinate cluster classification labels corresponding to different extended recommendation bases in the subordinate cluster classification label library. The embodiment of the present invention adaptively determines the second recommended cluster classification label based on the extended recommendation basis, which improves the applicability of the system and further improves the user experience.

[0107] The embodiment of the present invention provides a clustering classification system for audio data, such as Figure 2 As shown, including:

[0108] A first user assistance module 1, used to assist the user in selecting audio clustering classification model training samples in a personalized manner;

[0109] Model training module 2, used for training the neural network model using the audio clustering classification model training samples to obtain the audio clustering classification model when the selection is completed;

[0110] Data acquisition module 3, used to acquire audio data to be clustered and classified;

[0111] The clustering classification module 4 is used to perform clustering classification on the audio data using the audio clustering classification model and output the clustering classification result.

[0112] The first user assistance module 1 assists the user in selecting audio clustering classification model training samples in a personalized manner, including:

[0113] Obtain multimodal data of users;

[0114] Based on the multimodal data, a recommended cluster classification label sequence is determined; the recommended cluster classification label sequence includes a first recommended cluster classification label sorted from large to small according to the recommendation degree;

[0115] Displaying a first information window to the user; the first information window scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in descending order according to the recommendation degree;

[0116] Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window;

[0117] Based on the sample selection criteria, the audio clustering classification model training samples are determined.

[0118] The first user assistance module 1 interactively assists the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window, including:

[0119] Continuously obtain the user's field of view;

[0120] Constructing a first direction vector, a second direction vector and a third direction vector of the first information window; wherein the first direction vector is constructed based on the window center position of the first information window and the scrolling display direction of the first information window; the second direction vector is constructed based on the window center position of the first information window and the opposite direction of the scrolling display direction of the first information window; the third direction vector is constructed based on the window center position of the first information window and a first vertical direction perpendicular to the scrolling display direction of the first information window;

[0121] When there are both overlapping areas and non-overlapping areas in the sight area and the first information window, determine that the non-overlapping area is greater than the area difference of the overlapping area;

[0122] When the area difference is greater than or equal to the area difference threshold, determining a fourth direction vector based on the region center position of the overlapping region and the direction of a line from the region center position of the overlapping region to the region center position of the non-overlapping region;

[0123] When the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, controlling the first information window to scroll forward to display the first recommended cluster classification label;

[0124] When the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, controlling the first information window to scroll in reverse to display the first recommended cluster classification label;

[0125] When the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0126] The clustering classification system for audio data also includes:

[0127] The second user assistance module is used to include:

[0128] Constructing a fifth direction vector of the first information window; wherein the fifth direction vector is constructed based on a window center position of the first information window and a second vertical direction perpendicular to a scrolling display direction of the first information window;

[0129] When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for the extended recommendation;

[0130] Based on the extended recommendation basis, determine the second recommended cluster classification label;

[0131] Displaying a second information window to the user; the second information window partially overlaps with the non-overlapping area, and all second recommended cluster classification labels are statically displayed in the second information window;

[0132] When the user's new visual area includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification labels in sequence;

[0133] When the vector angle between the new fourth direction vector and the third direction vector is smaller than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

[0134] The second user assistance module determines the second recommended cluster classification label based on the extended recommendation basis, including:

[0135] Determine the label cluster to which the extended recommendation basis belongs from the recommended cluster classification label sequence; the first recommended cluster classification labels in the same label cluster are arranged adjacent to each other in sequence and belong to the same parent label;

[0136] When the first information window has historically scrolled to display each first recommended cluster classification label in the label cluster and then reversely scrolled to display at least one first recommended cluster classification label in the label cluster, the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A clustering classification method for audio data, characterized in that: include: Assist users to select audio clustering classification model training samples in a personalized way; When the selection is completed, the audio clustering classification model training samples are used to train the neural network model to obtain the audio clustering classification model; Obtain audio data to be clustered and classified; Use the audio clustering classification model to cluster and classify the audio data, and output the clustering classification results; Assist users to select audio clustering classification model training samples in a personalized way, including: Obtain multimodal data of users; Based on the multimodal data, a recommended cluster classification label sequence is determined; the recommended cluster classification label sequence includes a first recommended cluster classification label sorted from large to small according to the recommendation degree; Displaying a first information window to the user; the first information window scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in descending order according to the recommendation degree; Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window; Based on the sample selection criteria, determine the audio clustering classification model training samples; Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window, including: Continuously obtain the user's field of view; Constructing a first direction vector, a second direction vector and a third direction vector of the first information window; wherein the first direction vector is constructed based on the window center position of the first information window and the scrolling display direction of the first information window; the second direction vector is constructed based on the window center position of the first information window and the opposite direction of the scrolling display direction of the first information window; the third direction vector is constructed based on the window center position of the first information window and a first vertical direction perpendicular to the scrolling display direction of the first information window; When there are both overlapping areas and non-overlapping areas in the sight area and the first information window, determine that the non-overlapping area is greater than the area difference of the overlapping area; When the area difference is greater than or equal to the area difference threshold, determining a fourth direction vector based on the region center position of the overlapping region and the direction of a line from the region center position of the overlapping region to the region center position of the non-overlapping region; When the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, controlling the first information window to scroll forward to display the first recommended cluster classification label; When the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, controlling the first information window to scroll in reverse to display the first recommended cluster classification label; When the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

2. The clustering classification method of audio data according to claim 1, characterized in that: Also includes: Constructing a fifth direction vector of the first information window; wherein the fifth direction vector is constructed based on a window center position of the first information window and a second vertical direction perpendicular to a scrolling display direction of the first information window; When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for the extended recommendation; Based on the extended recommendation basis, determine the second recommended cluster classification label; Displaying a second information window to the user; the second information window partially overlaps with the non-overlapping area, and all second recommended cluster classification labels are statically displayed in the second information window; When the user's new visual area includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification labels in sequence; When the vector angle between the new fourth direction vector and the third direction vector is smaller than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

3. The clustering classification method of audio data according to claim 2, characterized in that: The determining the second recommended cluster classification label based on the extended recommendation basis includes: Determine the label cluster to which the extended recommendation basis belongs from the recommended cluster classification label sequence; the first recommended cluster classification labels in the same label cluster are arranged adjacent to each other in sequence and belong to the same parent label; When the first information window has historically scrolled to display each first recommended cluster classification label in the label cluster and then reversely scrolled to display at least one first recommended cluster classification label in the label cluster, the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label.

4. A clustering classification system for audio data, characterized in that: include: A first user assistance module, used to assist users in selecting audio clustering classification model training samples in a personalized manner; A model training module is used to train the neural network model using the audio clustering classification model training samples to obtain the audio clustering classification model when the selection is completed; A data acquisition module, used to acquire audio data to be clustered and classified; The clustering classification module is used to cluster and classify the audio data using the audio clustering classification model and output the clustering classification results; The first user assistance module assists the user in selecting audio clustering classification model training samples in a personalized manner, including: Obtain multimodal data of users; Based on the multimodal data, a recommended cluster classification label sequence is determined; the recommended cluster classification label sequence includes a first recommended cluster classification label sorted from large to small according to the recommendation degree; Displaying a first information window to the user; the first information window scrolls and displays the first recommended cluster classification label in the recommended cluster classification label sequence in descending order according to the recommendation degree; Interactively assisting the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window; Based on the sample selection criteria, determine the audio clustering classification model training samples; The first user assistance module interactively assists the user to determine the sample selection basis from the first recommended cluster classification label by viewing the first information window, including: Continuously obtain the user's field of view; Constructing a first direction vector, a second direction vector and a third direction vector of the first information window; wherein the first direction vector is constructed based on the window center position of the first information window and the scrolling display direction of the first information window; the second direction vector is constructed based on the window center position of the first information window and the opposite direction of the scrolling display direction of the first information window; the third direction vector is constructed based on the window center position of the first information window and a first vertical direction perpendicular to the scrolling display direction of the first information window; When there are both overlapping areas and non-overlapping areas in the sight area and the first information window, determine that the non-overlapping area is greater than the area difference of the overlapping area; When the area difference is greater than or equal to the area difference threshold, determining a fourth direction vector based on the region center position of the overlapping region and the direction of a line from the region center position of the overlapping region to the region center position of the non-overlapping region; When the vector angle between the fourth direction vector and the first direction vector is less than the angle threshold, controlling the first information window to scroll forward to display the first recommended cluster classification label; When the vector angle between the fourth direction vector and the second direction vector is less than the angle threshold, controlling the first information window to scroll in reverse to display the first recommended cluster classification label; When the vector angle between the fourth direction vector and the third direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

5. The audio data clustering classification system according to claim 4, characterized in that: Also includes: The second user assistance module is used to include: Constructing a fifth direction vector of the first information window; wherein the fifth direction vector is constructed based on a window center position of the first information window and a second vertical direction perpendicular to a scrolling display direction of the first information window; When the vector angle between the fourth direction vector and the fifth direction vector is less than the angle threshold, the first recommended cluster classification label currently scrolled and displayed in the first information window is used as the basis for the extended recommendation; Based on the extended recommendation basis, determine the second recommended cluster classification label; Displaying a second information window to the user; the second information window partially overlaps with the non-overlapping area, and all second recommended cluster classification labels are statically displayed in the second information window; When the user's new visual area includes the second information window, the first information window is controlled to jump in line and scroll and display the second recommended cluster classification labels in sequence; When the vector angle between the new fourth direction vector and the third direction vector is smaller than the angle threshold, the second recommended cluster classification label currently scrolled and displayed in the first information window is used as a basis for sample selection.

6. The clustering classification system for audio data as claimed in claim 5, characterized in that: The second user assistance module determines the second recommended cluster classification label based on the extended recommendation basis, including: Determine the label cluster to which the extended recommendation basis belongs from the recommended cluster classification label sequence; the first recommended cluster classification labels in the same label cluster are arranged adjacent to each other in sequence and belong to the same parent label; When the first information window has historically scrolled to display each first recommended cluster classification label in the label cluster and then reversely scrolled to display at least one first recommended cluster classification label in the label cluster, the subordinate cluster classification label corresponding to the extended recommendation basis is determined from the subordinate cluster classification label library and used as the second recommended cluster classification label.