Multi-modal multi-task large model training method and system and medium

Abstract

The invention provides a multi-modal multi-task large model training method and system and a medium. The method comprises the steps of collecting music multi-modal data, performing model training according to the music multi-modal data to obtain a user music preference model, testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data, processing according to the model universality test data to obtain a model universality evaluation index, processing according to the model calculation efficiency data to obtain a calculation efficiency evaluation index, processing according to the model universality evaluation index and the calculation efficiency evaluation index to obtain a model performance evaluation index, and obtaining corresponding model correction strategy data; therefore, the purpose of improving the model training precision and the calculation efficiency is achieved.

Description

Technical Field

[0001] The present application relates to the field of model training technology, and more specifically, to a training method, system and medium for a multi-modal, multi-task large model. Background Art

[0002] With the booming music industry and the widespread popularity of digital music platforms, users' demand for personalized music experience is increasing day by day. Although music itself has rich multimodal resources covering audio waveforms, music spectra, lyrics texts, and users' diverse behaviors, most existing music recommendation systems and music analysis methods rely on a single modality or a simple combination of modalities, and cannot fully explore and utilize the comprehensive information contained in multimodal data. In addition, in practical applications, music-related tasks are often diverse and complex, such as music classification (classified by genre, age, emotion, etc.), music recommendation, music style conversion, etc. At present, it is usually necessary to train multiple independent models for these different tasks, which not only leads to high training costs and low efficiency, but also the inability to share knowledge and features between models, making it difficult to give full play to the synergy of multimodal data between different tasks. And with the continuous increase in model complexity and the continuous expansion of data scale, the computational cost and time cost of model training have also risen sharply. In practical application scenarios, how to efficiently train a multimodal multi-task model with excellent performance under limited computing resources and time constraints has become an important practical problem. Summary of the invention

[0003] The purpose of this application is to provide a multi-modal, multi-task large model training method, system and medium, which integrates music data of multiple modalities for model training, and conducts a comprehensive analysis of the performance indicators of the trained model in multiple dimensions such as accuracy, computational efficiency, and generalization ability of new data, so as to provide key feedback information for model training optimization, thereby achieving targeted model training optimization, improving the overall performance and practicality of the model, and helping to provide users with a more personalized and high-quality music service experience.

[0004] The present application also provides a method for training a multi-modal multi-task large model, comprising the following steps: Collect multimodal data related to music users, perform model training based on the multimodal data, and obtain a user music preference model; Testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data; Obtaining a model universality evaluation index according to the model universality test data processing; Obtaining a computational efficiency evaluation index according to the computational efficiency data of the model; The model performance evaluation index is obtained according to the model universality evaluation index and the computational efficiency evaluation index, and the corresponding model correction strategy data is obtained.

[0005] Optionally, in the training method of the multimodal multitask large model described in the present application, the collecting of multimodal data related to music users, performing model training according to the multimodal data, and obtaining the user music preference model include: The multimodal data includes user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data; Model training is performed based on the user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data to obtain a user music preference model.

[0006] Optionally, in the multimodal multitask large model training method described in the present application, the user music preference model is tested to obtain model test data, including model universality test data and model calculation efficiency data, including: The model universality test data includes model accuracy test data, cross-dataset learning accuracy data and task performance consistency data; The model accuracy test data includes music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model calculation efficiency data includes memory occupancy, CPU usage and model inference time.

[0007] Optionally, in the multi-modal multi-task large model training method described in the present application, the step of obtaining a model universality evaluation index according to the model universality test data processing includes: Obtaining a model accuracy evaluation index according to the music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model universality evaluation index is obtained based on the model accuracy evaluation index, cross-dataset learning accuracy data and task performance consistency data.

[0008] Optionally, in the training method of the multimodal multitask large model described in the present application, the step of obtaining a computational efficiency evaluation index according to the model computational efficiency data processing includes: A computational efficiency evaluation index is obtained based on the memory occupancy rate, CPU usage rate and model reasoning time.

[0009] Optionally, in the multimodal multitask large model training method described in the present application, the processing according to the model universality evaluation index and the computational efficiency evaluation index to obtain the model performance evaluation index, and obtaining the corresponding model correction strategy data, includes: Obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index; Comparing the model performance evaluation index with a preset model performance evaluation index threshold, and determining the model performance evaluation level according to the range level to which the threshold comparison result belongs; The model performance evaluation level is input into the preset model performance evaluation platform database for matching and identification to obtain corresponding model correction strategy data.

[0010] Optionally, the training method of the multi-modal multi-task large model described in the present application further includes: Recommend music to the user according to the user music preference model, and obtain the click rate, skip rate, number of plays and play time of the recommended music; Obtaining a recommendation effect evaluation index based on the click rate, skip rate, number of plays and play time; The recommendation effect evaluation index is compared with a preset recommendation effect evaluation index threshold. If the threshold comparison result does not meet the preset threshold comparison result requirement, feedback of poor model recommendation effect is given.

[0011] In a second aspect, the present application provides a training system for a multimodal multitask large model, the system comprising: a memory and a processor, the memory storing a program of a training method for a multimodal multitask large model, and the program of the training method for a multimodal multitask large model is executed by the processor to implement the following steps: Collect multimodal data related to music users, perform model training based on the multimodal data, and obtain a user music preference model; Testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data; Obtaining a model universality evaluation index according to the model universality test data processing; Obtaining a computational efficiency evaluation index according to the computational efficiency data of the model; The model performance evaluation index is obtained according to the model universality evaluation index and the computational efficiency evaluation index, and the corresponding model correction strategy data is obtained.

[0012] Optionally, in the multimodal multitask large model training system described in the present application, the collecting of multimodal data related to music users, performing model training according to the multimodal data, and obtaining the user music preference model include: The multimodal data includes user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data; Model training is performed based on the user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data to obtain a user music preference model.

[0013] In a third aspect, the present application also provides a computer-readable storage medium, which stores a training method program for a multimodal multi-task large model. When the training method program for a multimodal multi-task large model is executed by a processor, the steps of the training method for a multimodal multi-task large model as described in any one of the above items are implemented.

[0014] From the above, it can be seen that the present application provides a multi-modal, multi-task large model training method, system and medium, which integrates music data of multiple modalities for model training, and conducts a comprehensive analysis of the performance indicators of the trained model in multiple dimensions such as accuracy, computational efficiency, and generalization ability of new data, to provide key feedback information for model training optimization, thereby achieving targeted model training optimization, improving the overall performance and practicality of the model, and helping to provide users with a more personalized and high-quality music service experience.

[0015] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a method for training a multi-modal, multi-task large model provided in an embodiment of the present application; Figure 2 A flowchart of obtaining a user music preference model for a training method of a multi-modal multi-task large model provided in an embodiment of the present application; Figure 3 A flowchart of obtaining a model universality evaluation index for a training method of a multi-modal, multi-task large model provided in an embodiment of the present application; Figure 4A flowchart of obtaining corresponding model correction strategy data for the training method of the multimodal multi-task large model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0019] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0020] Please refer to Figure 1 , Figure 1 1 is a flowchart of a method for training a multimodal multitask large model in some embodiments of the present application. The method for training a multimodal multitask large model is used in a terminal device, such as a computer, a mobile phone terminal, etc. The method for training a multimodal multitask large model comprises the following steps: S11, collecting multimodal data related to music users, performing model training based on the multimodal data, and obtaining a user music preference model; S12, testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data; S13, obtaining a model universality evaluation index according to the model universality test data processing; S14, obtaining a calculation efficiency evaluation index according to the calculation efficiency data of the model; S15. Obtain a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index, and obtain corresponding model correction strategy data.

[0021] It should be noted that this application integrates music data in multiple modalities such as user basic information data, user operation record data, music playback history data, music basic attribute data, and music audio feature data to perform model training, and evaluates the versatility and computational efficiency of the trained model, thereby obtaining a comprehensive evaluation result of the model performance, and finally obtaining the corresponding model correction strategy data, thereby achieving the purpose of improving model training accuracy and computational efficiency.

[0022] Please refer to Figure 2 , Figure 2 The flowchart of the method for training a multimodal multitask large model in some embodiments of the present application for obtaining a user music preference model is shown in FIG. According to an embodiment of the present invention, the method of collecting multimodal data related to music users, performing model training based on the multimodal data, and obtaining a user music preference model includes: S21, the multimodal data includes user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data; S22. Perform model training based on the user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data to obtain a user music preference model.

[0023] It should be noted that user basic information data includes age data, gender data, and regional data; user operation record data includes collection data, like data, sharing data, and comment data; music play history data includes song name data, play times, and play duration; music basic attribute data includes song type data, singer information data, release time, lyrics content data, and music style label data; music audio feature data includes rhythm feature data, melody feature data, and timbre feature data. In the model training process, emotional tendency data and theme keyword data are extracted from song name data and lyrics content data as text feature data for model training to obtain the user music preference model.

[0024] According to an embodiment of the present invention, the user music preference model is tested to obtain model test data, including model universality test data and model calculation efficiency data, including: The model universality test data includes model accuracy test data, cross-dataset learning accuracy data and task performance consistency data; The model accuracy test data includes music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model calculation efficiency data includes memory occupancy, CPU usage and model inference time.

[0025] It should be noted that the cross-dataset learning accuracy represents the accuracy of the model for different data sets, which can be expressed by the ratio of the number of correctly predicted sample sets to the total number of test sample sets. The model task performance consistency data is quantitative data used to measure the degree of performance balance of the model in the three tasks of music preference prediction, music classification and music sentiment analysis, which can be expressed by calculating the variance of the music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy.

[0026] Please refer to Figure 3 , Figure 3 The flowchart of the method for training a multi-modal multi-task large model in some embodiments of the present application is to obtain a model universality evaluation index. According to an embodiment of the present invention, the method of obtaining a model universality evaluation index according to the model universality test data processing includes: S31, obtaining a model accuracy evaluation index according to the music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; S32. Obtain a model universality evaluation index based on the model accuracy evaluation index, cross-dataset learning accuracy data, and task performance consistency data.

[0027] It should be noted that the model accuracy evaluation index is calculated based on the accuracy of the model in performing the three tasks of music preference prediction, music classification, and music sentiment analysis; The calculation formula of the model accuracy evaluation index is: ; in, is the model accuracy evaluation index, , and They are respectively the accuracy of music preference prediction, music classification accuracy and music sentiment analysis accuracy. , and is the preset characteristic coefficient (which can be obtained by querying the preset model performance evaluation platform); The calculation formula of the model universality evaluation index is: ; in, is the model universality evaluation index, is the model accuracy evaluation index, and They are respectively the cross-dataset learning accuracy data and task performance consistency data, and is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0028] According to an embodiment of the present invention, the step of obtaining a computing efficiency evaluation index according to the model computing efficiency data processing includes: A computational efficiency evaluation index is obtained based on the memory occupancy rate, CPU usage rate and model reasoning time.

[0029] It should be noted that the calculation formula of the computational efficiency evaluation index is: ; in, is the computational efficiency evaluation index, , and They are memory usage, CPU usage and model inference time, , and is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0030] Please refer to Figure 4 , Figure 4 The present invention is a flowchart of a method for training a multi-modal multi-task large model in some embodiments of the present application to obtain corresponding model correction strategy data. According to an embodiment of the present invention, the processing of obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index, and obtaining corresponding model correction strategy data, includes: S41, obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index; S42, comparing the model performance evaluation index with a preset model performance evaluation index threshold, and determining the model performance evaluation level according to the range level to which the threshold comparison result belongs; S43: Input the model performance evaluation level into the preset model performance evaluation platform database for matching and identification, and obtain corresponding model correction strategy data.

[0031] It should be noted that the calculation formula of the model performance evaluation index is: ; in, is the model performance evaluation index, is the model universality evaluation index, is the computational efficiency evaluation index, is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0032] According to an embodiment of the present invention, it also includes: Recommend music to the user according to the user music preference model, and obtain the click rate, skip rate, number of plays and play time of the recommended music; Obtaining a recommendation effect evaluation index based on the click rate, skip rate, number of plays and play time; The recommendation effect evaluation index is compared with a preset recommendation effect evaluation index threshold. If the threshold comparison result does not meet the preset threshold comparison result requirement, feedback of poor model recommendation effect is given.

[0033] It should be noted that music recommendations are made to users based on the user music preference model, and the recommendation effect is evaluated; The calculation formula of the recommendation effect evaluation index is: ; in, is the recommendation effect evaluation index, , , and They are click-through rate, skip rate, number of plays, and play time. , , and is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0034] The present invention also discloses a multi-modal multi-task large model training system, comprising a memory and a processor, wherein the memory stores a multi-modal multi-task large model training method program, and when the multi-modal multi-task large model training method program is executed by the processor, the following steps are implemented: Collect multimodal data related to music users, perform model training based on the multimodal data, and obtain a user music preference model; Testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data; Obtaining a model universality evaluation index according to the model universality test data processing; Obtaining a computational efficiency evaluation index according to the computational efficiency data of the model; The model performance evaluation index is obtained according to the model universality evaluation index and the computational efficiency evaluation index, and the corresponding model correction strategy data is obtained.

[0035] It should be noted that this application integrates music data in multiple modalities such as user basic information data, user operation record data, music playback history data, music basic attribute data, and music audio feature data to perform model training, and evaluates the versatility and computational efficiency of the trained model, thereby obtaining a comprehensive evaluation result of the model performance, and finally obtaining the corresponding model correction strategy data, thereby achieving the purpose of improving model training accuracy and computational efficiency.

[0036] According to an embodiment of the present invention, the collecting of multimodal data related to music users, performing model training based on the multimodal data, and obtaining a user music preference model includes: The multimodal data includes user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data; Model training is performed based on the user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data to obtain a user music preference model.

[0037] It should be noted that user basic information data includes age data, gender data, and regional data; user operation record data includes collection data, like data, sharing data, and comment data; music play history data includes song name data, play times, and play duration; music basic attribute data includes song type data, singer information data, release time, lyrics content data, and music style label data; music audio feature data includes rhythm feature data, melody feature data, and timbre feature data. In the model training process, emotional tendency data and theme keyword data are extracted from song name data and lyrics content data as text feature data for model training to obtain the user music preference model.

[0038] According to an embodiment of the present invention, the user music preference model is tested to obtain model test data, including model universality test data and model calculation efficiency data, including: The model universality test data includes model accuracy test data, cross-dataset learning accuracy data and task performance consistency data; The model accuracy test data includes music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model calculation efficiency data includes memory occupancy, CPU usage and model inference time.

[0039] It should be noted that the cross-dataset learning accuracy represents the accuracy of the model for different data sets, which can be expressed by the ratio of the number of correctly predicted sample sets to the total number of test sample sets. The model task performance consistency data is quantitative data used to measure the degree of performance balance of the model in the three tasks of music preference prediction, music classification and music sentiment analysis, which can be expressed by calculating the variance of the music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy.

[0040] According to an embodiment of the present invention, the step of obtaining a model universality evaluation index according to the model universality test data processing includes: Obtaining a model accuracy evaluation index according to the music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model universality evaluation index is obtained based on the model accuracy evaluation index, cross-dataset learning accuracy data and task performance consistency data.

[0041] It should be noted that the model accuracy evaluation index is calculated based on the accuracy of the model in performing the three tasks of music preference prediction, music classification, and music sentiment analysis; The calculation formula of the model accuracy evaluation index is: ; in, is the model accuracy evaluation index, , and They are respectively the accuracy of music preference prediction, music classification accuracy and music sentiment analysis accuracy. , and is the preset characteristic coefficient (which can be obtained by querying the preset model performance evaluation platform); The calculation formula of the model universality evaluation index is: ; in, is the model universality evaluation index, is the model accuracy evaluation index, and They are respectively the cross-dataset learning accuracy data and task performance consistency data, and is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0042] According to an embodiment of the present invention, the step of obtaining a computing efficiency evaluation index according to the model computing efficiency data processing includes: A computational efficiency evaluation index is obtained based on the memory occupancy rate, CPU usage rate and model reasoning time.

[0043] It should be noted that the calculation formula of the computational efficiency evaluation index is: ; in, is the computational efficiency evaluation index, , and They are memory usage, CPU usage and model inference time, , and is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0044] According to an embodiment of the present invention, the processing of obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index, and obtaining corresponding model correction strategy data, includes: Obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index; Comparing the model performance evaluation index with a preset model performance evaluation index threshold, and determining the model performance evaluation level according to the range level to which the threshold comparison result belongs; The model performance evaluation level is input into the preset model performance evaluation platform database for matching and identification to obtain corresponding model correction strategy data.

[0045] It should be noted that the calculation formula of the model performance evaluation index is: ; in, is the model performance evaluation index, is the model universality evaluation index, is the computational efficiency evaluation index, is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0046] According to an embodiment of the present invention, it also includes: Recommend music to the user according to the user music preference model, and obtain the click rate, skip rate, number of plays and play time of the recommended music; Obtaining a recommendation effect evaluation index based on the click rate, skip rate, number of plays and play time; The recommendation effect evaluation index is compared with a preset recommendation effect evaluation index threshold. If the threshold comparison result does not meet the preset threshold comparison result requirement, feedback of poor model recommendation effect is given.

[0047] It should be noted that music recommendations are made to users based on the user music preference model, and the recommendation effect is evaluated; The calculation formula of the recommendation effect evaluation index is: ; in, is the recommendation effect evaluation index, , , and They are click-through rate, skip rate, number of plays, and play time. , , and is the preset characteristic coefficient (which can be obtained through the preset model performance evaluation platform).

[0048] A third aspect of the present invention provides a readable storage medium, which stores a training method program for a multimodal multi-task large model. When the training method program for the multimodal multi-task large model is executed by a processor, the steps of the training method for the multimodal multi-task large model as described in any one of the above items are implemented.

[0049] The present invention discloses a multi-modal, multi-task large model training method, system and medium, which integrates music data of multiple modalities for model training, and conducts a comprehensive analysis of the performance indicators of the trained model in multiple dimensions such as accuracy, computational efficiency and generalization ability of new data, so as to provide key feedback information for model training optimization, thereby achieving targeted model training optimization, improving the overall performance and practicality of the model, and helping to provide users with a more personalized and high-quality music service experience.

[0050] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0051] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0052] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0053] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, the aforementioned program can be stored in a readable storage medium, and when the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0054] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A training method for a multi-modal multi-task large model, characterized in that: The following steps are involved: Collect multimodal data related to music users, perform model training based on the multimodal data, and obtain a user music preference model; Testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data; Obtaining a model universality evaluation index according to the model universality test data processing; Obtaining a computational efficiency evaluation index according to the computational efficiency data of the model; The model performance evaluation index is obtained according to the model universality evaluation index and the computational efficiency evaluation index, and the corresponding model correction strategy data is obtained.

2. The method for training a multi-modal multi-task large model according to claim 1, characterized in that: The collecting of multimodal data related to music users, performing model training according to the multimodal data, and obtaining a user music preference model includes: The multimodal data includes user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data; Model training is performed based on the user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data to obtain a user music preference model.

3. The training method of the multi-modal multi-task large model according to claim 2 is characterized in that: The step of testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data, includes: The model universality test data includes model accuracy test data, cross-dataset learning accuracy data and task performance consistency data; The model accuracy test data includes music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model calculation efficiency data includes memory occupancy, CPU usage and model inference time.

4. The method for training a multi-modal multi-task large model according to claim 3, characterized in that: The step of obtaining a model universality evaluation index according to the model universality test data processing includes: Obtaining a model accuracy evaluation index according to the music preference prediction accuracy, music classification accuracy and music sentiment analysis accuracy; The model universality evaluation index is obtained based on the model accuracy evaluation index, cross-dataset learning accuracy data and task performance consistency data.

5. The method for training a multi-modal multi-task large model according to claim 4, characterized in that: The step of obtaining a computational efficiency evaluation index according to the computational efficiency data of the model comprises: A computational efficiency evaluation index is obtained based on the memory occupancy rate, CPU usage rate and model reasoning time.

6. The method for training a multi-modal multi-task large model according to claim 5, characterized in that: The step of obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index, and obtaining corresponding model correction strategy data, includes: Obtaining a model performance evaluation index according to the model universality evaluation index and the computational efficiency evaluation index; Comparing the model performance evaluation index with a preset model performance evaluation index threshold, and determining the model performance evaluation level according to the range level to which the threshold comparison result belongs; The model performance evaluation level is input into the preset model performance evaluation platform database for matching and identification to obtain corresponding model correction strategy data.

7. The method for training a multi-modal multi-task large model according to claim 6, characterized in that: Also includes: Recommend music to the user according to the user music preference model, and obtain the click rate, skip rate, number of plays and play time of the recommended music; Obtaining a recommendation effect evaluation index based on the click rate, skip rate, number of plays and play time; The recommendation effect evaluation index is compared with a preset recommendation effect evaluation index threshold. If the threshold comparison result does not meet the preset threshold comparison result requirement, feedback of poor model recommendation effect is given.

8. A multi-modal multi-task large model training system, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a program of a training method for a multi-modal multi-task large model, and when the program of the training method for the multi-modal multi-task large model is executed by the processor, the following steps are implemented: Collect multimodal data related to music users, perform model training based on the multimodal data, and obtain a user music preference model; Testing the user music preference model to obtain model test data, including model universality test data and model calculation efficiency data; Obtaining a model universality evaluation index according to the model universality test data processing; Obtaining a computational efficiency evaluation index according to the computational efficiency data of the model; The model performance evaluation index is obtained according to the model universality evaluation index and the computational efficiency evaluation index, and the corresponding model correction strategy data is obtained.

9. The multi-modal multi-task large model training system according to claim 8, characterized in that: The collecting of multimodal data related to music users, performing model training according to the multimodal data, and obtaining a user music preference model includes: The multimodal data includes user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data; Model training is performed based on the user basic information data, user operation record data, music playing history data, music basic attribute data and music audio feature data to obtain a user music preference model.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a training program for a multimodal, multi-task large model. When the training program for the multimodal, multi-task large model is executed by a processor, the steps of the training method for a multimodal, multi-task large model as described in any one of claims 1 to 7 are implemented.

Images