AI capability fusion scheduling method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and particularly discloses an AI capability fusion scheduling method and device, electronic equipment and a storage medium, and the AI capability fusion scheduling method comprises the steps: obtaining input data from a product application and corresponding task information in response to a request of the product application; disassembling the task information to obtain a plurality of AI capability types; under the guidance of the task information, jointly extracting AI models from a pre-accessed model pool according to each AI capability type to form a target AI model group; scheduling the target AI model group according to the task information to process the input data so as to generate a processing result; according to the method, the matching of the customized AI model does not need to be designed according to the product application, the matching strategy of the AI model can be automatically provided to complete the task according to the use requirement of the product application, the AI capability expansion of the central control platform is facilitated, the cost input is reduced, the flexibility and the processing effect of the AI model are improved, and the user experience is optimized.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to an AI capability fusion scheduling method, device, electronic device, and storage medium. Background Art

[0002] In today's technological wave, the central control platform has been widely used in many fields with its excellent functionality and convenience, and the most eye-catching one is the home field. With the rapid development of smart homes, the central control platform has not only become the core hub for connecting various smart devices, but also the key to improving living experience and realizing home intelligence.

[0003] However, despite the excellent functionality and practicality of the central control platform, there are still some urgent problems to be solved in its configuration and application. Especially when it comes to the coordination of AI models and product applications, the existing fixed combination is particularly rigid. This combination means that whenever there are new user needs or product applications, it is necessary to readjust or develop a matching AI model, which undoubtedly increases development costs and time investment. This fixed combination also limits the flexibility and processing effect of the AI ​​model in dealing with user needs, which makes the AI ​​model not only affect the user experience when dealing with complex or diverse user needs, but also limits the further development and application of the central control platform in the field of smart home.

[0004] There is currently no effective technical solution to the above problems. Summary of the invention

[0005] The purpose of this application is to provide an AI capability fusion scheduling method, device, electronic device and storage medium, so as to facilitate the low-cost expansion of AI capabilities of the central control platform, and improve the flexibility and processing effect of the AI ​​model to optimize the user experience.

[0006] In a first aspect, the present application provides an AI capability fusion scheduling method, which is applied in a central control platform. The AI ​​capability fusion scheduling method includes the following steps: S1. In response to a request from a product application, obtaining input data and corresponding task information from the product application; S2. Decomposing the task information to obtain multiple AI capability types; S3. Under the guidance of the task information, jointly extract AI models from a pre-connected model pool according to each of the AI ​​capability types to form a target AI model group, wherein the model pool has multiple AI models corresponding to each AI capability type; S4. Scheduling the target AI model group to process the input data according to the task information to generate a processing result.

[0007] The AI ​​capability fusion scheduling method of the present application is applied in the central control platform. It can parse out multiple AI capability types required for processing input data according to the task information published by the product application, and under the guidance of the task information, it can schedule appropriate AI models from the model pool according to these AI capability types to form a target AI model group to process the input data. There is no need to design customized AI models according to product applications. It can automatically provide AI model matching strategies according to the usage requirements of product applications to complete tasks, which is convenient for the central control platform to expand AI capabilities and reduce cost investment, and improves the flexibility and processing effect of AI models to optimize user experience.

[0008] The AI ​​capability fusion scheduling method, wherein step S3 includes the following sub-steps: S31, determining the priority level of each AI capability type corresponding to the task information; S32. Extract AI models corresponding to various AI capability types matching the task information from the model pool according to the priority levels and predetermined conflict relationships to form a target AI model group.

[0009] The AI ​​capability fusion scheduling method of the present application sets priorities for different AI capability types, so that in the process of extracting AI models to form a target AI model group, a suitable AI model is selected based on the AI ​​capability type with a high priority level, and the AI ​​model corresponding to the AI ​​capability type with a low priority level is selected as an auxiliary model based on the AI ​​model, so as to form a target AI model group with high core capabilities, so as to improve the processing effect of the processing results.

[0010] The AI ​​capability fusion scheduling method, wherein step S31 includes the following sub-steps: S311. Determine the priority level of each AI capability type corresponding to the task information according to the task information and the user preference requirement information.

[0011] In this example, step S311 can be to determine the initial priority level of each AI capability type corresponding to the task information based on the task information, and then adjust the priority level of these AI capability types based on the user preference requirement information. It can also be to set the corresponding AI capability type to the highest priority level based on the user preference requirement information, and then set the priority level of the remaining AI capability types according to the task information, so that the target AI model group determined based on the priority level can better meet the user's usage needs.

[0012] The AI ​​capability fusion scheduling method, wherein when the input data includes voice data, the step between step S1 and step S2 further includes: SA, obtaining the user's voice feature information according to the input data, and obtaining the user's preference demand information according to the voice feature information.

[0013] The AI ​​capability fusion scheduling method, wherein step S32 includes: S321. Determine specific AI models corresponding to several AI capability types based on user needs or application needs; S322. Extract AI models corresponding to the remaining AI capability types corresponding to the task information from the model pool according to the priority level and the predetermined conflict relationship, and form the target AI model group with the designated AI model.

[0014] The AI ​​capability fusion scheduling method further includes the following steps between step S3 and step S4: SB, extracting multiple AI models of various AI capability types matching the task information and different from the target AI model group from the model pool according to the priority level and the predetermined conflict relationship to form a backup AI model group; S5. When executing step S4, the backup AI model group is scheduled in the background according to the task information to process the input data to generate a backup result, and the backup result is output after the user is not satisfied with the processing result or the processing result fails to be generated.

[0015] The AI ​​capability fusion scheduling method, wherein the model pool is composed of multiple AI models accessed based on a standardized API.

[0016] In a second aspect, the present application further provides an AI capability fusion scheduling device, which is applied in a central control platform. The AI ​​capability fusion scheduling device includes: An acquisition module, configured to acquire input data and corresponding task information from the product application in response to a request from the product application; A disassembly module, used to disassemble the task information to obtain multiple AI capability types; An extraction module, for jointly extracting AI models from a pre-connected model pool according to each of the AI ​​capability types under the guidance of the task information to form a target AI model group, wherein the model pool has a plurality of AI models corresponding to each AI capability type; A scheduling processing module is used to schedule the target AI model group to process the input data according to the task information to generate a processing result.

[0017] The AI ​​capability fusion scheduling device of the present application is applied in the central control platform. It can parse out multiple AI capability types required for processing input data according to the task information published by the product application, and under the guidance of the task information, it can schedule appropriate AI models from the model pool according to these AI capability types to form a target AI model group to process the input data. There is no need to design customized AI models according to product applications. It can automatically provide AI model matching strategies according to the usage requirements of product applications to complete tasks, which is convenient for the central control platform to expand AI capabilities and reduce cost investment, and improves the flexibility and processing effect of AI models to optimize user experience.

[0018] In a third aspect, the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are executed.

[0019] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the method provided in the first aspect are executed.

[0020] From the above, it can be seen that the present application provides an AI capability fusion scheduling method, device, electronic device and storage medium, wherein the AI ​​capability fusion scheduling method is applied in the central control platform, and can parse out multiple AI capability types required for processing input data according to the task information released by the product application, and under the guidance of the task information, according to these AI capability types, schedule appropriate AI models from the model pool to form a target AI model group to process the input data. There is no need to design a customized combination of AI models according to the product application, and it can automatically provide an AI model combination strategy to complete the task according to the usage requirements of the product application, which is convenient for the central control platform to expand AI capabilities and reduce cost investment, and improves the flexibility and processing effect of the AI ​​model to optimize the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Flow chart of the AI ​​capability fusion scheduling method provided in an embodiment of the present application.

[0022] Figure 2 A schematic diagram of the structure of the AI ​​capability fusion scheduling device provided in an embodiment of the present application.

[0023] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0024] Figure numerals: 201, acquisition module; 202, disassembly module; 203, extraction module; 204, scheduling processing module; 301, processor; 302, memory; 303, communication bus. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0026] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0027] First, please refer to Figure 1 Some embodiments of the present application provide an AI capability fusion scheduling method, which is applied in a central control platform. The AI ​​capability fusion scheduling method includes the following steps: S1. Responding to a request from a product application, obtaining input data from the product application and corresponding task information; S2. Disassemble task information to obtain multiple AI capability types; S3. Under the guidance of the task information, jointly extract AI models from a pre-connected model pool according to each AI capability type to form a target AI model group, wherein the model pool has multiple AI models corresponding to each AI capability type; S4. Schedule the target AI model group to process the input data according to the task information to generate processing results.

[0028] Specifically, the central control platform is widely used in systems in the fields of automobile design, home, industry, medical care, and traffic monitoring. The AI ​​capability fusion scheduling method of the embodiment of the present application aims to provide AI function decision-making for the central control platform so that the central control platform can automatically select the appropriate AI model to perform related functions according to usage requirements.

[0029] More specifically, the AI ​​capability fusion scheduling method of the embodiment of the present application is particularly suitable for application in the central control platform of a home smart terminal to provide users with a more humane interactive experience and an easy and convenient digital home life.

[0030] More specifically, the product application is an application or application port that connects the central control platform to the corresponding foreground product. The foreground product is the device end that interacts with the user, such as a mobile phone, remote control, microphone, camera, etc. The same foreground product can be connected to the central control platform through one or more types of product applications. The AI ​​capability fusion scheduling method of the embodiment of the present application actually configures a suitable target AI model group to execute the corresponding task to generate processing results when the product application accesses the AI ​​capability of the central control platform based on the corresponding task information.

[0031] More specifically, the task information is a series of processing behaviors that need to be performed on the input data. In the embodiments of the present application, it mainly refers to a series of processing behaviors involving AI model processing. Generally, a type of product application corresponds to one or more types of task information. The task information is actually determined by the type of product application, so that step S1 can obtain task information based on the type of product application, or it can obtain task information based on the content of the interface document of the standardized interface of the product application, or it can directly input task information based on the product application.

[0032] More specifically, the request of the product application is a request command issued to the central control platform when the corresponding foreground product needs to perform related operations through the central control platform. It can be issued when the foreground product is awakened (it can be awakened by a button or voice), or it can be issued when the product application receives input data from the user. The AI ​​capability fusion scheduling method of the embodiment of the present application receives input data from the product application after obtaining the request of the product application, or receives input data input by the foreground product in real time through the product application, and simultaneously determines the task information of the corresponding product application.

[0033] More specifically, since the task information is a series of processing behaviors that need to be performed on the input data and involve AI model processing, the central control platform needs to configure multiple AI models to complete the corresponding task processing on the input data. For example, for the task information of generating corresponding images from voice data, the central control platform needs to configure multiple AI models such as voice recognition, keyword extraction / semantic understanding, and text graphics to complete the corresponding task processing. Different types of AI models have different AI capabilities, which refers to the classification function of the AI ​​model, such as speech synthesis, speech recognition, face recognition, etc. The task information can be divided into multiple different sub-tasks with dependencies. These sub-tasks need to be executed using AI models with different AI capabilities. Step S2 is equivalent to disassembling the task information to obtain the corresponding multiple sub-tasks, and determining the types of AI capabilities required for different sub-tasks.

[0034] More specifically, AI models belonging to different AI capability types can process different subtasks, and AI models belonging to the same AI capability type also produce different results when performing subtasks on the same data. In particular, the processing effects of AI models provided or fine-tuned by different manufacturers vary greatly. Moreover, for the overall task information, interdependent subtasks corresponding to the use of AI models in different combinations to complete the tasks will ultimately produce different processing results. The AI ​​capability fusion scheduling method of the embodiment of the present application can process input data by combining different AI models to form a target AI model group to produce processing results with different processing effects.

[0035] More specifically, compared with the traditional one-to-many fixed configuration method, the AI ​​capability fusion scheduling method of the embodiment of the present application uniformly schedules all AI models according to the request of the product application, and calls multiple suitable AI models to form a target AI model group according to the characteristics of the task information of the product application to complete the corresponding task to produce a suitable processing result; the AI ​​models contained in the extracted target AI model group in step S3 are AI models in the model pool that match the corresponding AI capability type and are suitable for the corresponding task information. The model pool is a pre-integrated collection of all available AI models. The process of extracting the AI ​​model in step S3 can be to use a learning model or a deep learning model to automatically extract a suitable AI model from the model pool according to the AI ​​capability type, or it can be based on the product application and the supplier relationship of the AI ​​model combined with the AI ​​capability type to automatically extract the appropriate AI model from the model pool. Extracting a suitable AI model from the model pool can also be based on the user's usage habits and / or usage requirements and / or the ability (or score) of different AI models of different AI capability types to execute the corresponding task information, and automatically extracting a suitable AI model from the model pool to form a target AI model group to complete the task released by the corresponding product application; under these extraction methods, the AI ​​capability fusion scheduling method of the embodiment of the present application can obtain the target AI model group corresponding to the task information of completing the product application from the model pool, and after adding a new product application or adding an AI model to the model pool, the AI ​​capability fusion scheduling method of the embodiment of the present application does not need to manually re-configure multiple suitable AI models according to the usage characteristics of the product application, which facilitates the central control platform to expand the AI ​​capabilities, and also improves the flexibility and processing effect of the AI ​​model to optimize the user experience.

[0036] More specifically, in step S3, the task information is used to guide the AI ​​model extraction process. It can be to guide the extraction order of the AI ​​model, such as extracting the AI ​​model according to the usage order of the AI ​​capability types in the task information, or it can be to guide the product application's selection tendency for the AI ​​capability type. For example, if a product application focuses more on semantic understanding, the task information preferentially guides the selection of high-quality AI models for semantic understanding, and then on this basis selects AI models with other AI capability types that can be used in combination with the AI ​​model (without conflicting use). Based on the guidance of the task information, the target AI model group obtained by the AI ​​capability fusion scheduling method of the embodiment of the present application is highly matched with the product application and can meet its usage requirements.

[0037] More specifically, the input data is data input into the product application based on the foreground product. It can be data of various modalities or any combination, depending on how the foreground product is used, such as one or more of voice data, static image data, video data, and text data. The processing result is the final result generated by executing the task information provided by the product application through the target AI model group, which can be fed back to the corresponding interactive product (which can be the foreground product itself or other products) through the central control platform for output.

[0038] It should be noted that the AI ​​capability fusion scheduling method of the embodiment of the present application is mainly used in situations where multiple AI models need to be scheduled to complete task information released by product applications, and is particularly suitable for situations where the processing results generated by different AI model combinations are significantly different.

[0039] It should be noted that the AI ​​capability fusion scheduling method of the embodiment of the present application is applicable to data processing in the voice direction, image direction, video direction, and NLP (natural language processing) direction, and is preferably applicable to data processing in the voice direction.

[0040] The AI ​​capability fusion scheduling method of the embodiment of the present application is applied in the central control platform. It can parse out multiple AI capability types required for processing input data according to the task information published by the product application, and under the guidance of the task information, schedule appropriate AI models from the model pool according to these AI capability types to form a target AI model group to process the input data. There is no need to design a customized combination of AI models according to the product application. It can automatically provide an AI model combination strategy to complete the task according to the usage requirements of the product application, which is convenient for the central control platform to expand AI capabilities and reduce cost investment, and improves the flexibility and processing effect of the AI ​​model to optimize the user experience.

[0041] In some preferred embodiments, step S3 includes the following sub-steps: S31, determining the priority level of each AI capability type corresponding to the task information; S32. Extract AI models corresponding to each AI capability type matching the task information from the model pool according to the priority level and the predetermined conflict relationship to form a target AI model group.

[0042] Specifically, the priority level of each AI capability type determined based on the task information in step S31 is used to guide AI model extraction. Different task information has different task requirements, which have different tendencies for different AI capabilities. In the embodiment of the present application, the priority level of the AI ​​capability type with a higher tendency is higher. The AI ​​capability type with a high priority level corresponds to the core subtask in the task information, and determines the core capability of the entire task processing effect. Taking into account that different AI models have different effects when used in combination, the AI ​​capability fusion scheduling method of the embodiment of the present application sets priorities for different AI capability types, so that in the process of extracting AI models to form a target AI model group, a suitable AI model is selected based on the AI ​​capability type with a high priority level, and the priority model is selected based on the AI ​​model as the leading one. The AI ​​models corresponding to the low-level AI capability types are used as auxiliary models to form a target AI model group with high core capabilities to improve the processing effect of the processing results; in addition, some AI models may have conflicts (mainly manifested as bugs) when used in combination, resulting in the inability to complete the corresponding tasks. In step S32, the predetermined conflict relationship is the conflict mark of AI models of different AI capability types. For example, if there is a conflict mark between model a of capability A and model b of capability B, then after extracting model a, model b will be regarded as an unextractable object to avoid the failure of step S4; thus, step S32 obtains multiple AI models in combination with the priority level and the predetermined conflict relationship to ensure that the target AI model group composed of these AI models can process the input data smoothly and with high quality to obtain high-quality processing results.

[0043] More specifically, in this embodiment, the task information includes capability preference information, which characterizes the importance of each AI capability type required in the task of the product application release, which is determined by the type of the foreground product. For example, a product application release task of a foreground product is to generate images based on voice data, and the required AI capability types include voice recognition processing, semantic understanding processing, keyword extraction processing, and vincent image processing. If the foreground product focuses on providing high-quality or high-resolution images, then vincent image processing is the core capability of the task, and the capability preference information is more inclined to vincent image processing, so that step S31 can give vincent image processing a higher priority, so that step S32 can match AI models of other AI capability types to form a target AI model group when the AI ​​model that is more inclined to provide high-quality vincent image processing is used.

[0044] More specifically, the AI ​​models in the model pool have scores for corresponding AI capability types. Step S32 can extract AI models based on the priority level and the predetermined conflict relationship combined with the score of the AI ​​capability type. For example, the priority level is used as the extraction order, and the AI ​​models with the highest scores and non-conflicting AI models corresponding to each AI capability type are extracted to form a target AI model group. Another example is to assign different weights to different AI capability types based on the priority level, and then determine the bias scores of each AI model based on the product of the score of the AI ​​model and the weight, and then extract the AI ​​models with the highest total bias scores and non-conflicting AI models to form a target AI model group (equivalent to evaluating the total scores of different AI model combinations, and then selecting the AI ​​model combination with the highest total score as the target AI model group). The former processing method can quickly determine the target AI model group with high core capabilities, and the latter processing method can determine the target AI model group with the strongest comprehensive performance. Both can process the input data smoothly and with high quality to obtain high-quality processing results.

[0045] In addition, since different front-end products may input the same input data due to product characteristics, environmental characteristics, noise interference and other factors, which may cause deviations in the input data, the processing results output by the same target AI model group may also be different. Therefore, in some embodiments, the task information obtained in step S1 also includes characteristic information and / or type information of the product application, so that step S31 can further set an appropriate priority level based on the characteristic information or type information, that is, step S31 comprehensively considers the capability bias information and the characteristic information and / or type information of the product application to determine the priority level of each corresponding AI capability type. The specific processing method may be: preliminarily determine the priority level of each corresponding AI capability type based on the capability bias information, and then compensate and adjust the priority level of some AI capability types according to the characteristic information and / or type information of the product application.

[0046] More specifically, the score of the AI ​​model can be a user score, such as an average score determined by comprehensively analyzing the scores of various users, or a score adjusted according to the user's satisfaction with the processing results. The user can be a user of the current central control platform or a user of other central control platforms. The target AI model group obtained by this implementation can better meet the user's usage needs; the score of the AI ​​model can also be a performance score of the model, such as a comprehensive score combining the accuracy score, timeliness score and load score of the AI ​​model, wherein the accuracy score is the score of the processing accuracy of the AI ​​model in processing the corresponding subtask, the timeliness score is the score of the processing speed of the AI ​​model in processing the corresponding subtask, and the load score is the score of the computing resources occupied or consumed by the AI ​​model in processing the corresponding subtask. The comprehensive score determined based on these scores can accurately characterize the performance differences of different AI models, and the target AI model group obtained by this implementation can better provide high-quality performance processing capabilities.

[0047] More specifically, in an implementation method in which the score of an AI model is a performance score of the model, the comprehensive score also includes a matching score, which includes a correlation matching score between AI models and a matching score between an AI model and a front-end product; wherein, the use of some specific multiple AI models in combination will produce a better processing effect, which is generally manifested in that the same manufacturer adapts and fine-tunes the multiple AI models it provides, so that the combination of these AI models can produce a better processing effect. In this case, the corresponding matching scores (correlation matching scores between AI models) are added to the AI ​​models extracted at the same time to more accurately distinguish the advantages and disadvantages of different AI model combinations; in addition, there are also some AI models that will produce better processing effects when paired with front-end products, which is generally manifested in that the same manufacturer fine-tunes the AI ​​model according to the characteristics of the input data obtained by the front-end product it provides, so that the AI ​​model it provides can produce a better processing effect for the input processing provided by the front-end application. In this case, the matching scores (matching scores of AI models and front-end products) are added to the extracted AI models to more accurately distinguish the advantages and disadvantages of different AI model combinations.

[0048] In some preferred embodiments, step S31 includes the following sub-steps: S311. Determine the priority level of each AI capability type corresponding to the task information according to the task information and the user preference requirement information.

[0049] Specifically, in this implementation, step S311 may be to determine the initial priority level of each AI capability type corresponding to the task information based on the task information, and then adjust the priority level of these AI capability types based on the user preference requirement information. It may also be to set the corresponding AI capability type to the highest priority level based on the user preference requirement information, and then set the priority levels of the remaining AI capability types based on the task information, so that the target AI model group determined based on the priority level can better meet the user's usage needs.

[0050] More specifically, the user preference demand information can be a user tag pre-set by the user, or a user tag determined based on the user's usage habits or big data information. For example, if a product application release task of a front-end product is to generate images based on voice data, if the user tag tends to provide high-quality images, then step S311 will increase the priority level of the AI ​​capability type for text image processing based on the user preference demand information; if the user tag tends to provide images with more accurate picture content, then step S311 will increase the priority level of the AI ​​capability type for semantic understanding processing based on the user preference demand information.

[0051] In some embodiments, the task information also includes subtask information for disassembling and obtaining multiple AI capability types, or the task information includes the task type, and the subtask information is obtained based on the task type analysis.

[0052] In some preferred embodiments, when the input data includes voice data, the following steps are further included between step S1 and step S2: SA, obtain the user's voice feature information based on the input data, and obtain the user's preference demand information based on the voice feature information.

[0053] Specifically, the sound feature information in step SA is sound fingerprint data extracted based on voice data and characterizing the user's sound features, which is used to determine the user's identity so that step SA can extract the corresponding user preference demand information from a database that pre-stores user preference demand information corresponding to the user identity; the AI ​​capability fusion scheduling method of an embodiment of the present application can determine the user's identity based on sound feature information, and call the corresponding user preference demand information as a setting rule for the priority level of the AI ​​capability type, so as to provide a target AI model group that meets the user's preference requirements to process the input data to meet the user's personalized usage needs.

[0054] In some other implementations, when the input data includes voice data, the following is further included between step S1 and step S2: SA', obtain the user's voice feature information based on the input data, infer the user's age information and gender information based on the voice feature information, and obtain the user's preference demand information based on the age information and gender information.

[0055] Specifically, in cases where the user is an unfamiliar user or there is no database that pre-stores user preference demand information corresponding to the user identity, the AI ​​capability fusion scheduling method of the embodiment of the present application can extract sound feature information based on voice data, and use this as a basis to infer the user's age information and gender information, and then obtain user preference demand information corresponding to the age information and gender information based on the big data method, or extract the corresponding user preference demand information from a database that pre-stores user preference demand information associated with different ages and genders based on the age information and gender information, so as to provide a target AI model group that meets user preference needs to process input data, so as to basically meet the personalized usage needs of unfamiliar users.

[0056] More specifically, the process of inferring the user's age information and gender information based on the sound feature information is actually a classification process, which can be implemented using a classification model based on voice input, and will not be elaborated here.

[0057] In some other embodiments, step SA and step SA' can be used in combination, specifically: after obtaining the sound feature information obtained according to the input data, determine whether the current user is a private user based on the sound feature information; if so, extract the corresponding user preference demand information from a database that pre-stores user preference demand information corresponding to the user identity based on the sound feature information; otherwise, infer the user's age information and gender information based on the sound feature information, and obtain the user preference demand information based on the age information and gender information.

[0058] Specifically, a private user refers to a user whose corresponding user identity information is recorded on the central control platform. This implementation method is equivalent to first determining whether the current user has ever used the central control platform based on voice input. If so, directly calling the corresponding user preference demand information; otherwise, inferring the voice characteristics of the new unfamiliar user to infer his age and gender to provide a target AI model group that meets the corresponding user preference requirements to process the input data.

[0059] In some other implementations, when the input data includes portrait data, the following is further included between step S1 and step S2: SA'', obtains the user's facial information based on the input data, and obtains the user's preference demand information based on the facial information.

[0060] Specifically, the facial information in step SA'' is a facial image of a person captured based on portrait data, which is used to determine the user identity, so that step SA'' can extract the corresponding user preference demand information from a database that pre-stores user preference demand information corresponding to the user identity; the AI ​​capability fusion scheduling method of an embodiment of the present application can determine the user identity based on facial information, and call the corresponding user preference demand information as a setting rule for the priority level of the AI ​​capability type, so as to provide a target AI model group that meets the user preference requirements to process the input data to meet the user's personalized usage needs.

[0061] In some other implementations, when the input data includes portrait data, the following is further included between step S1 and step S2: SA''', obtains the user's facial information based on the input data, infers the user's age information and gender information based on the facial information, and obtains the user's preference demand information based on the age information and gender information.

[0062] Specifically, for situations where the user is an unfamiliar user or there is no database that pre-stores user preference demand information corresponding to the user identity, the AI ​​capability fusion scheduling method of the embodiment of the present application can input data to extract facial information, and use this as a basis to infer the user's age information and gender information, and then obtain user preference demand information corresponding to the age information and gender information according to the big data method, or extract the corresponding user preference demand information from a database that pre-stores user preference demand information associated with different ages and genders based on the age information and gender information, so as to provide a target AI model group that meets user preference needs to process the input data, so as to basically meet the personalized usage needs of unfamiliar users.

[0063] More specifically, the process of inferring the user's age and gender information based on facial information is actually a classification process, which can be implemented using a classification model based on voice input, and will not be elaborated here.

[0064] In some other embodiments, step SA'' and step SA''' can be used in combination, specifically: after obtaining the facial information obtained according to the input data, determine whether the current user is a private user based on the facial information; if so, extract the corresponding user preference demand information from a database that pre-stores user preference demand information corresponding to the user identity based on the facial information; otherwise, infer the user's age information and gender information based on the facial information, and obtain the user preference demand information based on the age information and gender information.

[0065] In some preferred embodiments, step S32 includes: S321. Determine specific AI models corresponding to several AI capability types based on user needs or application needs; S322. Extract AI models corresponding to the remaining AI capability types corresponding to the task information from the model pool according to the priority level and the predetermined conflict relationship, and form a target AI model group with the designated AI model.

[0066] Specifically, based on user needs or application needs, some product applications must cooperate with the designated AI models corresponding to a specific class or multiple classes of AI capability types. The AI ​​capability fusion scheduling method of the embodiment of the present application can match the AI ​​models corresponding to the remaining AI capability types in the scenario that meets the requirements to form a target AI model group with these designated AI models determined based on user needs or application needs to process the input data, so that the AI ​​capability fusion scheduling method of the embodiment of the present application has the advantage of flexible use and can meet the usage requirements of various scenarios.

[0067] More specifically, the specified AI model determined based on user needs can be an AI model that is pre-set by the user and must be used when using the corresponding AI capability type, or it can be an AI model that is pre-set by the user and must be used when using the corresponding AI capability type under the product application. This allows users to set a specified AI model for AI functions or product applications, so that the AI ​​capability fusion scheduling method of the embodiment of the present application has the advantage of flexible use.

[0068] More specifically, the designated AI model determined based on application requirements may be an AI model that is pre-set by the manufacturer and must be used for the corresponding AI capability type under the application of the product.

[0069] In some preferred embodiments, the following steps are further included between step S3 and step S4: SB, extracting multiple AI models of various AI capability types matching the task information and different from the target AI model group from the model pool according to the priority level and the predetermined conflict relationship to form a backup AI model group; S5. When executing step S4, the backup AI model group is scheduled in the background according to the task information to process the input data to generate a backup result, and the backup result is output after the user is not satisfied with the processing result or the processing result fails to be generated.

[0070] Specifically, the target AI model group obtained in step S3 is equivalent to the most preferred AI model combination for processing the task information released by the corresponding product application in the current state, and the backup AI model group obtained in step SB is equivalent to the sub-preferred AI model combination that is completely different from the target AI model group for processing the task information released by the corresponding product application in the current state; the AI ​​capability fusion method of the embodiment of the present application is equivalent to providing two decisions with certain differences in processing effects for processing the task information released by the product application, and when the processing result output by the target AI model group as the main decision does not meet the user's needs or fails to be generated, the backup result of the backup AI model group as the secondary decision is output, so that the user can obtain the backup result different from the processing result in time after being dissatisfied with the processing result or the processing result fails to be generated, which avoids the user having to spend more time waiting for the generation of new processing results, and avoids the repetition of the regenerated processing results and the previous processing results that the user is not satisfied with too high.

[0071] More specifically, the processing results generated by the target AI model group are directly fed back to the corresponding interactive products through the central control platform for output, and the backup results generated by the backup AI model group are retained in the background of the central control platform. When the user is not satisfied with the processing results or the processing results fail to be generated, they are fed back to the corresponding interactive products through the central control platform for output.

[0072] More specifically, the AI ​​models in the target AI model group and the AI ​​models in the backup AI model group are completely different; the backup AI model group composed of completely different AI models can produce backup results that are significantly different from the processing results obtained by the target AI model group and can still meet user needs. During the processing process, there is no need to copy the same AI model to perform subtasks to save processing time and space, and ensure that the processing process of the target AI model group and the processing process of the backup AI model group will not interfere with each other.

[0073] More specifically, the backup AI model group can start processing input data at the same time as the target AI model group processes input data, or it can start processing input data after a delay in the target AI model group processing input data. In an embodiment of the present application, the latter is preferred to stagger the enabling nodes of the target AI model group and the backup AI model group to reduce the operating load of the central control platform.

[0074] In some embodiments, when the user is not satisfied with the processing result or the processing result fails to be generated, it indicates that the current target AI model group does not meet the user's preference or is not suitable for the current product application; therefore, the AI ​​capability fusion scheduling method of the embodiment of the present application also includes the steps of: S6. When the user is not satisfied with the processing result or the processing result fails to be generated, the comprehensive score of the AI ​​model corresponding to the target AI model group is reduced.

[0075] Specifically, in this embodiment, the comprehensive score can be reduced based on a fixed score or a percentage score, preferably the latter.

[0076] More specifically, the AI ​​capability fusion scheduling method of the embodiment of the present application reduces the comprehensive score of the AI ​​model corresponding to the target AI model group, which can reduce the probability of selecting these AI models to form an AI model group when the task information released by the subsequent execution product application is executed, so that the target AI model group scheduled by the AI ​​capability fusion scheduling method of the embodiment of the present application can better meet user needs.

[0077] In some implementations, the AI ​​capability fusion scheduling method of the embodiment of the present application further includes the steps of: S7. When the user is satisfied with the backup results, improve the comprehensive score of the AI ​​model corresponding to the backup AI model group.

[0078] Specifically, in this embodiment, the comprehensive score can be improved based on a fixed score or a percentage score, preferably the latter.

[0079] More specifically, the AI ​​capability fusion scheduling method of the embodiment of the present application improves the comprehensive score of the AI ​​model corresponding to the standby AI model group, which can increase the probability of selecting these AI models to form an AI model group when subsequently executing the task information released by the product application, so that the target AI model group subsequently scheduled by the AI ​​capability fusion scheduling method of the embodiment of the present application can better meet user needs.

[0080] In some embodiments, the comprehensive scores of the AI ​​models in the model pool corresponding to different product applications or task information are calculated independently.

[0081] Specifically, in this implementation, each AI model has a corresponding, independently calculated comprehensive score corresponding to different product applications or task information, and these comprehensive scores may be the same or different; based on the foregoing content, it can be seen that when different front-end products input the same input data, the input data may be biased due to factors such as product characteristics, environmental characteristics, or noise interference, and the processing results output by the same target AI model group may be different, and for different task information, the same AI model will produce different processing effects. Therefore, the AI ​​capability fusion scheduling method of the embodiment of the present application designs the comprehensive scores of the AI ​​models in the model pool corresponding to different product applications or task information to be calculated independently, so that the AI ​​capability fusion scheduling method of the embodiment of the present application can provide users with a better target AI model group for different product applications or task information to complete corresponding tasks.

[0082] More specifically, in this embodiment, the comprehensive score changed in step S6 and step S7 is the comprehensive score of the AI ​​model under the corresponding product application or task information.

[0083] In some preferred embodiments, the model pool is composed of multiple AI models accessed based on a standardized API.

[0084] Specifically, multiple AI models are connected to the central control platform based on standardized APIs to form a model pool, realizing multi-source AI capability access and providing standard and unified services to the outside world.

[0085] More specifically, in this implementation, the process of standardized API accessing the AI ​​model is: adding the AI ​​model, generating a standard API based on the AI ​​capability type of the AI ​​model, and generating an interface document with a unified style, so that the AI ​​capability fusion scheduling method of the embodiment of the present application can call the corresponding AI model according to the content of the interface document when used.

[0086] More specifically, the AI ​​capability fusion scheduling method of the embodiment of the present application can conveniently access or delete AI models based on the above-mentioned access method to achieve optimization, expansion or reduction of AI capabilities and reduce the equipment update and maintenance costs of the central control platform.

[0087] Second, please refer to Figure 2 Some embodiments of the present application further provide an AI capability fusion scheduling device, which is applied in a central control platform. The AI ​​capability fusion scheduling device includes: The acquisition module 201 is used to obtain input data and corresponding task information from the product application in response to a request from the product application; A disassembly module 202 is used to disassemble task information to obtain multiple AI capability types; An extraction module 203 is used to jointly extract AI models from a pre-connected model pool according to each AI capability type under the guidance of the task information to form a target AI model group, wherein the model pool has multiple AI models corresponding to each AI capability type; The scheduling processing module 204 is used to schedule the target AI model group to process the input data according to the task information to generate a processing result.

[0088] The AI ​​capability fusion scheduling device of the embodiment of the present application is applied in the central control platform. It can parse out multiple AI capability types required for processing input data according to the task information published by the product application, and under the guidance of the task information, it can schedule suitable AI models from the model pool according to these AI capability types to form a target AI model group to process the input data. There is no need to design a customized combination of AI models according to the product application. It can automatically provide an AI model combination strategy to complete the task according to the usage requirements of the product application, which is convenient for the central control platform to expand AI capabilities and reduce cost investment, and improves the flexibility and processing effect of the AI ​​model to optimize the user experience.

[0089] In some preferred implementations, the AI ​​capability fusion scheduling device of the embodiment of the present application is used to execute the AI ​​capability fusion scheduling method provided in the first aspect above.

[0090] Third, please refer to Figure 3 Some embodiments of the present application also provide a structural diagram of an electronic device. The present application provides an electronic device, including: a processor 301 and a memory 302. The processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not marked). The memory 302 stores computer-readable instructions executable by the processor 301. When the electronic device is running, the processor 301 executes the computer-readable instructions to execute the method in any optional implementation of the above embodiments.

[0091] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method in any optional implementation of the above embodiment is executed. The computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0092] In summary, the embodiments of the present application provide an AI capability fusion scheduling method, device, electronic device and storage medium, wherein the AI ​​capability fusion scheduling method is applied in the central control platform, and can parse out multiple AI capability types required for processing input data according to the task information released by the product application, and under the guidance of the task information, according to these AI capability types, schedule appropriate AI models from the model pool to form a target AI model group to process the input data. There is no need to design a customized combination of AI models according to the product application, and it can automatically provide an AI model combination strategy to complete the task according to the usage requirements of the product application, which is convenient for the central control platform to expand AI capabilities and reduce cost investment, and improves the flexibility and processing effect of the AI ​​model to optimize the user experience.

[0093] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0094] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0096] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0097] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An AI capability fusion scheduling method, applied in a central control platform, characterized in that: The AI ​​capability fusion scheduling method comprises the following steps: S1. Responding to a request from a product application, obtaining input data from the product application and corresponding task information; S2. Decomposing the task information to obtain multiple AI capability types; S3. Under the guidance of the task information, jointly extract AI models from a pre-connected model pool according to each of the AI ​​capability types to form a target AI model group, wherein the model pool has multiple AI models corresponding to each AI capability type; S4. Scheduling the target AI model group to process the input data according to the task information to generate a processing result.

2. The AI ​​capability fusion scheduling method according to claim 1, characterized in that: Step S3 includes the following sub-steps: S31, determining the priority level of each AI capability type corresponding to the task information; S32. Extract AI models corresponding to various AI capability types matching the task information from the model pool according to the priority levels and predetermined conflict relationships to form a target AI model group.

3. The AI ​​capability fusion scheduling method according to claim 2 is characterized in that: Step S31 includes the following sub-steps: S311. Determine the priority level of each AI capability type corresponding to the task information according to the task information and the user preference requirement information.

4. The AI ​​capability fusion scheduling method according to claim 3 is characterized in that: When the input data includes voice data, the following steps are further included between step S1 and step S2: SA, obtaining the user's voice feature information according to the input data, and obtaining the user's preference demand information according to the voice feature information.

5. The AI ​​capability fusion scheduling method according to claim 2, characterized in that: Step S32 includes: S321. Determine specific AI models corresponding to several AI capability types based on user needs or application needs; S322. Extract AI models corresponding to the remaining AI capability types corresponding to the task information from the model pool according to the priority level and the predetermined conflict relationship, and form the target AI model group with the designated AI model.

6. The AI ​​capability fusion scheduling method according to claim 1, characterized in that: The following steps are also included between step S3 and step S4: SB, extracting multiple AI models of various AI capability types matching the task information and different from the target AI model group from the model pool according to the priority level and the predetermined conflict relationship to form a backup AI model group; S5. When executing step S4, the backup AI model group is scheduled in the background according to the task information to process the input data to generate a backup result, and the backup result is output after the user is not satisfied with the processing result or the processing result fails to be generated.

7. The AI ​​capability fusion scheduling method according to claim 1, characterized in that: The model pool is composed of multiple AI models accessed through a standardized API.

8. An AI capability fusion scheduling device, applied in a central control platform, characterized in that: The AI ​​capability fusion scheduling device includes: An acquisition module, used to respond to a request from a product application and acquire input data and corresponding task information from the product application; A disassembly module, used to disassemble the task information to obtain multiple AI capability types; An extraction module, for jointly extracting AI models from a pre-connected model pool according to each of the AI ​​capability types under the guidance of the task information to form a target AI model group, wherein the model pool has a plurality of AI models corresponding to each AI capability type; A scheduling processing module is used to schedule the target AI model group to process the input data according to the task information to generate a processing result.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 7 are executed.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are executed.

Images