Method, device and apparatus for generating content understanding model and content understanding method
By determining the target configuration information and processing sample data, a content understanding model adapted to different business scenarios is generated, which solves the problems of complex and inefficient training in existing technologies and achieves efficient model generation and applicability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING ZITIAO NETWORK TECH CO LTD
- Filing Date
- 2023-09-08
- Publication Date
- 2026-06-09
Smart Images
Figure CN117197564B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and more specifically, to a method, content understanding method, apparatus, and device for generating a content understanding model. Background Technology
[0002] With the advancement of computer technology, content understanding models have been widely applied. These models can analyze input business data such as video, images, audio, and text to obtain content understanding results. As business data increases, the business scenarios involved in content understanding models become increasingly complex. Currently, related technologies require separate model training for each business scenario to generate suitable content understanding models, making the training and generation of content understanding models complex and inefficient. Summary of the Invention
[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] According to a first aspect of the present disclosure, a method for generating a content understanding model is provided, the method comprising:
[0005] Determine the target configuration information; the target configuration information includes the pre-set model training mode, target data format, preprocessing parameters, and model processing parameters;
[0006] Obtain sample data based on the target data format;
[0007] The sample data is preprocessed according to the preprocessing parameters to obtain sample features;
[0008] The preset model is trained based on the sample features and the model training mode to obtain the first target model;
[0009] The first target model is processed according to the model processing parameters to obtain a second target model; the second target model is used to perform content recognition on the target data to obtain the content understanding result of the target data.
[0010] According to a second aspect of the present disclosure, a content understanding method is provided, the method comprising:
[0011] Obtain the target data;
[0012] Obtain the target data;
[0013] Content recognition of the target data is performed using a second target model to obtain a content understanding result of the target data; wherein, the second target model is a model obtained based on the method for generating a content understanding model described in the first aspect of this disclosure.
[0014] According to a third aspect of the present disclosure, an apparatus for generating a content understanding model is provided, the apparatus comprising:
[0015] The determination module is used to determine the target configuration information; the target configuration information includes the pre-set model training mode, target data format, preprocessing parameters, and model processing parameters;
[0016] The first acquisition module is used to acquire sample data based on the target data format;
[0017] The first processing module is used to perform feature preprocessing on the sample data according to the preprocessing parameters to obtain sample features; to train a preset model according to the sample features and the model training mode to obtain a first target model; to perform preset model processing on the first target model according to the model processing parameters to obtain a second target model; the second target model is used to perform content recognition on the target data to obtain the content understanding result of the target data.
[0018] According to a fourth aspect of the present disclosure, a content understanding apparatus is provided, the apparatus comprising:
[0019] The second acquisition module is used to acquire target data;
[0020] The second processing module is used to perform content recognition on the target data through a second target model to obtain the content understanding result of the target data; wherein, the second target model is a model obtained based on the method for generating a content understanding model described in the first aspect of this disclosure.
[0021] According to a fifth aspect of the present disclosure, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first or second aspect of the present disclosure.
[0022] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising:
[0023] A storage device on which computer programs are stored;
[0024] A processing device for executing the computer program in the storage device to implement the steps of the method described in the first or second aspect of this disclosure.
[0025] By adopting the above technical solution, target configuration information is determined, including pre-set model training mode, target data format, preprocessing parameters, and model processing parameters. Sample data is obtained based on the target data format. Feature preprocessing is performed on the sample data according to the preprocessing parameters to obtain sample features. A pre-set model is trained based on the sample features and the model training mode to obtain a first target model. The first target model is then subjected to pre-set model processing according to the model processing parameters to obtain a second target model. This second target model is used to perform content recognition on the target data to obtain content understanding results. In this way, content understanding models adapted to different business scenarios can be flexibly generated based on the target configuration information, realizing a content understanding model training framework applicable to multiple business scenarios and improving the efficiency of model generation.
[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0027] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:
[0028] Figure 1 This is a flowchart illustrating a method for generating a content understanding model according to an embodiment of this disclosure.
[0029] Figure 2 This is a flowchart illustrating another method for generating a content understanding model according to embodiments of this disclosure.
[0030] Figure 3 This is a flowchart illustrating another method for generating a content understanding model according to embodiments of this disclosure.
[0031] Figure 4 This is a schematic diagram illustrating a branch management system according to an embodiment of the present disclosure.
[0032] Figure 5 This is a schematic diagram illustrating a data management method according to an embodiment of the present disclosure.
[0033] Figure 6 This is a flowchart illustrating a content understanding method according to an embodiment of the present disclosure.
[0034] Figure 7 This is a block diagram of an apparatus for generating a content understanding model according to an embodiment of the present disclosure.
[0035] Figure 8This is a block diagram illustrating a content understanding device according to an embodiment of the present disclosure.
[0036] Figure 9 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation
[0037] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0038] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0039] The term "comprising" and its variations as used in this disclosure are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the description below.
[0040] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0041] It should be noted that the terms "one" and "multiple" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that they should be understood as "one or more" unless explicitly stated in the context. In the description of this disclosure, unless otherwise stated, "multiple" means two or more, and other quantifiers are similar; "at least one," "one or more," or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one 'a' can represent any number of 'a's; as another example, one or more of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple; "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " indicates that the objects before and after it are in an "or" relationship. The singular forms "a," "a kind," "an item," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.
[0042] Although operations or steps are described in a specific order in the accompanying drawings in the embodiments of this disclosure, it should not be construed as requiring these operations or steps to be performed in the specific order or serial order shown, or requiring all of the shown operations or steps to be performed to obtain the desired result. In the embodiments of this disclosure, these operations or steps may be performed serially; they may be performed in parallel; or a portion of these operations or steps may be performed.
[0043] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0044] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0045] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
[0046] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0047] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0048] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.
[0049] The present disclosure will now be described in conjunction with specific embodiments.
[0050] Figure 1 This is a flowchart illustrating a method for generating a content understanding model according to embodiments of this disclosure. The method can be applied to electronic devices, which may include terminal devices such as smartphones, smart wearable devices, smart speakers, smart tablets, PDAs (Personal Digital Assistants), CPEs (Customer Premise Equipment), personal computers, in-vehicle terminals, etc.; the electronic device may also include a server, such as a local server or a cloud server. Figure 1 As shown, the method may include:
[0051] S101. Obtain sample data.
[0052] The sample data can be labeled or unlabeled, and this disclosure does not limit it.
[0053] In some embodiments, the sample data may include at least one of the following: video, images, audio, text, etc. For example, the sample data may be multimodal data including images, audio, and text.
[0054] S102. Train the preset model based on the sample data and target configuration information to obtain the target model.
[0055] The target model is used to perform content recognition on target data to obtain content understanding results of the target data, and the target configuration information includes a pre-set model training mode.
[0056] In some embodiments, the model training mode includes any one of the following: distillation training mode, semi-supervised training mode, supervised training mode, language model training mode, and normal training process.
[0057] It should be noted that this model training mode can also be referred to as the model training process. In some embodiments, the same model training mode (e.g., the normal training mode) can be reused in different business scenarios. In this way, the process of training the preset model is relatively fixed. In other embodiments, different model training modes can be applied to different business scenarios. For example, for semi-supervised training, distillation, and other scenarios that require significant adjustments to the training process, new model training modes can be registered and configured and used through the model training modes in the target configuration information.
[0058] In some embodiments, the target model may be a first target model.
[0059] In some other embodiments, the target model can be a second target model. For example, the second target model can be obtained by performing preset model processing on the first target model according to the model processing parameters.
[0060] Using the above method, sample data is acquired, and a preset model is trained based on the sample data and target configuration information to obtain the target model. This target model is used to perform content recognition on the target data to obtain content understanding results. The target configuration information includes pre-set model training modes. This allows for the flexible generation of content understanding models adapted to different business scenarios based on pre-set model training modes, realizing a content understanding model training framework applicable to multiple business scenarios and improving the efficiency of model generation.
[0061] Figure 2 This is a flowchart illustrating another method for generating a content understanding model according to embodiments of this disclosure. This method can be applied to electronic devices. Figure 2 As shown, the method may include:
[0062] S201. Determine the target configuration information.
[0063] In some embodiments, the target configuration information may include at least one of the following:
[0064] Model training mode, which indicates the mode in which the model is trained;
[0065] The target data format, which is used to indicate the format for storing and / or retrieving sample data;
[0066] Preprocessing parameters, which indicate the method of feature preprocessing for the sample data;
[0067] Model processing parameters, which indicate the post-processing method for the first target model.
[0068] In some embodiments, the target configuration information may include pre-set model training mode, target data format, preprocessing parameters, and model processing parameters.
[0069] In one implementation, the model training mode can include any of the following: distillation training mode, semi-supervised training mode, supervised training mode, language model training mode, and normal training process.
[0070] In one implementation, the target data format may include the Parquet format, which can be used to indicate that sample data is stored and retrieved based on the Parquet format.
[0071] In one implementation, the preprocessing parameter may include at least one of the following: the preprocessing operator corresponding to the sample data, and the data encoding / decoding format corresponding to the sample data.
[0072] In one implementation, the model processing parameter can be used to indicate a preset model processing for the first target model, the preset model processing including at least one of the following:
[0073] Floating-point parameter conversion (e.g., converting float point parameters to hexadecimal);
[0074] Model conversion (onnx2TRT) processing;
[0075] FasterTransformer processing;
[0076] Layer optimization processing.
[0077] Based on this target configuration information, configuration can be achieved. Configuration is the most basic and important function of a framework. Configuration is the blueprint for the process and a prerequisite for rapid experimentation. Different developers can quickly understand the relevant situation of the experiment through configuration, so as to easily reuse various modules. Through the accumulation of a large number of business experiences, we have developed an efficient configuration strategy: dividing the configuration into default configuration, task configuration, and real-time configuration. The actual training configuration will be the sum of the three, with the coverage priority as follows: real-time configuration > task configuration > default configuration.
[0078] There are several ways to determine the target configuration information, for example:
[0079] In some embodiments, at least one preset configuration information and a configuration method corresponding to each preset configuration information may be determined; target configuration information may be determined from at least one preset configuration information according to the priority of the preset configuration method.
[0080] The configuration methods include at least one of the following: global configuration (also known as default configuration), task configuration, and temporary configuration (also known as real-time configuration).
[0081] The global configuration can be a configuration shared by multiple target tasks, the task configuration can be a configuration corresponding to a single target task, and the temporary configuration can be a configuration temporarily set by the user before or during the execution of the target task.
[0082] In some embodiments, the priority of the above-mentioned preset configuration method may include: the priority of temporary configuration is greater than that of task configuration, and the priority of task configuration is greater than that of global configuration, that is: temporary configuration > task configuration > global configuration.
[0083] In some embodiments, the preset configuration information corresponding to the highest priority configuration method can be used as the target configuration information. For example, parameters overridden by tasks can override parameters in the global configuration, and parameters in temporary configuration can override parameters in the task configuration.
[0084] Take the configuration of the optimizer, a common component in model training, as an example:
[0085] Global configuration: This allows the system's built-in configuration to be used by default if no optimizer is specified for the target task. For example:
[0086] "optimizer":{
[0087] "optimizer_name":"AdamW",
[0088] "learning_rate": 0.0002
[0089] }
[0090] Task configuration: Configuration specific to the target task. For example, if the target task specifies the use of Adam with a learning rate of 0.001, this will override the system configuration.
[0091] "optimizer":{
[0092] "optimizer_name":"Adam",
[0093] "learning_rate": 0.0001
[0094] }
[0095] Temporary configuration: Configurations temporarily set by the user before or during the execution of a target task. For example, based on a task configuration, if you want to compare the effects of different optimizers, you can modify the configuration in real time through external commands. For instance, if an experiment compares the effects of the Adam and AdamW optimizers, it can be done using the following two commands on the same task_config, supporting parallel execution:
[0096] 1.bash train.sh--conf task_config optimizer.optimizer_name=AdamW;
[0097] 2.bash train.sh--conf task_config optimizer.optimizer_name=Adam;
[0098] S202. Obtain sample data based on the target data format.
[0099] In some embodiments, the target data format may be the data format corresponding to the sample data included in the target configuration information.
[0100] In some embodiments, the process of acquiring sample data may include generating sample data and reading sample data, wherein generating sample data can be achieved through data production, and reading sample data can be achieved through data factory. For example:
[0101] Data production: In terms of data format, a unified target data format (e.g., Parquet) can be used in a columnar storage manner. This effectively filters the features needed by the model and facilitates feature refresh. In terms of production, a cluster mode (e.g., PySpark on Yarn) can be used to improve data production efficiency through distributed packaging. For example, it is possible to complete the packaging of 220 million data points within 2 days.
[0102] Data Factory: Supports reading data in target data formats. Optionally, it can support parallel reading of large-scale data (hundreds of millions) across multiple machines and GPUs, mixed reading of multiple datasets, and breakpoint reading of data streams, ultimately feeding the data to the model in the form of tensors.
[0103] In some embodiments, a distributed system may be used to store and / or retrieve sample data, such as the Hadoop Distributed File System (HDFS).
[0104] S203. Perform feature preprocessing on the sample data according to the preprocessing parameters to obtain sample features.
[0105] The above preprocessing parameters include at least one of the following: the preprocessing operator corresponding to the sample data, and the data encoding / decoding format corresponding to the sample data.
[0106] For example, after reading sample data from a distributed system, it can undergo a decoding process to obtain raw data. Then, it will go through a conversion process to transform it into the corresponding data encoding / decoding format (e.g., Tensor format). Optionally, different preprocessing operators can be configured for different data types such as images, text, audio, and arrays. Preprocessing for different modalities can be combined according to the business scenario.
[0107] S204. Train the preset model according to the sample features and model training mode to obtain the first target model.
[0108] In one implementation, the model training mode can include any of the following: distillation training mode, semi-supervised training mode, supervised training mode, language model training mode, and normal training process.
[0109] In some embodiments, multiple pre-trained models can be managed based on a model marketplace (ModelZoo). These pre-trained models can be pre-trained models that have accumulated a large amount of business data, and multiple pre-trained models can be managed in a unified manner through ModelZoo for reuse across different training frameworks.
[0110] In some embodiments, the process of training a model can be a combination of the "data" and "model" processes, so that the model can continuously refresh its weights through the forward feed of data and the feedback of labels, and produce the optimal model weights under specific metrics.
[0111] In some embodiments, the first target model can also be tested. The testing process differs from training in that the model does not participate in backpropagation. Optionally, the model's output can be recorded for model evaluation.
[0112] S205. Perform preset model processing on the first target model according to the model processing parameters to obtain the second target model.
[0113] The second target model can be used to perform content recognition on target data to obtain content understanding results of the target data.
[0114] The model processing parameters can be parameters included in the target configuration information, used to indicate the preset model processing for the first target model.
[0115] In some embodiments, preset model processing includes at least one of the following:
[0116] Floating-point parameter conversion (e.g., converting float point parameters to hexadecimal);
[0117] Model conversion (onnx2TRT) processing;
[0118] FasterTransformer processing;
[0119] Layer optimization processing.
[0120] In some embodiments, model-level optimizations can be performed using floating-point parameter conversion (e.g., parameter float point 16 conversion), onnx2TRT, FasterTransformer, and Layer Optimization; for image preprocessing, some operators can be accelerated using the Compute Unified Device Architecture (CUDA).
[0121] This allows for model optimization and improves computational efficiency.
[0122] In some embodiments, after the first target model is transformed and optimized to obtain the second target model, the second target model can be deployed on the inference framework and some online preparation tests can be performed, such as evaluating whether the results of the online service are different from those offline, and performance stress testing.
[0123] In some embodiments, a second target model can be obtained by compiling the first target model through a compilation module. The second target model can be deployed on an inference framework. For example, the compiled second target model can be an intermediate structure of Open Neural Network Exchange (ONNX), which is convenient for deployment to a downstream C++ service environment.
[0124] By adopting the above training framework and methods, the reuse of models and / or data processing between business processes can be achieved, thereby improving the efficiency of content understanding model generation and iteration.
[0125] Figure 3 This is a flowchart illustrating another method for generating a content understanding model according to embodiments of this disclosure. This method can be applied to electronic devices. Figure 3 As shown, the method may include:
[0126] S301. Obtain the first business development data corresponding to the first content understanding business from the server.
[0127] The first business development data may include business configuration data, business model, and feature preprocessing module corresponding to the first content understanding business.
[0128] S302. Based on the second content understanding business, update the first business development data to obtain the second business development data corresponding to the second content understanding business.
[0129] S303. Merge the second business development data with the first business development data and store them on the server.
[0130] In some embodiments, the first business development data and the second business development data described above can be data stored based on a development branch. The second business development data and the first business development data can be merged and stored in the development branch on the server, or they can be stored in the master branch on the server.
[0131] Figure 4 This is a schematic diagram illustrating branch management according to an embodiment of this disclosure. For example... Figure 4 As shown, the development of the content understanding model can be achieved through branch management. The Master branch can be used to store and deploy stable models, while the Develop branch can be used for model development. Optionally, the Develop branch can be used for global development, and different business units can branch out from the Develop branch for different development tasks.
[0132] In some embodiments, the Develop branch can be divided into business branches (e.g., Figure 4 (bA, bB, bC) and functional branches (e.g.) Figure 4 In the framework of Master and Master branches (fA and fB), business branches manage different business scenarios and have a longer lifecycle. Functionally, different branches are created for different functions, allowing for agile development and resulting in a shorter lifecycle. Once a function is developed, it can be merged into the Master branch. Businesses obtain functional updates through the Master branch, and these updates can be merged into the Master branch after accumulating a certain amount of information. Optionally, the Master branch can act as a proxy for the Master branch, providing the latest functionality but not necessarily the most stable.
[0133] In this way, through the above branch management, the model training framework can be developed in parallel for various business functions, thereby improving the efficiency of model development.
[0134] In some embodiments, the server may store business development data corresponding to multiple content understanding services. The business development data includes business configuration data, business models, and feature preprocessing modules corresponding to the content understanding services. The business configuration data, business models, and feature preprocessing modules are stored independently on the server.
[0135] Figure 5 This is a schematic diagram illustrating a data management system according to an embodiment of this disclosure. Figure 5 As shown, the business development data for multiple content understanding services (A / B / C) all include business configuration data, business models, and feature preprocessing modules, which can be reused among themselves. For example, if business understanding content A and business understanding content B have the same feature preprocessing module, that module can be directly reused. If business understanding content B and business understanding content C have the same business model, that model can be directly reused. In this way, similar business understanding content can quickly reference or reuse existing data. Furthermore, the business configuration data, business models, and feature preprocessing modules are stored independently, ensuring that different services do not interfere with each other during parallel development. Different service developments share a single environment; problems can be reproduced and optimized collaboratively by switching branches.
[0136] In this way, through the aforementioned branch management and data management, the model training framework can be developed in parallel for various business operations, while similar business operations can learn from each other, thereby improving the efficiency of model development.
[0137] Figure 6 This is a flowchart illustrating a content understanding method according to an embodiment of this disclosure. This method can be applied to electronic devices. Figure 6 As shown, the method may include:
[0138] S601, Obtain target data.
[0139] S602. Perform content recognition on the target data using the target model to obtain the content understanding results of the target data.
[0140] The target model can be either a first target model or a second target model.
[0141] In some embodiments, the target model may be a second target model obtained by any optional implementation of the method for generating a content understanding model based on the foregoing embodiments of this disclosure.
[0142] In other embodiments, the target model may also be a first target model obtained by any optional implementation of the method for generating content understanding models in the foregoing embodiments of this disclosure.
[0143] Figure 7 This is a block diagram illustrating an apparatus for generating a content understanding model according to embodiments of the present disclosure. Figure 7 As shown, the apparatus 1100 for generating the content understanding model may include:
[0144] The determination module 1101 is used to determine the target configuration information; the target configuration information includes the pre-set model training mode, target data format, preprocessing parameters and model processing parameters;
[0145] The first acquisition module 1102 is used to acquire sample data based on the target data format;
[0146] The first processing module 1103 is used to perform feature preprocessing on the sample data according to the preprocessing parameters to obtain sample features; to train a preset model according to the sample features and the model training mode to obtain a first target model; to perform preset model processing on the first target model according to the model processing parameters to obtain a second target model; the second target model is used to perform content recognition on the target data to obtain the content understanding result of the target data.
[0147] In some embodiments, the model training mode includes any one of the following: distillation training mode, semi-supervised training mode, supervised training mode, and language model training mode.
[0148] In some embodiments, the preprocessing parameters include at least one of the following:
[0149] The preprocessing operator corresponding to the sample data;
[0150] The data encoding / decoding format corresponding to the sample data.
[0151] In some embodiments, the target configuration information further includes model processing parameters, and the first processing module 1103 is further configured to perform preset model processing on the first target model according to the model processing parameters to obtain a second target model.
[0152] In some embodiments, the preset model processing includes at least one of the following:
[0153] Floating-point parameter conversion;
[0154] Model conversion processing;
[0155] Fast converter processing;
[0156] Layer optimization processing.
[0157] In some embodiments, the first processing module 1103 is further configured to determine at least one preset configuration information and a configuration method corresponding to each preset configuration information; wherein, the configuration method includes at least one of global configuration, task configuration and temporary configuration, the global configuration is a configuration shared by multiple target tasks, the task configuration is a configuration corresponding to one target task, and the temporary configuration is a configuration temporarily set by the user before or during the execution of the target task;
[0158] The target configuration information is determined from at least one preset configuration information according to the priority of the preset configuration method.
[0159] In some embodiments, the first processing module 1103 is further configured to use the preset configuration information corresponding to the configuration method with the highest priority as the target configuration information.
[0160] In some embodiments, the priority of the preset configuration method includes: the priority of the temporary configuration is greater than that of the task configuration, and the priority of the task configuration is greater than that of the global configuration.
[0161] In some embodiments, the first processing module 1103 is further configured to obtain first business development data corresponding to the first content understanding service from the server; wherein the first business development data includes business configuration data, business model and feature preprocessing module corresponding to the first content understanding service; update the first business development data according to the second content understanding service to obtain second business development data corresponding to the second content understanding service; and merge the second business development data and the first business development data and store them in the server.
[0162] In some embodiments, the server stores business development data corresponding to multiple content understanding services. The business development data includes business configuration data, business models, and feature preprocessing modules corresponding to the content understanding services. The business configuration data, business models, and feature preprocessing modules are stored independently on the server.
[0163] Figure 8 This is a block diagram illustrating a content understanding device according to an embodiment of the present disclosure. Figure 8 As shown, the content understanding device 1200 may include:
[0164] The second acquisition module 1201 is used to acquire target data;
[0165] The second processing module 1202 is used to perform content recognition on the target data through a second target model to obtain the content understanding result of the target data; wherein, the second target model is a model obtained based on the method for generating a content understanding model in the foregoing embodiments of this disclosure.
[0166] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0167] The following is for reference. Figure 9This document illustrates a structural diagram of an electronic device 2000 (e.g., a terminal device or a server) suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The server in the embodiments of the present disclosure may include, but is not limited to, local servers, cloud servers, single servers, and distributed servers. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0168] like Figure 9 As shown, electronic device 2000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 2001, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 2002 or a program loaded from storage device 2008 into random access memory (RAM) 2003. RAM 2003 also stores various programs and data required for the operation of electronic device 2000. Processing device 2001, ROM 2002, and RAM 2003 are interconnected via bus 2004. Input / output (I / O) interface 2005 is also connected to bus 2004.
[0169] Typically, the following devices can be connected to the input / output interface 2005: input devices 2006 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 2007 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 2008 including, for example, magnetic tapes, hard disks, etc.; and communication devices 2009. Communication device 2009 allows electronic device 2000 to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 9 An electronic device 2000 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0170] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 2009, or installed from storage device 2008, or installed from ROM 2002. When the computer program is executed by processing device 2001, it performs the functions defined in the methods of embodiments of this disclosure.
[0171] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0172] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0173] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0174] The aforementioned computer-readable medium carries one or more programs. When the electronic device executes one or more of these programs, the electronic device causes the following actions: determining target configuration information; the target configuration information includes a pre-set model training mode, target data format, preprocessing parameters, and model processing parameters; acquiring sample data based on the target data format; performing feature preprocessing on the sample data according to the preprocessing parameters to obtain sample features; training a preset model according to the sample features and the model training mode to obtain a first target model; performing preset model processing on the first target model according to the model processing parameters to obtain a second target model; the second target model is used to perform content recognition on the target data to obtain a content understanding result of the target data.
[0175] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire target data; perform content recognition on the target data using a second target model to obtain a content understanding result of the target data; wherein the second target model is a model obtained based on the method for generating a content understanding model in the foregoing embodiments of this disclosure.
[0176] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0178] The modules described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the modules are not necessarily limiting in certain circumstances; for example, the first acquisition module can also be described as a "module for acquiring sample data".
[0179] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0180] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0181] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0182] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0183] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.
Claims
1. A method for generating a content understanding model, characterized in that, The method includes: Determine the target configuration information; the target configuration information includes the pre-set model training mode, target data format, preprocessing parameters, and model processing parameters; Obtain sample data based on the target data format; The sample data is preprocessed according to the preprocessing parameters to obtain sample features; The preset model is trained based on the sample features and the model training mode to obtain the first target model; The first target model is processed according to the model processing parameters to obtain a second target model; the second target model is used to perform content recognition on the target data to obtain the content understanding result of the target data; The method further includes: Determine at least one preset configuration information and a configuration method corresponding to each preset configuration information; wherein, the configuration method includes at least one of global configuration, task configuration and temporary configuration, the global configuration is a configuration shared by multiple target tasks, the task configuration is a configuration corresponding to one target task, and the temporary configuration is a configuration temporarily set by the user before or during the execution of the target task; The target configuration information is determined from at least one preset configuration information according to the priority of the preset configuration method.
2. The method according to claim 1, characterized in that, The model training mode includes any one of the following: distillation training mode, semi-supervised training mode, supervised training mode, and language model training mode.
3. The method according to claim 1, characterized in that, The preprocessing parameters include at least one of the following: The preprocessing operator corresponding to the sample data; The data encoding / decoding format corresponding to the sample data.
4. The method according to claim 1, characterized in that, The preset model processing includes at least one of the following: Floating-point parameter conversion; Model conversion processing; Fast converter processing; Layer optimization processing.
5. The method according to claim 1, characterized in that, The step of determining the target configuration information from the at least one preset configuration information according to the priority of the preset configuration method includes: The preset configuration information corresponding to the highest priority configuration method is used as the target configuration information.
6. The method according to claim 1, characterized in that, The priority of the preset configuration method includes: the priority of the temporary configuration is greater than that of the task configuration, and the priority of the task configuration is greater than that of the global configuration.
7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain first business development data corresponding to the first content understanding business from the server; wherein, the first business development data includes business configuration data, business model and feature preprocessing module corresponding to the first content understanding business; Based on the second content understanding business, the first business development data is updated to obtain the second business development data corresponding to the second content understanding business; The second business development data and the first business development data are merged and stored on the server.
8. The method according to claim 7, characterized in that, The server stores business development data corresponding to multiple content understanding services. The business development data includes business configuration data, business models, and feature preprocessing modules corresponding to the content understanding services. The business configuration data, business models, and feature preprocessing modules are stored independently on the server.
9. A content understanding method, characterized in that, The method includes: Obtain the target data; The target data is subjected to content recognition by a second target model to obtain the content understanding result of the target data; wherein, the second target model is a model obtained by the method of generating a content understanding model based on any one of claims 1 to 8.
10. An apparatus for generating a content understanding model, characterized in that, The device includes: The determination module is used to determine the target configuration information; the target configuration information includes the pre-set model training mode, target data format, preprocessing parameters, and model processing parameters; The first acquisition module is used to acquire sample data based on the target data format; The first processing module is used to perform feature preprocessing on the sample data according to the preprocessing parameters to obtain sample features; to train a preset model according to the sample features and the model training mode to obtain a first target model; to perform preset model processing on the first target model according to the model processing parameters to obtain a second target model; the second target model is used to perform content recognition on the target data to obtain the content understanding result of the target data; The first processing module is further configured to determine at least one preset configuration information and a configuration method corresponding to each preset configuration information; wherein, the configuration method includes at least one of global configuration, task configuration and temporary configuration, the global configuration is a configuration shared by multiple target tasks, the task configuration is a configuration corresponding to one target task, and the temporary configuration is a configuration temporarily set by the user before or during the execution of the target task; and to determine the target configuration information from the at least one preset configuration information according to the priority of the preset configuration method.
11. A content understanding device, characterized in that, The device includes: The second acquisition module is used to acquire target data; The second processing module is used to perform content recognition on the target data through a second target model to obtain the content understanding result of the target data; wherein the second target model is a model obtained based on the method for generating a content understanding model according to any one of claims 1 to 8.
12. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processing device, it implements the steps of the method according to any one of claims 1 to 8, or when the computer program is executed by the processing device, it implements the steps of the method according to claim 9.
13. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing apparatus for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8, or to implement the steps of the method according to claim 9.