Business processing method, apparatus, system, device, storage medium and program product

Abstract

The application discloses a business processing method, device, system, equipment, storage medium and program product. The method comprises the following steps: generating a policy execution flow according to user-input policy customization information, wherein the policy execution flow comprises identification of a target model and policy execution information; inputting to-be-processed business data into the target model and obtaining an output result of the target model; generating a target basic signal based on the output result; and processing the target basic signal according to the policy execution information to obtain a processing result that can adapt to a large number of business demands related to human factors, so that the processing rule can be coped with, and the processing capacity of the system is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a business processing method, a business processing apparatus, a business processing system, a business processing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Considering the real-time and scalable nature of short video / image review and recommendation, most review and recommendation processes currently tend to use a combination of machine and human processes, with machine processing taking a primary role. For example, machine review can detect and address over 90% of violations at high speed while maintaining a certain level of accuracy.

[0003] Current mainstream deep learning machine review and recommendation systems tend to use a flat approach to address a wide range of complex review and recommendation scenarios, employing multiple models with different functionalities in parallel to address diverse business needs. This approach works well for tasks with limited human understanding, such as identifying pets, cars, guns, and knives. However, when the requirements are combined with the human understanding of the context, the problem becomes abstract and complex, significantly reducing the processing power of systems based on this flat approach. Because the short video and image business landscape changes rapidly over time, review and recommendation scenarios are often complex and frequently vary with data distribution. Therefore, pattern-based review and recommendation methods often fail to keep pace with these changes, resulting in significantly reduced effectiveness and far less flexibility than manual processing. Furthermore, because review and recommendation scenarios are also susceptible to human understanding, flat pattern recognition review and recommendation methods struggle to cope with complex and ever-changing review and recommendation rules. Summary of the Invention

[0004] The present application provides a business processing method, apparatus, system, equipment, storage medium and program product to solve the problem in the prior art that when the demand is combined with the human understanding factors of the environment in which it is located, the system processing capability based on the planar method will be greatly reduced, and it will be difficult to cope with complex and changeable review / recommendation rules.

[0005] In a first aspect, an embodiment of the present application provides a service processing method, the method comprising:

[0006] Generate a policy execution flow according to the policy customization information input by the user, wherein the policy execution flow includes an identifier of the target model and policy execution information;

[0007] Inputting the business data to be processed into the target model and obtaining the output result of the target model;

[0008] generating a target basic signal based on the output result;

[0009] The target basic signal is processed according to the strategy execution information.

[0010] In a second aspect, an embodiment of the present application further provides a business processing system, the business processing system comprising a basic signal module, a decision module, and a policy construction layer, wherein the basic signal module comprises a basic model set, and the decision module comprises a decision model set;

[0011] The policy construction layer is used to provide a policy customization interface, receive policy customization information input by a user from the policy customization interface, and generate a policy execution flow according to the policy customization information, wherein the policy execution flow includes an identifier of a target model and policy execution information;

[0012] The basic signal module is used to determine the corresponding target basic model from the basic model set according to the identifier of the target model, and use the target basic model to process the service data to be processed, obtain the output result of the target basic model and generate a target basic signal;

[0013] The decision module is used to determine the corresponding target decision model from the decision model set according to the identifier of the target model, and use the target decision model to process the business data to be processed to obtain decision information output by the target decision model;

[0014] The strategy construction layer is further configured to perform fusion processing on the target basic signal and the decision information according to the strategy execution information.

[0015] In a third aspect, an embodiment of the present application further provides a service processing device, the device comprising:

[0016] A policy execution flow generating unit, configured to generate a policy execution flow according to the policy customization information input by the user, wherein the policy execution flow includes an identifier of the target model and policy execution information;

[0017] A target model processing unit, configured to input the business data to be processed into the target model and obtain an output result of the target model;

[0018] a basic signal generating unit, configured to generate a target basic signal based on the output result;

[0019] A signal processing unit is configured to process the target basic signal according to the policy execution information.

[0020] In a fourth aspect, an embodiment of the present application further provides a service processing device, the service processing device comprising:

[0021] one or more processors;

[0022] a storage device for storing one or more programs,

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the method of the first aspect described above.

[0024] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method of the first aspect described above is implemented.

[0025] In a sixth aspect, an embodiment of the present application further provides a computer program product, which includes computer-executable instructions, and the computer-executable instructions are used to implement the method of the first aspect when executed.

[0026] The technical solution provided in this application has the following beneficial effects:

[0027] In this embodiment, a policy execution flow is generated based on the policy customization information input by the user. The policy execution flow includes the identifier of the target model and the policy execution information to meet the complex requirements related to various human understanding factors. Then, when the business data to be processed is processed, the business data to be processed is input into the target model, and the output result signal of the obtained target model is converted into a target basic signal. This allows different complex requirements to be expressed through a set of basic signals, reducing the amount of information transmitted by the signal, so its multiplexing capability is relatively strong. The target basic signal is then processed according to the policy execution information to obtain a processing result that can adapt to a large number of business requirements related to human factors, can cope with complex and changing processing rules, and improve the processing capability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flowchart of an embodiment of a business processing method provided in Example 1 of the present application;

[0029] Figure 2 This is a structural block diagram of a business processing system embodiment provided in Example 2 of the present application;

[0030] Figure 3 This is a schematic diagram of the execution process of a business processing system provided in Example 2 of the present application during application and the interaction with users or other practical modules at different stages;

[0031] Figure 4 This is a schematic diagram of the policy execution information composition of a policy construction layer provided in Example 2 of the present application;

[0032] Figure 5 This is a signal integration schematic diagram provided in Example 2 of the present application;

[0033] Figure 6 This is a schematic diagram of a method for integrating data of different data granularities provided in Example 2 of the present application;

[0034] Figure 7 This is a structural block diagram of a service processing device embodiment provided in Embodiment 3 of the present application;

[0035] Figure 8 This is a structural diagram of a business processing device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the present application, not all of the structures.

[0037] Example 1

[0038] Figure 1 This is a flowchart of a business processing method embodiment provided in the first embodiment of the present application. This embodiment can be executed by a processing system and can be applied to various business processing scenarios, for example, business review or business recommendation scenarios, such as short video or image review or recommendation scenarios. Figure 1 As shown, this embodiment may include the following steps:

[0039] Step 110: Generate a policy execution flow according to the policy customization information input by the user, wherein the policy execution flow includes an identifier of the target model and policy execution information.

[0040] The policy execution flow refers to the execution process of a processing policy. This processing policy is a customized execution strategy for processing business data to meet complex requirements related to various human understanding factors. This policy execution flow can include the identification of the target model and policy execution information.

[0041] The model involved in the policy execution flow is called the target model. Depending on different business needs, the target model can be different, and there can be multiple target models.

[0042] In one embodiment, step 110 may further include the following steps:

[0043] Step 110 - 1 : Acquire policy customization information input by a user, where the policy customization information includes policy requirement information and policy execution information.

[0044] In one implementation, a policy customization interface can be displayed to the user, which may include multiple element information for customizing processing policies. The user can select target element information from the policy customization interface and enter corresponding information according to actual needs to generate policy customization information.

[0045] In other embodiments, the user may also input the policy customization information through commands such as command lines, which is not limited in this embodiment.

[0046] Through the policy customization information input by the user, the user can configure artificial conditions more flexibly, thereby enhancing the flexibility of handling business needs.

[0047] For example, policy customization information may include policy requirement information and policy execution information. Policy requirement information indicates the scenarios to which the current processing policy applies, including the required scenarios and processing objectives corresponding to the currently formulated processing policy. Examples of required scenarios include scene classification, behavior recognition, age regression, gender recognition, voice recognition, and special effects processing; and examples of processing objectives include content understanding and content review.

[0048] Policy execution information is used to define one or more constraint elements during the execution of the processing policy. The policy execution information may include one or more different types of constraint elements, each of which can constrain different execution directions of the processing policy. Different constraint elements can be set based on different business requirements, but this embodiment does not limit this. For example, the policy execution information may include constraint elements such as specified data granularity, specified embedding operations, post-processing operations, and preset post-processing policies.

[0049] Specifically, this embodiment may include the logic of policy application at different data granularities. Users can set the specified data granularity to build the policy execution flow according to actual needs. For example, a variety of data granularities can be provided for users to choose from in the policy customization interface, and users can select one of the data granularities as the specified data granularity as needed. In a further example, this embodiment can provide the following four data granularities from high to low: video granularity, frame-level granularity, image granularity, and feature granularity. Among them, video granularity and frame-level granularity refer to treating the entire video as a processing unit for processing, without considering the situation of each frame separately.

[0050] The specified embedded operation represents the logic used by the signal, that is, how the signals from different sources will be combined. Users can set the specified embedded operation to build the policy execution flow according to actual needs. For example, a variety of embedded operations can be provided for users to choose from in the policy customization interface, and users can select one of the embedded operations as the specified embedded operation according to their needs. In a further example, the embedded operation may include: filtering operation, integration operation and cascade operation. Among them, the filtering operation will carry a filtering strategy, which is used to block illegal outputs according to the filtering strategy. The integration operation will carry an integration strategy, which is used to fuse signals from different sources according to the integration strategy; the cascade operation will also have a corresponding cascade strategy, which is used to cascade the signals from different sources according to the cascade strategy.

[0051] It should be noted that the above three embedding operations also need to arrange the actual execution flow according to the task type and organization type of the signal. Different types of signals correspond to the three operations in different application methods.

[0052] Post-processing operations and pre-defined post-processing strategies are optional. A variety of post-processing operations are available for selection in the strategy customization interface, allowing users to select the desired operation. For example, these operations can include normalization, generating statistical metrics for specific signal types and output results, and generating specific outputs according to post-processing strategies. Furthermore, certain data that requires caching or transfer can also be centralized in the post-processing stage.

[0053] In other embodiments, the policy execution information may further include parameter reception strength, parameter reception type, etc. specified by the user, which are used to filter the basic signals.

[0054] Step 110 - 2 : Selecting an identifier of a target model that matches the policy requirement information from a pre-generated model set.

[0055] The models in the model set may be models pre-selected by developers. In one implementation, based on the demand scenarios and processing objectives included in the policy requirement information, the model set may include a first model set that matches the demand scenarios and a second model set that matches the processing objectives.

[0056] For example, the first model set may include basic models for extracting objective basic information, such as behavior recognition models, scene classification models, age regression models, gender models, speech recognition models, and special effects processing models. The second model set may include decision models for extracting decision information, such as a subset of audit models and a subset of content understanding models. The audit model subset may include, for example, a gun recognition model, a blood detection model, a knife detection model, and a sensitive word detection model; the content understanding model subset may include a multimodal integration model and a user profiling model.

[0057] After determining the policy requirement information input by the user, a model matching the policy requirement information can be selected from the model set as the target model. For example, a target base model can be determined from the first model set based on the requirement scenario, and a target decision model can be determined from the second model set based on the processing goal.

[0058] Step 110 - 3 : Generate a policy execution flow associated with the policy customization information according to the identifier of the target model and the policy execution information.

[0059] After determining the identification of the target model and the policy execution information, the two can be combined into a policy execution flow. Since the identification of the target model and the policy execution information are determined based on the demand information input by the user, the policy execution flow is a personalized execution flow that conforms to human factors or human-defined rules.

[0060] Step 120: Input the business data to be processed into the target model, and obtain the output result of the target model.

[0061] Among them, this embodiment does not limit the form of the business data to be processed, which can include pictures, short videos, videos, texts, etc.

[0062] During implementation, the business data to be processed can be data generated in real time, for example, short video data obtained in real time in a short video application scenario, or live video data in a live broadcast scenario, or content data containing pictures, etc. This embodiment does not limit this.

[0063] This embodiment can use the policy execution flow to process the business data to be processed according to the processing policy. First, the business data to be processed can be input into each target model of the policy execution flow for processing, and the output results of each target model can be obtained.

[0064] Among them, the business data to be processed can be input into different target models at the same time, or can be input into different target models in a set order. Those skilled in the art can set the input order according to actual needs, and this embodiment does not limit this.

[0065] Step 130: Generate a target basic signal based on the output result.

[0066] After obtaining the output results of each target model, the output results can be converted into basic signals to generate target basic signals corresponding to the output results. The target basic signals are used as basic data for executing the processing strategy.

[0067] In one embodiment, if the target model includes a target base model, the output result corresponding to the target base model is multimodal information. In this embodiment, the policy execution information includes a specified data granularity, and step 130 may further include the following steps:

[0068] Step 130 - 1 : Generate data of the specified data granularity from the multimodal information.

[0069] In order to meet the strategic requirements of the specified data granularity, after obtaining the multimodal information output by the target basic model, it is also necessary to integrate the multimodal information to form data of the specified granularity.

[0070] This embodiment can support highly rich multimodal information, such as signals from multiple deep learning visual tasks, deep learning speech sequence signals, etc. This highly rich multimodal information can be provided by a set of visual or audio models that encompass generalized information.

[0071] The multimodal information can also be represented by different data granularities, such as video granularity, frame granularity, image granularity, feature granularity, etc. These data granularities can be understood as the smallest elements representing multimodal information. The smallest elements of multimodal information for different tasks can also be different. For example, the smallest element of the key point recognition task can be [key point number, key point location name, key point coordinates]<x,y> ] definition; the minimum element of the box detection task can be [category number, coordinates of the upper left corner of the box<x,y> , frame size coordinates<w,h> ] definition; the minimum element of single / multi-classification task can be represented by [confidence] vector; the minimum element of semantic segmentation task can be represented by [category number, object pixel coordinates<x,y> ] to illustrate; the smallest element of the speech recognition task can be defined by a discrete sound wave vector, and so on.

[0072] Since the smallest element itself also has the attribute of data granularity, the method is more flexible when constructing data of different specified data granularities.

[0073] In one embodiment, if the specified data granularity includes video granularity, and the multimodal information is represented by a set minimum element (such as a feature element), step 130-1 may further include the following steps:

[0074] Step 130-1-1: Integrate the multimodal information into frame-level granularity data according to a set first integration strategy.

[0075] This step is the integration from the smallest element to the frame-level information, which can also be called the first-level integration. Taking into account that the smallest elements are often driven by the type of task they are in and are relatively diverse, the first integration strategy can be an integration strategy that uses simple integration methods to calculate the integrated information obtained by the fusion of the smallest elements according to different task types (such as pixel-level characters, local image tasks, image-level tasks, etc.). For example, the integrated information (i.e., frame-level granularity data) calculated using the first integration strategy can be: smoothing the pixel points output by the segmentation task and then calculating the total metric of the heat map, or using "non-maximum suppression" on the box output by the detection task to obtain the core box, or obtaining the representative category of the image based on the box ratio, and so on.

[0076] The process of integrating frame-level granularity data into video-granularity data can be called secondary integration. Since secondary integration primarily aggregates information from multiple frames into video information, it is limited by sparse sampling and results in significant loss of inter-frame correlation information. Direct fusion using the frame-level information of the original frame sequence can result in significant deviations. To address this issue, this embodiment performs secondary integration using steps 130-1-2 to 130-1-5:

[0077] Step 130-1-2: Calculate the frame difference between the previous and next frames based on the frame-level granularity data, generate frame difference frames, and group the frame difference frames into a first frame set.

[0078] During implementation, a preset frame difference method can be used to calculate the frame difference between pixels at corresponding positions between two frames. The frame differences of all pixels can then be used to construct a frame difference frame, and all frame difference frames can form a first frame set. The frame difference can be expressed as a difference between two pixels or a gradient value.

[0079] Step 130-1-3: Calculate linear interpolation between the two previous and next frames based on the frame-level granularity data to generate linear interpolation frames, and group the linear interpolation frames into a second frame set.

[0080] During implementation, a preset bilinear interpolation method may be used to calculate linear interpolation between pixels at corresponding positions between the previous and next frames. The linear interpolation of all pixels may construct a linear interpolation frame, and all linear interpolation frames may constitute a second frame set.

[0081] Step 130-1-4: According to the set second integration strategy, the first integration information of the frame-level granularity data, the second integration information corresponding to the first frame set, and the third integration information corresponding to the second frame set are respectively calculated.

[0082] Different task types may also have different second integration strategies. This embodiment does not limit the second integration strategy. For example, the second integration strategy may be used to calculate the maximum, minimum, or average value in a frame set. Using the same second integration strategy to calculate the first integrated information for all frame-level granularity data, the second integrated information corresponding to the first frame set, and the third integrated information corresponding to the second frame set can facilitate the subsequent fusion of these three types of integrated information.

[0083] Step 130-1-5: Perform weighted sum calculation on the first integrated information, the second integrated information, and the third integrated information to obtain video granularity data.

[0084] The weights of the first integrated information, the second integrated information, and the third integrated information may be preset weights. Different integrated information may be given different weights according to actual needs, which is not limited in this embodiment.

[0085] It should be noted that the first-level integration and the second-level integration are not in a strong cascade relationship. The second-level integration does not necessarily use the results of the first-level integration as its input. It can also use the minimum element integration between frames to obtain video information, or directly use the frame-level element integration to obtain video information. This embodiment does not limit this.

[0086] In other embodiments, if the smallest element is a frame-level element, video information can be directly obtained through the first-level integration. Alternatively, the smallest element can also be video-level data, so that data of a specified data granularity can be directly output through the model.

[0087] In other embodiments, if the data granularity is specified as frame-level granularity, data related to the minimum element can be obtained frame by frame based on the minimum element, and then concatenated in sequence according to the frame sequence; or the basic components of each frame can be directly extracted and spliced ​​together. If the data granularity is specified as image granularity, data at that image granularity can be directly output from the model, or the minimum elements can be integrated using a first-level method to obtain data at that image granularity.

[0088] Step 130 - 2 : Encode and encapsulate the data of the specified data granularity to generate the target basic signal.

[0089] In this step, in order to transmit in a unified format, after obtaining the data of the specified data granularity, the data of the specified data granularity may be encoded and encapsulated by a JSON encapsulator to generate a target basic signal.

[0090] In response to diverse visual / speech task types, this embodiment proposes the idea of ​​encoding basic elements and combines a multi-level integration mechanism to encapsulate basic signals, providing standardized basic signals for different data processing granularities and realizing customized strategy processing at multiple data granularities.

[0091] Step 140: Process the target basic signal according to the strategy execution information.

[0092] After the target basic signal is generated for the service data to be processed, corresponding service processing can be performed on the target basic signal according to the policy execution information.

[0093] In one embodiment, if the target model further includes a target decision model, the output result corresponding to the target decision model is the decision information. In this embodiment, when the policy execution information further includes a specified embedded operation, step 140 may further include the following steps:

[0094] The designated embedding operation is used to perform fusion processing on the target basic signal and the decision information.

[0095] In a further embodiment, if the specified embedding operation includes a filtering operation and setting a filtering strategy, the above fusion process may further include the following steps:

[0096] According to the set filtering strategy, the first illegal data in the target basic signal and the second illegal data in the decision information are determined; the first illegal data and the second illegal data are shielded; the processing result is determined by combining the target basic signal after shielding the first illegal data and the decision information after shielding the second illegal data, and the processing result is output.

[0097] In this embodiment, the filtering operation is typically performed after the output of the target decision model, but can also be performed after the target base signal. Based on a specified filtering strategy, such as parameter reception strength and parameter reception type, the first illegal data in the target base signal and the second illegal data in the decision information are determined, and then the first and second illegal data are screened. The final processing result is then determined and output by comprehensively considering the target base signal after the first illegal data is screened, and the decision information after the second illegal data is screened.

[0098] In another further embodiment, if the specified embedding operation includes an integration operation and setting a third integration strategy, the above fusion process may further include the following steps:

[0099] The target basic signal and the decision information are integrated according to the third integration strategy, and the integration result is output.

[0100] In this embodiment, the integration operation fuses the target basic signal and the decision information according to the third integration strategy, thereby obtaining a final integration result for decision making.

[0101] In yet another further embodiment, if the specified embedding operation includes a cascade operation, the above fusion process may further include the following steps:

[0102] The target basic signals are respectively input into each processing layer in the target decision model, processed by the target decision model, and a cascade result is output.

[0103] In this embodiment, the cascade operation will input different target basic signals into each processing layer of the target decision model. In this way, each processing layer will include the output of the previous layer and the target basic signal, and executing layer by layer can obtain a high-precision decision result.

[0104] In one embodiment, after obtaining the cascade result, the following steps may be further included:

[0105] The business data to be processed is marked based on the cascade result; and the business data to be processed carrying the mark is pushed to a downstream application or a target user.

[0106] In this embodiment, the cascade results output by the target decision model can be used as the basis for data labeling, and the business data to be processed can be labeled according to the cascade results. For example, in an audit scenario, high-risk data can be marked according to the cascade results, and high-risk data can be pushed to the human review link for manual review, while non-high-risk data can be sampled and inspected. For another example, in a content understanding scenario, data with higher confidence can be marked according to the cascade results, and the data with higher confidence can be pushed out to assist in completing personalized recommendations for users, and the corresponding content quality status mark will also be stored in the database.

[0107] Furthermore, the aforementioned processing results, combined with those from manual processing, can serve as a basis for rapid model optimization and upgrades. Specifically, based on the data status recorded by the status tags, some data can be automatically used by the system to adjust the model set, while other data can be submitted to the annotation team as a periodic task. This modular design not only enables machine processing to meet business needs, but also connects multiple manual teams, improving efficiency.

[0108] In one embodiment, the policy execution information further includes a post-processing operation and a preset post-processing policy. This embodiment may further include the following steps:

[0109] At least one of the target basic signal, the decision information and the result of the fusion processing that conforms to the post-processing strategy is processed using the post-processing operation.

[0110] In this embodiment, a post-processing operation is performed by setting a post-processing strategy, and at least one of the target basic signal, decision information, and fusion processing result that conforms to the post-processing strategy can be processed using the post-processing operation.

[0111] In this embodiment, a policy execution flow is generated based on the policy customization information input by the user. The policy execution flow includes the identifier of the target model and the policy execution information to meet the complex requirements related to various human understanding factors. Then, when the business data to be processed is processed, the business data to be processed is input into the target model, and the output result signal of the obtained target model is converted into a target basic signal. This allows different complex requirements to be expressed through a set of basic signals, reducing the amount of information transmitted by the signal, so its multiplexing capability is relatively strong. The target basic signal is then processed according to the policy execution information to obtain a processing result that can adapt to a large number of business requirements related to human factors, can cope with complex and changing processing rules, and improve the processing capability of the system.

[0112] Example 2

[0113] Figure 2 This is a structural block diagram of a business processing system embodiment provided in Example 2 of the present application. The business processing system may include a basic signal module 210, a decision module 220 and a policy construction layer 230, wherein the basic signal module 210 includes a basic model set, and the decision module 220 includes a decision model set.

[0114] The policy construction layer 230 is used to provide a policy customization interface, receive policy customization information input by the user from the policy customization interface, and generate a policy execution flow according to the policy customization information, wherein the policy execution flow includes the identification of the target model and the policy execution information;

[0115] The basic signal module 210 is used to determine the corresponding target basic model from the basic model set according to the identifier of the target model, and use the target basic model to process the service data to be processed, obtain the output result of the target basic model and generate a target basic signal;

[0116] The decision module 220 is used to determine the corresponding target decision model from the decision model set according to the identifier of the target model, and use the target decision model to process the business data to be processed to obtain decision information output by the target decision model;

[0117] The strategy construction layer 230 is further configured to perform fusion processing on the target basic signal and the decision information according to the strategy execution information.

[0118] In one embodiment, when generating a policy execution flow, the policy construction layer 230 is specifically used to: obtain policy customization information input by the user, wherein the policy customization information includes policy requirement information and policy execution information; select the identifier of the target model that matches the policy requirement information from a pre-generated model set; and generate a policy execution flow associated with the policy customization information based on the identifier of the target model and the policy execution information.

[0119] In one embodiment, the output result of the target basic model is multimodal information; the policy execution information includes a specified data granularity; when generating the target basic signal, the basic signal module 210 is specifically used to: generate data of a specified data granularity from the multimodal information; encode and encapsulate the data of the specified data granularity to generate the target basic signal.

[0120] In one embodiment, the specified data granularity includes video granularity; the multimodal information is represented by a set minimum element; and when the basic signal module 210 generates data of the specified data granularity from the multimodal information, it is further configured to:

[0121] The multimodal information is integrated into frame-level granularity data according to a set first integration strategy; based on the frame-level granularity data, the frame difference between the previous and next frames is calculated respectively to generate frame difference frames, and the frame difference frames are grouped into a first frame set; based on the frame-level granularity data, the linear interpolation between the previous and next frames is calculated respectively to generate linear interpolation frames, and the linear interpolation frames are grouped into a second frame set; according to the set second integration strategy, the first integration information of the frame-level granularity data, the second integration information corresponding to the first frame set, and the third integration information corresponding to the second frame set are calculated respectively; the first integration information, the second integration information and the third integration information are weightedly summed to obtain video granularity data.

[0122] In one embodiment, the policy execution information further includes a specified embedded operation, which includes a filtering operation and a set filtering policy. When performing the fusion process, the policy construction layer 230 is specifically used to:

[0123] According to the set filtering strategy, the first illegal data in the target basic signal and the second illegal data in the decision information are determined; the first illegal data and the second illegal data are shielded; the processing result is determined by combining the target basic signal after shielding the first illegal data and the decision information after shielding the second illegal data, and the processing result is output.

[0124] In another embodiment, the designated embedding operation includes an integration operation and setting a third integration strategy. Then, when performing the fusion process, the strategy construction layer 230 is specifically configured to:

[0125] The target basic signal and the decision information are integrated according to the third integration strategy, and the integration result is output.

[0126] In another embodiment, the designated embedding operation includes a cascade operation, and the target decision model is a model including multiple processing layers. Then, when performing the fusion processing, the strategy construction layer 230 is specifically used to:

[0127] The target basic signals are respectively input into each processing layer in the target decision model, processed by the target decision model, and a cascade result is output.

[0128] In one embodiment, the decision module 220 is further configured to: mark the business data to be processed based on the cascade result; and push the marked business data to be processed to a downstream application or a target user.

[0129] In one embodiment, the strategy execution information also includes post-processing operations and preset post-processing strategies; the basic signal module 210 is also used to process the target basic signal that complies with the post-processing strategy using the post-processing operation; the decision module 220 is also used to process the decision information that complies with the post-processing strategy using the post-processing operation; the strategy construction layer 230 is also used to process the result of the fusion processing that complies with the post-processing strategy using the post-processing operation.

[0130] The following is a comprehensive description of the above embodiments with reference to the accompanying figures. Figure 3 The diagram shows the execution flow of the business processing system of this embodiment during application and the interaction with users or other practical modules at different stages, which mainly includes a basic signal module, a decision module and a policy construction layer.

[0131] The real-time video or picture from the online is first input into the basic signal module, which includes the basic model set (i.e. Figure 3 The basic signal model set in is used to generate high-rich multimodal information. Figure 3 As shown, the basic model set may include, for example, a behavior recognition model, a scene classification model, an age regression model, a gender model, and a speech recognition model. The corresponding multimodal information may include, for example, behavior signals, scene signals, age and gender signals, speech feature signals, and special effects processing signals. This multimodal information is packaged in a specific manner as a target basic signal and input into the strategy construction layer in parallel.

[0132] The decision module includes a set of decision models, which can further include a review model set and a content understanding model set. Examples of review models include a cascaded gun recognition model, a bloody model, a knife detection model, and a pornographic / light pornographic multi-task model. Examples of content understanding models include a primary labeling model, a secondary labeling model, and a user profiling model. The input to the decision model set is also real-time online video or images, and its output is transmitted to the policy construction layer.

[0133] The strategy construction layer then coordinates the input target base signals and the outputs of the decision models in the decision module, resulting in processing results that can adapt to a wide range of business needs related to human factors. These processing results, combined with the manual processing results, can serve as the basis for rapid optimization and upgrade of models in the basic model set and decision model set.

[0134] Furthermore, to achieve flexibility comparable to manual processing, this embodiment interacts with different users at different stages of system execution. Users can manually set the policy construction layer's reception and strength parameters for the underlying signal, as well as the data granularity and post-processing methods applied to the underlying signal, to align with the artificial conditions or rules defined by business requirements. Different combinations of parameters or methods can be used to customize diverse processing strategies.

[0135] like Figure 3 As shown in the figure, for the processing results of the audit category, in addition to being automatically banned, some high-risk data will also enter the manual review process for several rounds of review. The remaining large amount of non-high-risk data will also be randomly sampled and inspected. The comprehensive results of the three (i.e., the basic signal module, the decision module, and the policy construction layer) are stored in the database in the form of status tags. For the processing results of the content understanding category, the tag information and relay information are directly transmitted to the practical module on the recommendation side to assist in completing personalized recommendations for users, and the corresponding content quality status tags are also stored in the database. Finally, based on the data status recorded by the status tag, part of the data is automatically used by the system to adjust the model set, and the other part of the data is submitted to the annotation side as a periodic task of the annotation team.

[0136] This embodiment is achieved by Figure 3 The modular design not only completes the machine processing of business needs, but also connects multiple manual teams to improve efficiency.

[0137] The core component of this embodiment is the policy construction layer used to meet diverse human requirements. By configuring policy parameters and policy methods, the corresponding processing policy is customized, thereby generating a policy execution flow for executing the processing policy. Figure 4The policy execution information of the policy construction layer is shown. For example, the policy execution information may include data granularity, embedding operation, and optional post-processing operation. The embedding operation corresponds to the usage logic of the underlying signal, the data granularity corresponds to the data granularity application logic of the policy, and the post-processing operation corresponds to the post-processing logic.

[0138] Among them, the embedding operation means that the signal content will be combined with the discriminant elements of the decision layer in a specific signal usage method to jointly give the final discrimination result. Exemplarily, the embedding operation can include three types of filtering operations, integration operations and cascade operations. Specifically, the filtering operation is usually used after the output of the decision model to block illegal outputs based on filtering strategies such as basic signal strength or artificial conditional logic; or, the filtering operation can also be used to filter the basic signal and the output of the decision model. The integration operation will fuse the basic signal content and the decision model output content to obtain the final output for judgment. The cascade operation will input different basic signals into different processing layers of the decision model, and execute layer by layer to obtain high-precision judgment results.

[0139] In practice, the above three embedding operations can also generate actual policy execution flows based on the task type and organization type of the signal. Different types of signals correspond to the three operations in different application methods.

[0140] In order to expand the support scope of the system, this embodiment designs the logic of policy application at different data granularities. Figure 4 The policy construction layer supports processing strategies at four data granularities, from high to low: video granularity, frame sequence granularity (also known as frame-level granularity), image granularity, and feature granularity. It's important to note that video granularity and frame sequence granularity treat the entire video as a single processing element, rather than considering each frame individually. Building policies at different granularities utilizes fundamental signals at different encapsulation levels.

[0141] Post-processing logic is optional and is primarily used for normalization, generating statistical metrics and specific outputs for specific types of signals and decision outputs. Of course, some data that needs to be cached or transferred will also be concentrated in the post-processing stage.

[0142] In other embodiments, in addition to the aforementioned policy execution information, building a specific processing strategy also requires receiving a rich variety of basic signals from the basic signal module and receiving the output information of the corresponding model from the decision module. The received signal content will be selected by setting a filter based on the set signal parameters, and the data granularity of the corresponding strategy will be matched through the selected signal encapsulation type. Whether it is the basic signal or the output information of the decision model, it can transmit multi-level content such as scores, features, and vectors. These contents enrich the number of policy instances that can be created and enhance the flexibility of the system in processing business needs. Compared with the "flat" system, this embodiment can match human conditions more flexibly.

[0143] In one embodiment, the multimodal information output by the basic model set will be transmitted to the policy construction layer in the form of basic signals for policy processing only after a specific type of integration and encapsulation. This embodiment can support the provision of high-rich multimodal information, including a variety of deep learning visual task signals, deep learning speech sequence signals, etc. Among them, the basic components (i.e., the minimum elements) of the basic signals of different tasks may be different, and they can be integrated into basic signals for different data granularities through different signal integration methods. Specifically, the basic components of the signal may include coordinates, categories, association relationships, etc., such as: Figure 5 As shown in the figure, the basic components of the key point identification task related signals can be used [key point number, key point location name, key point coordinates<x,y> ]Definition, the basic components of the signal related to the box detection task can be used [category number, coordinates of the upper left corner of the box<x,y> , frame size coordinates<w,h> ] definition, the basic components of the signal related to single / multi-classification tasks can be represented by [confidence] vectors, and the basic components of the signal related to semantic segmentation tasks can be represented by [category number, object pixel coordinates<x,y> The basic components of the signals related to speech recognition tasks can be defined using discrete sound wave vectors. These simple basic components can provide rich basic signals for the strategy construction layer using specific integration or simple splicing methods.

[0144] In implementation, since the basic elements themselves also have the attribute of data granularity, the method is more flexible when constructing basic signals of different data granularities. Figure 5As shown, the basic signal for constructing video granularity can be obtained through two-level integration starting from the smallest element (elements contained within the image, such as frame information, segmented pixel information, etc.), or through one-level integration starting from the frame-level element. The model can also directly output the video granularity information based on the video as a whole. The basic signal for constructing frame sequence granularity (i.e., frame-level granularity) can be obtained by obtaining the corresponding information frame by frame from the smallest element and concatenating them in sequence according to the frame sequence, or by directly extracting the basic components of each frame and splicing them together. The basic signal for constructing image granularity can be obtained directly or through one-level integration with the smallest element. After completing the integration of the basic signals, in order to transmit them in a unified format, these basic signals must be encoded and encapsulated by the JSON wrapper before being transmitted to the policy construction layer. The policy construction layer can select the data granularity combination of the basic signals as needed to assist in customizing specific policies, which further increases the flexibility of the policy.

[0145] When integrating the basic signal, Figure 5 It can be seen that there are two main levels of integration: integration from the smallest element to frame-level information (first-level integration), and integration from frame-level information to video-level information (second-level integration). Taking into account that the smallest element is often driven by the type of task and is relatively diverse, the integration of the smallest element can use simple integration methods to calculate the integrated information according to the task type, such as: smoothing the pixels output by the segmentation task and then calculating the total metric of the heat map, using the non-maximum suppression method for the box output by the detection task to obtain the core box or obtain the representative category of the image based on the box ratio, etc. Since the second-level integration is mainly responsible for summarizing the information of multiple frames into video information, it is limited by the sparse sampling method, and the correlation information between frames is lost to a large extent, which makes the direct fusion of the frame-level information of the original frame sequence produce a large deviation. Based on this, if Figure 6 As shown, during the second-level integration process, this embodiment first calculates the frame difference between the preceding and following frames before calculating the simplified integrated information. This generates a frame difference frame, and then constructs a frame difference frame set based on all the frame difference frames. Simultaneously, bilinear interpolation is performed between the preceding and following frames to form a linear interpolation frame. A linear interpolation frame set is then constructed based on all the linear interpolation frames. Next, integrated information is calculated for each of these two frame sets. Finally, a weighted summation is performed with the integrated information of the original frame sequence to obtain video-level information.

[0146] In other implementations, when integrating frame-level information into video information, a VLAD (Vector of Locally Aggregated Descriptors) mechanism based on cluster centers and internal mapping can also be used to avoid the effect being affected by generalization ability.

[0147] It should be noted that the first-level integration and the second-level integration are not in a strong cascade relationship. The second-level integration does not necessarily use the output results of the first-level integration as its own input. It can also use the minimum element integration between frames to obtain video information, or directly use the frame-level element integration to obtain video information.

[0148] It should be noted that the information for the decision-making module can also be designed as a basic signal mode. Through a unified basic signal encapsulation and basic signal screening mechanism, the amount of transmitted information and the flexibility of strategy construction can be reduced.

[0149] In response to diverse visual / speech task types, this embodiment proposes the idea of ​​encoding basic elements and combines a multi-level integration mechanism to integrate basic signals, providing standardized basic signals for different data processing granularities and realizing strategy customization for multiple data granularities.

[0150] In addition to the functions of data granularity and signal processing methods, the strategy construction layer can also add reasoning functions, that is, signaling artificial conditions, reasoning and improving some rule conditions that cannot be taken into account manually, and improving the accuracy of the final output.

[0151] This embodiment, through a foundational signal system that provides rich information and a logically modularized policy construction layer, effectively addresses the limitations of flat machine processing systems, which are limited to simple, human-defined rules. It also offers customized policy solutions that flexibly address the complex and ever-changing human factors in business requirements. Because policies are constructed for diverse and complex requirements using a single set of foundational signals, they offer strong reuse capabilities. While initial development costs are high, subsequent maintenance and expansion are relatively simple, making it easy to meet complex and ever-changing business requirements that rely on human understanding.

[0152] In terms of the integration of video granularity information, this embodiment proposes a method of using inter-frame information and interpolation information to enhance the correlation strength when integrating previous and next frames. It also integrates simple integration methods of different task types to meet different needs.

[0153] Example 3

[0154] Figure 7 The structural block diagram of a service processing device embodiment provided in the third embodiment of the present application may include the following units:

[0155] The policy execution flow generating unit 710 is configured to generate a policy execution flow according to the policy customization information input by the user, wherein the policy execution flow includes an identifier of the target model and policy execution information;

[0156] The target model processing unit 720 is used to input the business data to be processed into the target model and obtain the output result of the target model;

[0157] A basic signal generating unit 730 is configured to generate a target basic signal based on the output result;

[0158] The signal processing unit 740 is configured to process the target basic signal according to the policy execution information.

[0159] In one embodiment, the policy execution flow generation unit 710 is specifically configured to:

[0160] Acquire policy customization information input by a user, wherein the policy customization information includes policy requirement information and policy execution information;

[0161] Selecting an identifier of a target model that matches the policy requirement information from a pre-generated model set;

[0162] A policy execution flow associated with the policy customization information is generated according to the identifier of the target model and the policy execution information.

[0163] In one embodiment, the target model includes a target base model; the output result corresponding to the target base model is multimodal information; the policy execution information includes a specified data granularity;

[0164] The basic signal generating unit 730 may further include the following subunits:

[0165] a designated data granularity data generation subunit, configured to generate data of the designated data granularity from the multimodal information;

[0166] The encoding and encapsulation subunit is used to encode and encapsulate the data of the specified data granularity to generate the target basic signal.

[0167] In one embodiment, the specified data granularity includes video granularity; the multimodal information is represented by a set minimum element;

[0168] The designated data granularity data generation subunit is specifically used for:

[0169] Integrating the multimodal information into frame-level granularity data according to a set first integration strategy;

[0170] Calculating the frame differences between the preceding and following frames according to the frame-level granularity data, generating frame difference frames, and grouping the frame difference frames into a first frame set;

[0171] Calculating linear interpolation between two frames respectively according to the frame-level granularity data to generate linear interpolation frames, and grouping the linear interpolation frames into a second frame set;

[0172] According to the set second integration strategy, respectively calculating the first integration information of the frame-level granularity data, the second integration information corresponding to the first frame set, and the third integration information corresponding to the second frame set;

[0173] Perform weighted sum calculation on the first integrated information, the second integrated information, and the third integrated information to obtain video granularity data.

[0174] In one embodiment, the target model further includes a target decision model; the output result corresponding to the target decision model is decision information; the policy execution information further includes a specified embedded operation;

[0175] The signal processing unit 740 may further include the following subunits:

[0176] The fusion processing subunit is used to adopt the specified embedding operation to perform fusion processing on the target basic signal and the decision information.

[0177] In one embodiment, the specified embedding operation includes a filtering operation and setting a filtering strategy, and the fusion processing subunit is specifically configured to:

[0178] determining, according to the set filtering strategy, first illegal data in the target basic signal and second illegal data in the decision information;

[0179] shielding the first illegal data and the second illegal data;

[0180] The target basic signal after shielding the first illegal data and the decision information after shielding the second illegal data are combined to determine a processing result, and the processing result is output.

[0181] In another embodiment, the specified embedding operation includes an integration operation and setting a third integration strategy, and the fusion processing subunit is specifically configured to:

[0182] The target basic signal and the decision information are integrated according to the third integration strategy, and the integration result is output.

[0183] In yet another embodiment, the designated embedding operation includes a cascade operation, the target decision model is a model including multiple processing layers, and the fusion processing subunit is specifically configured to:

[0184] The target basic signals are respectively input into each processing layer in the target decision model, processed by the target decision model, and a cascade result is output.

[0185] In one embodiment, the apparatus may further include the following units:

[0186] A data marking unit, configured to mark the service data to be processed based on the concatenation result;

[0187] The data push unit is used to push the marked business data to be processed to the downstream application or target user.

[0188] In one embodiment, the policy execution information further includes post-processing operations and a preset post-processing policy;

[0189] The device further comprises the following units:

[0190] A post-processing unit is used to process at least one of the target basic signal, the decision information and the result of the fusion processing that conforms to the post-processing strategy using the post-processing operation.

[0191] A business processing device provided in an embodiment of the present application can execute a business processing method described in embodiment 1 of the present application, or be applied to a business processing system in embodiment 2, and has functional modules and beneficial effects corresponding to the execution method.

[0192] Example 4

[0193] Figure 8 This is a structural diagram of a business processing device provided in Example 4 of the present application, such as Figure 8 As shown, the business processing device includes a processor 810, a memory 820, an input device 830 and an output device 840; the number of processors 810 in the business processing device can be one or more. Figure 8 In the example, a processor 810 is used; the processor 810, memory 820, input device 830 and output device 840 in the business processing device can be connected via a bus or other means. Figure 8 The bus connection is taken as an example.

[0194] The memory 820, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the first or second embodiment of the present application. The processor 810 executes the software programs, instructions, and modules stored in the memory 820 to execute various functional applications and data processing of the service processing device, thereby implementing the methods or modules mentioned in the first or second method embodiments.

[0195] The memory 820 may primarily include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal, etc. Furthermore, the memory 820 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 820 may further include memory remotely located relative to the processor 810, and these remote memories may be connected to the device / terminal / server via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0196] The input device 830 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the business processing device. The output device 840 may include a display device such as a display screen.

[0197] Example 5

[0198] Embodiment 5 of the present application further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the method of the above-mentioned method embodiment 1 or embodiment 2.

[0199] Of course, the storage medium containing computer-executable instructions provided in an embodiment of the present application, whose computer-executable instructions are not limited to the method operations described above, can also execute related operations in the method provided in any embodiment of the present application.

[0200] Example 6

[0201] Embodiment 6 of the present application further provides a computer program product, which includes computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to execute the method or module of the above-mentioned method embodiment 1 or embodiment 2.

[0202] Of course, the computer program product provided in the embodiment of the present application, whose computer executable instructions are not limited to the method operations described above, can also execute related operations in the method provided in any embodiment of the present application.

[0203] Through the above description of the implementation methods, technical personnel in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a business processing device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0204] It is worth noting that in the embodiments of the above-mentioned device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0205] Note that the above are only preferred embodiments of the present application and the technical principles employed. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the appended claims.

Claims

1. A business processing method, characterized in that: The method comprises: Obtaining policy customization information input by a user, the policy customization information including policy requirement information and policy execution information, wherein the policy requirement information is used to indicate the scenario information to which the customized processing policy is applicable; and the policy execution information is used to define one or more constraint elements during the execution of the processing policy; Selecting an identifier of a target model that matches the policy requirement information from a pre-generated model set; generating a policy execution flow associated with the policy customization information according to the identifier of the target model and the policy execution information; Inputting the business data to be processed into the target model and obtaining the output result of the target model; generating a target basic signal based on the output result; processing the target basic signal according to the strategy execution information; The target model includes a target base model; the output result corresponding to the target base model is multimodal information; the policy execution information includes a specified data granularity; the specified data granularity includes a video granularity; the multimodal information is represented by a set minimum element; Generating a target basic signal based on the output result includes: Integrating the multimodal information into frame-level granularity data according to a set first integration strategy; Calculating the frame differences between the preceding and following frames according to the frame-level granularity data, generating frame difference frames, and grouping the frame difference frames into a first frame set; Calculating linear interpolation between two frames respectively according to the frame-level granularity data to generate linear interpolation frames, and grouping the linear interpolation frames into a second frame set; According to the set second integration strategy, respectively calculating the first integration information of the frame-level granularity data, the second integration information corresponding to the first frame set, and the third integration information corresponding to the second frame set; Performing weighted sum calculation on the first integrated information, the second integrated information, and the third integrated information to obtain video granularity data as data of a specified data granularity; The data of the specified data granularity is encoded and encapsulated to generate the target basic signal.

2. The method according to claim 1, characterized in that The target model also includes a target decision model; the output result corresponding to the target decision model is decision information; The policy execution information also includes a specified embedding operation; The processing of the target basic signal according to the strategy execution information includes: The designated embedding operation is used to perform fusion processing on the target basic signal and the decision information.

3. The method according to claim 2, characterized in that The designated embedding operation includes a filtering operation and setting a filtering strategy. The designated embedding operation is used to perform fusion processing on the target basic signal and the decision information, including: determining, according to the set filtering strategy, first illegal data in the target basic signal and second illegal data in the decision information; shielding the first illegal data and the second illegal data; The target basic signal after shielding the first illegal data and the decision information after shielding the second illegal data are combined to determine a processing result, and the processing result is output.

4. The method according to claim 2, characterized in that The designated embedding operation includes an integration operation and setting a third integration strategy, and the adopting of the designated embedding operation to perform fusion processing on the target basic signal and the decision information includes: The target basic signal and the decision information are integrated according to the third integration strategy, and the integration result is output.

5. The method according to claim 2, characterized in that The designated embedding operation includes a cascade operation, the target decision model is a model including multiple processing layers, and the designated embedding operation is used to perform fusion processing on the target basic signal and the decision information, including: The target basic signals are respectively input into each processing layer in the target decision model, processed by the target decision model, and a cascade result is output.

6. The method according to claim 5, characterized in that The method further comprises: Marking the service data to be processed based on the cascade result; Pushes marked business data to be processed to downstream applications or target users.

7. The method according to claim 2, characterized in that The policy execution information also includes post-processing operations and preset post-processing policies; After the designated embedding operation is used to perform fusion processing on the target basic signal and the decision information, the method further includes: At least one of the target basic signal, the decision information and the result of the fusion processing that conforms to the post-processing strategy is processed using the post-processing operation.

8. A business processing system, characterized in that: The business processing system includes a basic signal module, a decision module and a strategy construction layer, wherein the basic signal module includes a basic model set, and the decision module includes a decision model set; The policy construction layer is used to provide a policy customization interface, receive policy customization information input by a user from the policy customization interface, and generate a policy execution flow according to the policy customization information, wherein the policy execution flow includes an identifier of a target model and policy execution information; The basic signal module is used to determine the corresponding target basic model from the basic model set according to the identifier of the target model, and use the target basic model to process the service data to be processed, obtain the output result of the target basic model and generate a target basic signal; The decision module is used to determine the corresponding target decision model from the decision model set according to the identifier of the target model, and use the target decision model to process the business data to be processed to obtain decision information output by the target decision model; The strategy construction layer is further used to perform fusion processing on the target basic signal and the decision information according to the strategy execution information; When generating the policy execution flow, the policy construction layer is specifically used to: obtain policy customization information input by the user, the policy customization information including policy requirement information and policy execution information; the policy requirement information is used to indicate the scenario information applicable to the customized processing policy; the policy execution information is used to limit one or more constraint elements during the execution of the processing policy; select the identifier of the target model that matches the policy requirement information from a pre-generated model set; and generate a policy execution flow associated with the policy customization information based on the identifier of the target model and the policy execution information; The output result corresponding to the target basic model is multimodal information; the policy execution information includes a specified data granularity; the specified data granularity includes a video granularity; the multimodal information is represented by a set minimum element; When generating a target basic signal, the basic signal module is specifically used to: integrate the multimodal information into frame-level granularity data according to a set first integration strategy; calculate the frame difference between the previous and next frames according to the frame-level granularity data, generate frame difference frames, and form each frame difference frame into a first frame set; calculate the linear interpolation between the previous and next frames according to the frame-level granularity data, generate linear interpolation frames, and form each linear interpolation frame into a second frame set; calculate the first integration information of the frame-level granularity data, the second integration information corresponding to the first frame set, and the third integration information corresponding to the second frame set according to the set second integration strategy; perform weighted sum calculation on the first integration information, the second integration information, and the third integration information to obtain video granularity data as data of a specified data granularity; encode and encapsulate the data of the specified data granularity to generate the target basic signal.

9. A service processing device, characterized in that: The device comprises: A policy execution flow generating unit, configured to generate a policy execution flow according to the policy customization information input by the user, wherein the policy execution flow includes an identifier of the target model and policy execution information; A target model processing unit, configured to input the business data to be processed into the target model and obtain an output result of the target model; a basic signal generating unit, configured to generate a target basic signal based on the output result; a signal processing unit, configured to process the target basic signal according to the strategy execution information; The policy execution flow generation unit is specifically used to obtain policy customization information input by a user, wherein the policy customization information includes policy requirement information and policy execution information; the policy requirement information is used to indicate the scenario information to which the customized processing policy is applicable; the policy execution information is used to define one or more constraint elements during the execution of the processing policy; select the identifier of the target model that matches the policy requirement information from a pre-generated model set; and generate a policy execution flow associated with the policy customization information based on the identifier of the target model and the policy execution information; The target model includes a target base model; the output result corresponding to the target base model is multimodal information; the policy execution information includes a specified data granularity; the specified data granularity includes a video granularity; the multimodal information is represented by a set minimum element; The basic signal generating unit includes: a designated data granularity data generation subunit, configured to integrate the multimodal information into frame-level granularity data according to a set first integration strategy; calculate the frame difference between two preceding and following frames based on the frame-level granularity data, generate frame difference frames, and form each frame difference frame into a first frame set; calculate the linear interpolation between two preceding and following frames based on the frame-level granularity data, generate linear interpolation frames, and form each linear interpolation frame into a second frame set; calculate the first integrated information of the frame-level granularity data, the second integrated information corresponding to the first frame set, and the third integrated information corresponding to the second frame set according to a set second integration strategy; perform a weighted sum calculation on the first integrated information, the second integrated information, and the third integrated information to obtain video granularity data as data of the designated data granularity; The encoding and encapsulation subunit is used to encode and encapsulate the data of the specified data granularity to generate the target basic signal.

10. A business processing device, characterized in that: The service processing device includes: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising computer executable instructions, wherein the computer executable instructions are used to implement the method according to any one of claims 1 to 7 when executed.

Images