A data processing method, a target container determination method, an apparatus, a device, a medium, and a product

Abstract

The application discloses a data processing method, a target container determination method, an apparatus, an equipment, a medium and a product. The method comprises the following steps: acquiring multi-modal data and a target data set, wherein the multi-modal data comprises acquisition data of at least two modes, and the target data set comprises a plurality of groups of target data and an action corresponding to each group of target data, each group of target data comprises target data of each mode; determining a correlation coefficient of the acquisition data of each mode and the target data of each mode in each group of target data; determining a fusion correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the acquisition data of each mode and the target data of each mode in each group of target data and a weight of each mode; determining a target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data; and executing the target action corresponding to the multi-modal data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer technology, and in particular to a data processing method, a target container determination method, an apparatus, a device, a medium and a product. BACKGROUND

[0002] Traditional intelligent running frameworks are usually designed based on high-performance computing platforms (such as desktop computers or servers) and rely on centralized computing resources. Such frameworks have significant defects in scenarios with high real-time requirements such as power systems:

[0003] Insufficient hardware compatibility: due to the limitations of computing power, memory and power consumption, embedded systems are difficult to directly deploy traditional frameworks, resulting in weak data processing capabilities on the edge side.

[0004] Low data fusion efficiency: power systems need to integrate multi-source heterogeneous data, and traditional methods use serial processing or simple weighted fusion, which has high computational complexity and significant delay.

[0005] Real-time bottleneck: existing frameworks often use batch processing mode for feature extraction and decision scheduling of fused data, which cannot meet the demand for microsecond-level response.

[0006] Therefore, there is an urgent need for a data processing method for multi-modal data that supports embedded system execution and has low complexity. SUMMARY

[0007] Embodiments of the present application provide a data processing method, a target container determination method, an apparatus, a device, a medium and a product, which can realize data processing for fused data through an embedded system and improve data processing efficiency.

[0008] According to an aspect of the present application, a data processing method is provided, which is executed by an embedded system, and the method comprises:

[0009] obtaining multi-modal data and a target data set, wherein the multi-modal data comprises at least two modalities of collected data, and the target data set comprises a plurality of groups of target data and an action corresponding to each group of target data, each group of target data comprising target data of each modality;

[0010] determining a correlation coefficient of the collected data of each modality and the target data of each modality in each group of target data;

[0011] determining a fused correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the collected data of each modality and the target data of each modality in each group of target data and the weight of each modality;

[0012] determining a target action corresponding to the multi-modal data according to the fused correlation coefficient of the multi-modal data and each group of target data.

[0013] performing a target action corresponding to the multi-modal data.

[0014] Further, the target action corresponding to the multi-modal data is determined according to the fusion correlation coefficients of the multi-modal data and each group of target data, including:

[0015] If the fusion correlation coefficient of the multi-modal data and at least one group of target data is greater than the correlation coefficient threshold, the target action corresponding to the multi-modal data is determined according to at least one group of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold.

[0016] If the fusion correlation coefficients of the multi-modal data and each group of target data are less than or equal to the correlation coefficient threshold, the multi-modal data is input into a target model to obtain the target action corresponding to the multi-modal data, wherein the target model sequentially includes a state perceiver, a learner and an action selector from the input to the output.

[0017] Further, the target action corresponding to the multi-modal data is determined according to at least one group of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold, including:

[0018] If there is a group of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold, the action corresponding to the group of target data is taken as the target action corresponding to the multi-modal data.

[0019] If there are multiple groups of target data whose fusion correlation coefficients with the multi-modal data are all greater than the correlation coefficient threshold, the action corresponding to the group of target data with the largest fusion correlation coefficient with the multi-modal data in the target data set is taken as the target action corresponding to the multi-modal data.

[0020] Further, the correlation coefficient of the collected data of each modality and the target data of each modality in each group of target data is determined, including:

[0021] The values of each dimension of the collected data of each modality and the dimension vector are determined.

[0022] The values of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data are determined.

[0023] The correlation coefficient of the collected data of each modality and the target data of each modality in each group of target data is determined according to the values of each dimension of the collected data of each modality, the dimension vector of the collected data of each modality, the values of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data.

[0024] Further, the values of each dimension of the collected data of each modality and the dimension vector are determined, including:

[0025] determining the covariance matrix corresponding to the acquisition data of each modality;

[0026] determining the value of each dimension and the dimension vector of the acquisition data of each modality according to the covariance matrix corresponding to the acquisition data of each modality.

[0027] Further, determining the covariance matrix corresponding to the acquisition data of each modality comprises:

[0028] processing the acquisition data of each modality to obtain the consistency matrix of the acquisition data of each modality;

[0029] determining the covariance matrix corresponding to the acquisition data of each modality based on the consistency matrix of the acquisition data of each modality.

[0030] Further, processing the acquisition data of each modality to obtain the consistency matrix of the acquisition data of each modality comprises:

[0031] obtaining the mean and variance of the acquisition data of each modality;

[0032] determining the consistency matrix of the acquisition data of each modality according to the acquisition data of each modality, the mean of the acquisition data of each modality and the variance of the acquisition data of each modality.

[0033] Further, determining the consistency matrix of the acquisition data of each modality according to the acquisition data of each modality, the mean of the acquisition data of each modality and the variance of the acquisition data of each modality comprises:

[0034] taking the difference between the acquisition data of each modality and the mean as the difference of each modality;

[0035] taking the ratio of the difference of each modality and the variance as the consistency matrix of the acquisition data of each modality.

[0036] Further, determining the covariance matrix corresponding to the acquisition data of each modality based on the consistency matrix of the acquisition data of each modality comprises:

[0037] obtaining the number of acquisition data types of each modality;

[0038] determining the covariance matrix corresponding to the acquisition data of each modality according to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality.

[0039] Further, determining the covariance matrix corresponding to the acquisition data of each modality according to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality comprises:

[0040] Transposing the consistency matrix of the acquisition data of each modality and multiplying the consistency matrix of the acquisition data of each modality as a first matrix;

[0041] The ratio of the first matrix and the number of acquisition data types of each modality is taken as the covariance matrix corresponding to the acquisition data of each modality.

[0042] Further, according to the value of each dimension of the acquisition data of each modality, the dimension vector of the acquisition data of each modality, the value of each dimension of the target data in each group of target data, and the dimension vector of the target data in each group of target data, the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data is determined, comprising:

[0043] According to the value of each dimension of the acquisition data of each modality and the dimension vector of the acquisition data of each modality, the vector index of the acquisition data of each modality is determined;

[0044] According to the value of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data, the vector index of the target data of each modality in each group of target data is determined;

[0045] According to the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data, the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data is determined.

[0046] Further, according to the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data, the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data is determined, comprising:

[0047] The ratio of the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data is taken as the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data.

[0048] Further, according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality, the fusion correlation coefficient of the multi-modal data and each group of target data is determined, comprising:

[0049] Obtaining the matrix proportion of each modality;

[0050] According to the matrix proportion of each modality, the weight of each modality is determined;

[0051] Based on the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data, and the weight of each modality, weighted summation is performed to obtain the fusion correlation coefficient of the multi-modal data and each group of target data.

[0052] Further, input the multi-modal data into a target model to obtain a target action corresponding to the multi-modal data, including:

[0053] Encode the multi-modal data into a state feature vector by the state perceiver;

[0054] Update parameters of the policy model based on the state feature vector and a reward function by the learner;

[0055] Select the target action corresponding to the multi-modal data from an action space according to the updated policy model by the action selector.

[0056] Further, the reward function is:

[0057] ;

[0058] wherein, is a reward function, A is an action space, is a reward vector, is a reward value obtained when the action is taken, is the kth action in the action space, s is a current state, is a next state after the action is executed, is a state change amount.

[0059] Further, the multi-modal data includes image data, text data and audio data.

[0060] Further, the method further includes:

[0061] Obtaining a set of instruction sets, wherein the set of instruction sets includes a plurality of versions of instruction sets corresponding to a plurality of hardware architectures;

[0062] Filtering the set of instruction sets based on an execution time of each version of the instruction set corresponding to the current hardware architecture to obtain a target instruction set;

[0063] Burning the target instruction set to the current hardware architecture.

[0064] Further, filtering the set of instruction sets based on the execution time of each version of the instruction set corresponding to the current hardware architecture to obtain the target instruction set includes:

[0065] Determining a mean value of the execution time, wherein the execution time is the execution time of each version of the instruction set corresponding to the current hardware architecture;

[0066] Selecting, as the target instruction set, an instruction set of a target version whose ratio to the mean value of the execution time is less than or equal to a set threshold.

[0067] Further, the overhead time for the current hardware architecture to execute the corresponding version of the instruction set is the time for the current hardware architecture to execute the corresponding version of the instruction set.

[0068] According to another aspect of the present application, a target container determination method is provided, which comprises:

[0069] Obtaining performance parameters of each container in a container cluster, wherein the performance parameters comprise: CPU utilization, resident memory occupancy, and transmission bit rate occupancy;

[0070] Taking a container whose at least one of the CPU utilization, the resident memory occupancy, and the transmission bit rate occupancy is greater than the corresponding threshold value as a target container;

[0071] Performing the data processing method according to any one of the embodiments of the present application based on the target container.

[0072] Further, taking a container whose at least one of the CPU utilization, the resident memory occupancy, and the transmission bit rate occupancy is greater than the corresponding threshold value as a target container comprises:

[0073] Taking a container whose at least one of the CPU utilization, the resident memory occupancy, and the transmission bit rate occupancy is greater than the corresponding threshold value as an initial container;

[0074] Scaling the initial container to obtain a target container.

[0075] Further, taking a container whose at least one of the CPU utilization, the resident memory occupancy, and the transmission bit rate occupancy is greater than the corresponding threshold value as a target container comprises:

[0076] Obtaining a target processing type, wherein the target processing type is a processing type corresponding to the data processing method according to any one of the embodiments;

[0077] Determining a CPU utilization threshold value, a resident memory occupancy threshold value, and a transmission bit rate occupancy threshold value corresponding to the target processing type;

[0078] Taking a container satisfying at least one of the following as a target container:

[0079] The CPU utilization of the container is greater than the CPU utilization threshold value;

[0080] The resident memory occupancy of the container is greater than the resident memory occupancy threshold value;

[0081] The transmission bit rate occupancy of the container is greater than the transmission bit rate occupancy threshold value.

[0082] Further, the method further comprises:

[0083] obtaining a shrink number, a CPU utilization shrink threshold, a resident memory ratio shrink threshold, and a transmission bit rate ratio shrink threshold;

[0084] screening each container in the container cluster based on the shrink number, the CPU utilization shrink threshold, the resident memory ratio shrink threshold, and the transmission bit rate ratio shrink threshold to obtain a to-be-shrunk container;

[0085] shrinking the to-be-shrunk container.

[0086] Further, screening each container in the container cluster based on the shrink number, the CPU utilization shrink threshold, the resident memory ratio shrink threshold, and the transmission bit rate ratio shrink threshold to obtain a to-be-shrunk container, comprising:

[0087] selecting, as a candidate container, a container in the container cluster whose CPU utilization is less than the CPU utilization shrink threshold, whose resident memory ratio is less than the resident memory ratio shrink threshold, and whose transmission bit rate ratio is less than the transmission bit rate ratio shrink threshold;

[0088] selecting, as the to-be-shrunk container, the candidate container corresponding to the shrink number.

[0089] Further, scaling the initial container to obtain a target container, comprising:

[0090] obtaining historical CPU utilization and historical resident memory ratio of the initial container;

[0091] determining a CPU scaling threshold according to the historical CPU utilization of the initial container;

[0092] determining a resident memory scaling threshold according to the historical resident memory ratio of the initial container;

[0093] obtaining an ingress bandwidth and / or an egress bandwidth of the initial container;

[0094] scaling the initial container according to the ingress bandwidth and / or the egress bandwidth of the initial container, the CPU scaling threshold, and the resident memory scaling threshold to obtain a target container.

[0095] Further, selecting, as the target container, a container in which at least one of the CPU utilization, the resident memory ratio, and the transmission bit rate ratio is greater than a corresponding threshold, comprising:

[0096] obtaining a core type corresponding to the target processing type, wherein the core type comprises a lightweight core, a basic core, and a real-time core;

[0097] screening containers in the container cluster based on the core type corresponding to the target processing type to obtain an initial container set;

[0098] At least one of the CPU utilization, the resident memory occupancy and the transmission bit rate occupancy of the container in the initial container set is greater than the corresponding threshold value, and the container is taken as a target container.

[0099] Further, the method further comprises:

[0100] According to the core type corresponding to the target processing type, a resource type corresponding to the target container is determined.

[0101] Based on the resource type corresponding to the target container, resources are allocated to the target container.

[0102] Further, according to the core type corresponding to the target processing type, the resource type corresponding to the target container is determined, comprising:

[0103] If the core type corresponding to the target processing type is a light core, the resource type corresponding to the target container is a shared resource.

[0104] If the core type corresponding to the target processing type is a basic core, the resource type corresponding to the target container comprises a shared resource and a protection resource.

[0105] If the core type corresponding to the target processing type is a real-time core, the resource type corresponding to the target container is a protection resource.

[0106] According to another aspect of the present application, a data processing device is provided, comprising:

[0107] A first obtaining module is configured to obtain multi-modal data and a target data set, wherein the multi-modal data comprises acquisition data of at least two modalities, and the target data set comprises a plurality of groups of target data and an action corresponding to each group of target data, each group of target data comprising target data of each modality.

[0108] A first determining module is configured to determine a correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data.

[0109] A second determining module is configured to determine a fusion correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and a weight of each modality.

[0110] A third determining module is configured to determine a target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data.

[0111] An action executing module is configured to execute the target action corresponding to the multi-modal data.

[0112] According to another aspect of the present application, there is provided a target container determination apparatus, comprising:

[0113] a second obtaining module configured to obtain performance parameters of each container in the container cluster, wherein the performance parameters comprise: CPU utilization, resident memory occupancy, and transmission bit rate occupancy;

[0114] a target container determination module configured to determine, as a target container, a container whose at least one of the CPU utilization, the resident memory occupancy, and the transmission bit rate occupancy is greater than a corresponding threshold value;

[0115] a method execution module configured to execute the data processing method according to any one of the embodiments of the present application based on the target container.

[0116] According to another aspect of the present application, there is provided an electronic device, comprising:

[0117] at least one processor; and

[0118] a memory connected to the at least one processor in communication; wherein

[0119] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data processing method according to any one of the embodiments of the present application, or the target container determination method.

[0120] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to implement the data processing method according to any one of the embodiments of the present application, or the target container determination method, when executed by the processor.

[0121] According to another aspect of the present application, there is provided a computer program product, wherein the computer program is executed by a processor to implement the data processing method according to any one of the embodiments of the present application, or the target container determination method.

[0122] The embodiment of the application acquires multi-modal data and target data sets first; determines the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data; determines the fusion correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality; then determines the target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data; finally, the target action corresponding to the multi-modal data is executed, the matching of the target data can be performed through the fusion correlation coefficient, the calculation amount is reduced, and then the data processing for the fusion data can be realized through the embedded system, and the data processing efficiency is improved.

[0123] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0124] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some of the embodiments of the application, and therefore should not be regarded as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0125] Figure 1 is a flow chart of a data processing method in the embodiments of the application;

[0126] Figure 2 is a flow chart of another data processing method in the embodiments of the application;

[0127] Figure 3 is a schematic diagram of a target model training process in the embodiments of the application;

[0128] Figure 4 is a hardware compatible flowchart in the embodiments of the application;

[0129] Figure 5 is a flow chart of a target container determination method in the embodiments of the application;

[0130] Figure 6 is a coordination decision flowchart in the embodiments of the application;

[0131] Figure 7 is a resource allocation rule schematic diagram in the embodiments of the application;

[0132] Figure 8 is a structure schematic diagram of an operating system in the embodiments of the application;

[0133] Figure 9 is a structural schematic diagram of a data processing device in an embodiment of the present application;

[0134] Figure 10 is a structural schematic diagram of a target container determination device in an embodiment of the present application;

[0135] Figure 11 is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0136] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the protection scope of the present application.

[0137] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0138] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0139] Embodiment one

[0140] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application, the present embodiment can be applicable to the case of data processing, and the method can be executed by a data processing device in an embodiment of the present application. The device can be realized in the form of software and / or hardware, as shown in the figure, and the method specifically includes the following steps: Figure 1

[0141] S110, acquiring multi-modal data and a target data set.​

[0142] In the embodiment, the multi-modal data comprises acquisition data of at least two modalities, and the target data set comprises a plurality of groups of target data and an action corresponding to each group of target data, each group of target data comprising target data of each modality.

[0143] In the embodiment, the multi-modal data is sensor node acquisition data, and comprises multi-modal data of text, sound and image.

[0144] In the embodiment, the target data is reference data, and each group of reference data is previously known to have a corresponding action.

[0145] Optionally, the multi-modal data comprises image data, text data and audio data.

[0146] In the embodiment, the multi-modal data comprises image data, text data and audio data, and each group of target data comprises target data of image data, target data of text data and target data of audio data.

[0147] In S120, a correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data is determined.

[0148] In the embodiment, the manner of determining the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data can be: determining values of each dimension of the acquisition data of each modality and a dimension vector; determining values of each dimension of the target data in each group of target data and a dimension vector; and determining the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data according to the values of each dimension of the acquisition data of each modality, the dimension vector of the acquisition data of each modality, the values of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data.

[0149] In a specific example, there are two groups of target data, one group of target data comprising target data 1 of image data, target data 1 of text data and target data 1 of audio data, and the other group of target data comprising target data 2 of image data, target data 2 of text data and target data 2 of audio data. The correlation coefficient of the image data and the target data 1 of image data, the correlation coefficient of the text data and the target data 1 of text data, the correlation coefficient of the audio data and the target data 1 of audio data, the correlation coefficient of the image data and the target data 2 of image data, the correlation coefficient of the text data and the target data 2 of text data, and the correlation coefficient of the audio data and the target data 2 of audio data are determined.

[0150] Optionally, determining the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data comprises:

[0151] determining values of each dimension of the acquisition data of each modality and dimension vectors.

[0152] In this embodiment, the manner of determining the values of each dimension of the acquisition data of each modality and the dimension vectors can be: determining a covariance matrix corresponding to the acquisition data of each modality; and determining the values of each dimension of the acquisition data of each modality and the dimension vectors according to the covariance matrix corresponding to the acquisition data of each modality.

[0153] determining values of each dimension of the target data in each group of target data and dimension vectors.

[0154] In this embodiment, the manner of determining the values of each dimension of the target data in each group of target data and the dimension vectors is similar to the manner of determining the values of each dimension of the acquisition data of each modality and the dimension vectors, which will not be described herein again.

[0155] determining correlation coefficients between the acquisition data of each modality and the target data of each modality in each group of target data according to the values of each dimension of the acquisition data of each modality, the dimension vectors of the acquisition data of each modality, the values of each dimension of the target data in each group of target data and the dimension vectors of the target data in each group of target data.

[0156] In this embodiment, the manner of determining the correlation coefficients between the acquisition data of each modality and the target data of each modality in each group of target data according to the values of each dimension of the acquisition data of each modality, the dimension vectors of the acquisition data of each modality, the values of each dimension of the target data in each group of target data and the dimension vectors of the target data in each group of target data can be: determining vector indexes of the acquisition data of each modality according to the values of each dimension of the acquisition data of each modality and the dimension vectors of the acquisition data of each modality; determining vector indexes of the target data of each modality in each group of target data according to the values of each dimension of the target data in each group of target data and the dimension vectors of the target data in each group of target data; and determining the correlation coefficients between the acquisition data of each modality and the target data of each modality in each group of target data according to the vector indexes of the acquisition data of each modality and the vector indexes of the target data of each modality in each group of target data.

[0157] Optionally, the determining the values of each dimension of the acquisition data of each modality and the dimension vectors comprises:

[0158] determining a covariance matrix corresponding to the acquisition data of each modality.

[0159] In this embodiment, the manner of determining the covariance matrix corresponding to the acquisition data of each modality can be: processing the acquisition data of each modality to obtain a consistency matrix of the acquisition data of each modality; and determining the covariance matrix corresponding to the acquisition data of each modality based on the consistency matrix of the acquisition data of each modality.

[0160] According to the covariance matrix corresponding to the acquisition data of each modality, values of each dimension of the acquisition data of each modality and a dimension vector are determined.

[0161] In this embodiment, values of each dimension of the image data and the dimension vector are determined based on the following formula:

[0162] ;

[0163] wherein St is the covariance matrix corresponding to the image data, Pt is the dimension vector, is the value of each dimension.

[0164] It should be noted that the processing manner of the text data and the audio data is similar to that of the image data, which will not be described herein again.

[0165] Optionally, the covariance matrix corresponding to the acquisition data of each modality is determined, including:

[0166] The acquisition data of each modality is processed to obtain a consistency matrix of the acquisition data of each modality.

[0167] In this embodiment, the manner of processing the acquisition data of each modality to obtain the consistency matrix of the acquisition data of each modality can be: obtaining the mean value and the variance of the acquisition data of each modality; and determining the consistency matrix of the acquisition data of each modality according to the acquisition data of each modality, the mean value of the acquisition data of each modality and the variance of the acquisition data of each modality.

[0168] Based on the consistency matrix of the acquisition data of each modality, the covariance matrix corresponding to the acquisition data of each modality is determined.

[0169] In this embodiment, the manner of determining the covariance matrix corresponding to the acquisition data of each modality based on the consistency matrix of the acquisition data of each modality can be: obtaining the number of acquisition data types of each modality; and determining the covariance matrix corresponding to the acquisition data of each modality according to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality.

[0170] Optionally, the acquisition data of each modality is processed to obtain the consistency matrix of the acquisition data of each modality, including:

[0171] The mean value and the variance of the acquisition data of each modality are obtained;

[0172] The consistency matrix of the acquisition data of each modality is determined according to the acquisition data of each modality, the mean value of the acquisition data of each modality and the variance of the acquisition data of each modality.

[0173] In the embodiment, the consistency matrix of the acquisition data of each modality is determined according to the acquisition data of each modality, the mean of the acquisition data of each modality and the variance of the acquisition data of each modality, and the manner can be: taking the difference between the acquisition data of each modality and the mean as the difference of each modality; and taking the ratio of the difference of each modality and the variance as the consistency matrix of the acquisition data of each modality.

[0174] Optionally, the consistency matrix of the acquisition data of each modality is determined according to the acquisition data of each modality, the mean of the acquisition data of each modality and the variance of the acquisition data of each modality, and the consistency matrix of the acquisition data of each modality is determined according to the acquisition data of each modality, the mean of the acquisition data of each modality and the variance of the acquisition data of each modality.

[0175] The difference between the acquisition data of each modality and the mean is taken as the difference of each modality.

[0176] In a specific example, the data processing flow is as shown in Figure 2 The acquisition data is structured into a data matrix according to a time period by preprocessing, the principal elements are constructed through the consistency matrix and the covariance, the vector index of the acquisition sample is established based on the principal elements, the correlation analysis is performed with the standard vector index, and the classification result is output according to the correlation index. The principal element matrix of each time period is expressed in a data structure of , wherein P is a sample dimension vector, and the values of each dimension of the data are compared with the system standard database. The sample dimension vector can be the amplitude, mean, variance, skewness, kurtosis, etc. of the time domain waveform, and the frequency band, coherence, power spectrum estimation, etc. of the frequency domain signal.

[0177] An m*n matrix X is established, n is the number of samples (sampling times), and m is the number of detected variables (the number of acquisition data types), which is represented as:

[0178]

[0179] The ratio of the difference of each modality and the variance is taken as the consistency matrix of the acquisition data of each modality.

[0180] In a specific example, the consistency matrix of the image data is determined based on the following formula:

[0181] ;

[0182] wherein, is the consistency matrix of the image data, is the image data, is the variance matrix of the image data, is the mean of the image data, e is a full rank matrix, the elements in the matrix are composed of 1, the mean is selected, is the mean matrix.

[0183] It should be noted that the determination manner of the consistency matrix of the text data and the audio data is similar to the determination manner of the consistency matrix of the image data, and details are not repeated here.

[0184] Optionally, based on the consistency matrix of the acquisition data of each modality, the covariance matrix corresponding to the acquisition data of each modality is determined, comprising:

[0185] The number of acquisition data types of each modality is obtained.

[0186] According to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality, the covariance matrix corresponding to the acquisition data of each modality is determined.

[0187] In the embodiment, the manner of determining the covariance matrix corresponding to the acquisition data of each modality according to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality can be that: taking the product of the transpose of the consistency matrix of the acquisition data of each modality and the consistency matrix of the acquisition data of each modality as a first matrix; taking the ratio of the first matrix and the number of acquisition data types of each modality as the covariance matrix corresponding to the acquisition data of each modality.

[0188] Optionally, according to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality, the covariance matrix corresponding to the acquisition data of each modality is determined, comprising:

[0189] The product of the transpose of the consistency matrix of the acquisition data of each modality and the consistency matrix of the acquisition data of each modality is taken as a first matrix.

[0190] The ratio of the first matrix and the number of acquisition data types of each modality is taken as the covariance matrix corresponding to the acquisition data of each modality.

[0191] In a specific example, the covariance matrix of the image data is determined based on the following formula:

[0192]

[0193] Wherein, St is the covariance matrix of Yt matrix, Yt is the consistency matrix of the image data, and m is the number of acquisition data types.

[0194] Optionally, according to the values of each dimension of the acquisition data of each modality, the dimension vector of the acquisition data of each modality, the values of each dimension of the target data in each group of target data, and the dimension vector of the target data in each group of target data, the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data is determined, comprising:

[0195] According to the value of each dimension of the acquisition data of each modality and the dimension vector of the acquisition data of each modality, a vector index of the acquisition data of each modality is determined.

[0196] In this embodiment, the vector index of the acquisition data is used to represent the overall state or trend of the acquisition data.

[0197] In this embodiment, the manner of determining the vector index of the acquisition data of each modality according to the value of each dimension of the acquisition data of each modality and the dimension vector of the acquisition data of each modality can be that the vector index of the acquisition data of each modality is generated by weighting, dimension reduction or a machine learning model based on the value of each dimension of the acquisition data of each modality and the dimension vector of the acquisition data of each modality.

[0198] According to the value of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data, a vector index of the target data of each modality in each group of target data is determined.

[0199] In this embodiment, the manner of determining the vector index of the target data of each modality in each group of target data according to the value of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data is similar to the manner of determining the vector index of the acquisition data of each modality according to the value of each dimension of the acquisition data of each modality and the dimension vector of the acquisition data of each modality, which will not be described herein again.

[0200] In a specific example, for the vector index of the acquisition data, for the vector index of the target data.

[0201] According to the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data, a correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data is determined.

[0202] In this embodiment, the ratio of the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data is taken as the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data.

[0203] Optionally, determining the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data according to the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data comprises:

[0204] taking the ratio of the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data as the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data.

[0205] In one specific example, the correlation coefficient of the image data and the target data is determined based on the following formula:

[0206] ;

[0207] is the vector index of the acquisition data, is the vector index of the target data, is the correlation coefficient of the image data and the target data of the image data.

[0208] The fast execution of the task is performed by calculating the correlation coefficient, and the higher the correlation coefficient is, the higher the correlation is. In essence, the acquisition data with higher similarity is combined to obtain a fast processing result.

[0209] S130, according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality, determine the fusion correlation coefficient of the multi-modal data and each group of target data.

[0210] In this embodiment, the way to determine the fusion correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality can be: based on the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality, weighted sum is performed to obtain the fusion correlation coefficient of the multi-modal data and each group of target data.

[0211] Optionally, the fusion correlation coefficient of the multi-modal data and each group of target data is determined according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality, comprising:

[0212] Obtain the matrix proportion of each modality.

[0213] In this embodiment, the matrix proportion of the image data, the matrix proportion of the text data and the matrix proportion of the audio data are obtained.

[0214] According to the matrix proportion of each modality, determine the weight of each modality.

[0215] In this embodiment, the sum of the matrix proportion of the image data, the matrix proportion of the text data and the matrix proportion of the audio data is taken as the total proportion, the ratio of the matrix proportion of the image data to the total proportion is taken as the weight of the image data, the ratio of the matrix proportion of the text data to the total proportion is taken as the weight of the text data, and the ratio of the matrix proportion of the audio data to the total proportion is taken as the weight of the audio data.

[0216] The fusion correlation coefficient of the multi-modal data and each group of target data is obtained by weighted summation based on the correlation coefficient of the collected data of each modality and the target data of each modality in each group of target data and the weight of each modality.

[0217] In this embodiment, the fusion correlation coefficient of the multi-modal data and the i-th group of target data is determined based on the following formula:

[0218] ;

[0219] wherein, is the target data of the image data (belonging to the i-th group of target data), is the target data of the audio data (belonging to the i-th group of target data), is the target data of the text data (belonging to the i-th group of target data), is the fusion correlation coefficient of the multi-modal data and the i-th group of target data, is the weight of the image data, is the weight of the audio data, is the weight of the text data.

[0220] S140, determining the target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data.

[0221] In this embodiment, the manner of determining the target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data can be: if the fusion correlation coefficient of the multi-modal data and at least one group of target data is greater than a correlation coefficient threshold, determining the target action based on a standard index model; if the fusion correlation coefficient of the multi-modal data and each group of target data is less than or equal to the correlation coefficient threshold, inputting the multi-modal data into a target model to obtain the target action corresponding to the multi-modal data. The manner of determining the target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data can also be: if the fusion correlation coefficient of the multi-modal data and at least one group of target data is greater than a correlation coefficient threshold, determining the target action corresponding to the multi-modal data according to the at least one group of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold; if the fusion correlation coefficient of the multi-modal data and each group of target data is less than or equal to the correlation coefficient threshold, inputting the multi-modal data into a target model to obtain the target action corresponding to the multi-modal data, wherein the target model sequentially includes a state perceiver, a learner and an action selector from the input to the output direction.

[0222] Optionally, determining the target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data comprises:

[0223] If the fusion correlation coefficient of the multi-modal data and at least one set of target data is greater than the correlation coefficient threshold, a target action corresponding to the multi-modal data is determined according to at least one set of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold.

[0224] In this embodiment, if there is a set of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold, an action corresponding to the set of target data is taken as the target action corresponding to the multi-modal data; if there are multiple sets of target data whose fusion correlation coefficients with the multi-modal data are all greater than the correlation coefficient threshold, an action corresponding to a set of target data whose fusion correlation coefficient with the multi-modal data is the largest in the set of target data is taken as the target action corresponding to the multi-modal data. Alternatively, an action corresponding to any set of target data is taken as the target action corresponding to the multi-modal data.

[0225] If the fusion correlation coefficients of the multi-modal data and each set of target data are less than or equal to the correlation coefficient threshold, the multi-modal data is input into a target model to obtain a target action corresponding to the multi-modal data.

[0226] In this embodiment, the target model sequentially includes a state perceiver, a learner and an action selector from the input to the output direction.

[0227] It should be noted that the target model can be obtained by iteratively training a basic training model.

[0228] Optionally, determining a target action corresponding to the multi-modal data according to at least one set of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold includes:

[0229] If there is a set of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold, an action corresponding to the set of target data is taken as the target action corresponding to the multi-modal data.

[0230] If there are multiple sets of target data whose fusion correlation coefficients with the multi-modal data are all greater than the correlation coefficient threshold, an action corresponding to a set of target data whose fusion correlation coefficient with the multi-modal data is the largest in the set of target data is taken as the target action corresponding to the multi-modal data.

[0231] Optionally, inputting the multi-modal data into a target model to obtain a target action corresponding to the multi-modal data includes:

[0232] Encoding the multi-modal data into a state feature vector by the state perceiver;

[0233] The parameter of the policy model is updated by the learner based on the state feature vector and a reward function;

[0234] The target action corresponding to the multi-modal data is selected from the action space by the action selector according to the updated policy model.

[0235] In the embodiment, the agent corresponds to the standard vector library from the perception state s, selects the action a based on the standard vector index, and delivers to the vector library; after receiving the action of the agent, the state s changes to ; the learner obtains the reward r of the interaction between the agent and the vector library, which is used for the training of the action a.

[0236] Optionally, the reward function is:

[0237] ;

[0238] wherein, is a reward function, A is an action space, is a reward vector, is a reward value obtained when the action is taken, is the kth action in the action space, s is the current state, is the next state after the action is executed, is the change amount of the state.

[0239] In the embodiment, is a selection factor, which is used to represent the weight of the reward at different times, and the target optimal action is realized by updating R through the policy learner Q.

[0240] The solution of the target can be based on the gradient descent method . In the original action set A of the agent, the agent selects the optimal action by calculating the action value when each action is taken, and selects the minimum action value as the optimal action, so that .

[0241] In a specific example, the target model training process is as shown in Figure 3 . The target model is obtained by iteratively training a basic training model, and the basic training model includes a state perceiver S, an action selection basic training modeler A, and a policy learner Q. The agent corresponds to the standard vector library from the perception state s, and selects the action a based on the standard vector index, and delivers to the vector library; after receiving the action of the agent, the state s changes to ; the learner obtains the reward r of the interaction between the agent and the vector library, which is used for the training of the action a. Figure 3 The standard index model in the basic training model is used to store the corresponding relationship between the standard data and the action.

[0242] The target model provided by the embodiment is used to compare the collected data, and then control results are quickly output.

[0243] S150, performing a target action corresponding to the multi-modal data.

[0244] The power system intelligent control algorithm usually needs to identify the operation mode in the information with noise, high dimension, complex correlation and time translation, which often brings problems such as large calculation amount and lag delay. The embodiment provides a data processing method using simplified parallel index comparison, multi-modal fusion extraction technology and multi-channel parallel reference strategy. The data extraction and comparison strategy is optimized, the reasoning calculation complexity is effectively reduced, and the data processing speed is improved. Multi-channel parallel fully utilizes the parallel computing capability of the multi-core platform, and further improves the data processing efficiency.

[0245] Optionally, the method further comprises:

[0246] Obtaining a set of instruction sets.

[0247] In the embodiment, the set of instruction sets comprises a plurality of versions of instruction sets corresponding to a plurality of hardware architectures. For example, the set of instruction sets comprises a version a instruction set, a version b instruction set and a version c instruction set corresponding to a hardware architecture A; and a version a instruction set, a version b instruction set and a version c instruction set corresponding to a hardware architecture B.

[0248] Based on the overhead time of executing the corresponding versions of instruction sets of the current hardware architecture, the set of instruction sets is screened to obtain a target instruction set.

[0249] In the embodiment, the way of screening the set of instruction sets based on the overhead time of executing the corresponding versions of instruction sets of the current hardware architecture to obtain the target instruction set can be that: the instruction set of the target version with a ratio to the average of the overhead time less than or equal to a set threshold is taken as the target instruction set.

[0250] The target instruction set is burned to the current hardware architecture.

[0251] Optionally, based on the overhead time of executing the corresponding versions of instruction sets of the current hardware architecture, the set of instruction sets is screened to obtain a target instruction set, comprising:

[0252] Determine the average of the overhead time.

[0253] In the embodiment, the overhead time is the overhead time of executing the corresponding versions of instruction sets of the current hardware architecture.

[0254] The instruction set of the target version with a ratio to the average of the overhead time less than or equal to a set threshold is taken as the target instruction set.

[0255] In the embodiment, the set threshold value can be a pre-set value, for example, the set threshold value can be 1.

[0256] In the embodiment, the way of taking the instruction set of the target version whose ratio to the average overhead time as the target instruction set can be:

[0257] The target instruction set satisfies the following formula:

[0258] ;

[0259] Wherein, is the average overhead time of executing the instruction set of each version, is the overhead time of the target instruction set.

[0260] Optionally, the overhead time of the current hardware architecture executing the corresponding instruction set of each version is the time of the current hardware architecture executing the corresponding instruction set of each version.

[0261] In a specific example, the hardware compatibility process is as shown in Figure 4 The firmware initialization interface, the hardware feature set Wherein, , hz1, hz2, hz3, hz4 all represent the architecture of hardware z. The kernel abstraction set, that is, the instruction set collection dependent on the architecture The kernel code set includes: a plurality of versions of instruction sets corresponding to a plurality of hardware architectures, for example: a version of instruction set, a version of instruction set, a version of instruction set corresponding to hardware architecture A; a version of instruction set, a version of instruction set, a version of instruction set corresponding to hardware architecture B.

[0262] The mapping function is established ;

[0263] ; is the average overhead time of executing the instruction set of each version.

[0264] Mapping efficiency: ; The above formula means: taking the instruction set of the target version whose ratio to the average overhead time as the target instruction set.

[0265] represents the overhead time of executing the best instruction set, .

[0266] The best mapping efficiency represents the optimal kernel code combination of the compatible hardware architecture. The cross-architecture performance loss can be less than 10%, the startup time can be optimized by 30-50%, and the bottom support is provided for the lightweight operating system and heterogeneous hardware compatibility.

[0267] The embodiment of the application proposes a hardware compatible instruction set construction method based on a kernel code library, fixed codes based on a code library, translated codes, and mapping of hardware-kernel through mathematical modeling into a constrained optimization problem, so as to improve execution efficiency and realize multi-architecture compatibility.

[0268] The technical scheme of the embodiment, by acquiring multi-modal data and a target data set, wherein the multi-modal data includes acquisition data of at least two modalities, and the target data set includes multiple groups of target data and an action corresponding to each group of target data, each group of target data including target data of each modality, determining a correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data, determining a fusion correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and the weight of each modality, determining a target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data, and executing the target action corresponding to the multi-modal data, can realize data processing for fusion data through an embedded system and improve data processing efficiency.

[0269] Embodiment two

[0270] Figure 5 A flowchart of a target container determination method provided by the embodiment of the application, the embodiment can be applied to a target container determination case, the method can be executed by a target container determination device in the embodiment of the application, and the device can be realized in a software and / or hardware manner, for example, as shown in the figure, and the method specifically includes the following steps. Figure 5

[0271] S510, acquiring performance parameters of each container in a container cluster.

[0272] In the embodiment, the performance parameters include CPU utilization, resident memory occupancy, and transmission bit rate occupancy.

[0273] In the embodiment, the performance parameters of each container in the container cluster are acquired based on the following formula:

[0274]

[0275]

[0276]

[0277] wherein, represents the percentage of CPU usage of the container d to the CPU allocated by the system in a unit of statistical time; ​​​​, indicates that the size of the resident memory occupied by the container d in a unit of statistical time accounts for a percentage of the physical memory allocated by the system; , indicates that the transmission bit rate of the container d in a unit of statistical time accounts for a percentage of the allocated rate of the system.

[0278] S520, taking the container whose at least one of the CPU utilization rate, the resident memory occupancy ratio and the transmission bit rate ratio is greater than the corresponding threshold value as a target container.

[0279] In this embodiment, the manner of taking the container whose at least one of the CPU utilization rate, the resident memory occupancy ratio and the transmission bit rate ratio is greater than the corresponding threshold value as a target container can be: taking the container whose at least one of the CPU utilization rate, the resident memory occupancy ratio and the transmission bit rate ratio is greater than the corresponding threshold value as an initial container; expanding the initial container to obtain a target container.

[0280] Optionally, taking the container whose at least one of the CPU utilization rate, the resident memory occupancy ratio and the transmission bit rate ratio is greater than the corresponding threshold value as a target container comprises:

[0281] Taking the container whose at least one of the CPU utilization rate, the resident memory occupancy ratio and the transmission bit rate ratio is greater than the corresponding threshold value as an initial container.

[0282] In this embodiment, the manner of taking the container whose at least one of the CPU utilization rate, the resident memory occupancy ratio and the transmission bit rate ratio is greater than the corresponding threshold value as an initial container can be: obtaining a target processing type, wherein the target processing type is a processing type corresponding to the data processing method described in the above embodiments; determining a CPU utilization rate threshold value, a resident memory occupancy ratio threshold value and a transmission bit rate ratio threshold value corresponding to the target processing type; and taking the container satisfying at least one of the following as a target container: the CPU utilization rate of the container is greater than the CPU utilization rate threshold value; the resident memory occupancy ratio of the container is greater than the resident memory occupancy ratio threshold value; and the transmission bit rate ratio of the container is greater than the transmission bit rate ratio threshold value.

[0283] Expanding the initial container to obtain a target container.

[0284] In this embodiment, the manner of expanding the initial container to obtain a target container can be: obtaining a historical CPU utilization rate and a historical resident memory occupancy ratio of the initial container; determining a CPU expansion threshold value according to the historical CPU utilization rate of the initial container; determining a resident memory expansion threshold value according to the historical resident memory occupancy ratio of the initial container; obtaining an inlet and / or outlet bandwidth of the initial container; and expanding the initial container according to the inlet and / or outlet bandwidth of the initial container, the CPU expansion threshold value and the resident memory expansion threshold value to obtain a target container.

[0285] Optionally, the container whose at least one of the CPU utilization, the resident memory ratio, and the transmission bit rate ratio is greater than the corresponding threshold value is taken as the target container, including:

[0286] The target processing type is obtained, wherein the target processing type is a processing type corresponding to the data processing method.

[0287] In this embodiment, the target processing type can be obtained in the following manner: obtaining a processing type corresponding to the data processing method. For example, if the data processing method is artificial intelligence calculation, the target processing type is determined to be an artificial intelligence calculation type. The CPU utilization threshold value, the resident memory ratio threshold value, and the transmission bit rate ratio threshold value corresponding to the target processing type are determined.

[0288] In this embodiment, a correspondence table of processing types and performance parameter threshold values is established in advance. For example, processing type A corresponds to performance parameter threshold values: PU utilization threshold value x1, resident memory ratio threshold value x2, and transmission bit rate ratio threshold value x3. Processing type B corresponds to performance parameter threshold values: PU utilization threshold value x4, resident memory ratio threshold value x5, and transmission bit rate ratio threshold value x6.

[0289] In this embodiment, the correspondence table is queried to obtain the CPU utilization threshold value, the resident memory ratio threshold value, and the transmission bit rate ratio threshold value corresponding to the target processing type.

[0290] The container satisfying at least one of the following conditions is taken as the target container:

[0291] The CPU utilization of the container is greater than the CPU utilization threshold value;

[0292] The resident memory ratio of the container is greater than the resident memory ratio threshold value;

[0293] The transmission bit rate ratio of the container is greater than the transmission bit rate ratio threshold value.

[0294] In a specific example, the coordination decision-making process is as follows Figure 6As shown, the real-time running data of the container cluster is acquired, threshold upper limit values Vcmax and Vcmin, threshold upper limit values Vmmax and Vmmin, and threshold upper limit values Vnmax and Vnmin are set for Vc (CPU utilization), Vm (resident memory occupancy), and Vn (transmission bit rate occupancy) respectively. When any one of the container resource consumption indexes VC, VM, and VN exceeds the threshold upper limit value for the first time, the evaluation stage is entered, and an array is established. When the subsequent resource consumption index again exceeds the threshold upper limit value, it is added to the array again. If one of the subsequent resource consumption indexes is less than the threshold upper limit value, the array is cleared and the next evaluation is waited for again. If the actual use value of the resource is always greater than the threshold upper limit value, it is considered that the corresponding service needs to be expanded, and the set threshold of the container access resource is increased. The contraction process is the same (less than the threshold lower limit value).

[0295] In the embodiment, an optimal container is selected from the container cluster for the data processing method of the above embodiment. The specific selection rule is: the Vc threshold, the Vm threshold, and the Vn threshold corresponding to the data processing method are acquired, and the container with Vc less than the Vc threshold, Vm less than the Vm threshold, and Vn less than the Vn threshold is selected as the target container.

[0296] Optionally, the method further comprises:

[0297] The contraction number, the CPU utilization contraction threshold, the resident memory occupancy contraction threshold, and the transmission bit rate occupancy contraction threshold are acquired.

[0298] In the embodiment, the contraction number, the CPU utilization contraction threshold, the resident memory occupancy contraction threshold, and the transmission bit rate occupancy contraction threshold can be pre-set.

[0299] In the embodiment, the contraction number can be the number of containers to be contracted.

[0300] The containers in the container cluster are screened based on the contraction number, the CPU utilization contraction threshold, the resident memory occupancy contraction threshold, and the transmission bit rate occupancy contraction threshold to obtain the containers to be contracted.

[0301] In the embodiment, the way of screening the containers in the container cluster based on the contraction number, the CPU utilization contraction threshold, the resident memory occupancy contraction threshold, and the transmission bit rate occupancy contraction threshold to obtain the containers to be contracted can be: the containers with CPU utilization less than the CPU utilization contraction threshold, resident memory occupancy less than the resident memory occupancy contraction threshold, and transmission bit rate occupancy less than the transmission bit rate occupancy contraction threshold in the container cluster are selected as candidate containers; and the candidate containers corresponding to the contraction number are selected as the containers to be contracted.

[0302] shrink the to-be-shrunk container.

[0303] Optionally, the containers in the container cluster are screened based on the shrink number, the CPU utilization shrink threshold, the resident memory ratio shrink threshold, and the transmission bit rate ratio shrink threshold to obtain the to-be-shrunk container, including:

[0304] The container in the container cluster whose CPU utilization is less than the CPU utilization shrink threshold, whose resident memory ratio is less than the resident memory ratio shrink threshold, and whose transmission bit rate ratio is less than the transmission bit rate ratio shrink threshold is taken as a to-be-selected container.

[0305] The to-be-selected container corresponding to the shrink number is taken as the to-be-shrunk container.

[0306] In this embodiment, if there are multiple to-be-selected containers, the multiple to-be-selected containers are screened according to the shrink number to obtain the to-be-shrunk container.

[0307] In a specific example, if there are 10 to-be-selected containers and the shrink number is 6, the 10 to-be-selected containers are sorted in the order of CPU utilization, resident memory ratio, and transmission bit rate ratio from small to large to obtain a container sequence, and the first 6 containers in the container sequence are selected as the to-be-shrunk container.

[0308] Optionally, the initial container is scaled to obtain a target container, including:

[0309] The historical CPU utilization and the historical resident memory ratio of the initial container are obtained.

[0310] The CPU scaling threshold is determined according to the historical CPU utilization of the initial container.

[0311] The resident memory scaling threshold is determined according to the historical resident memory ratio of the initial container.

[0312] The initial container inlet bandwidth and / or outlet bandwidth is obtained.

[0313] The initial container is scaled according to the initial container inlet bandwidth and / or outlet bandwidth, the CPU scaling threshold, and the resident memory scaling threshold to obtain a target container.

[0314] In this embodiment, if the initial container inlet bandwidth exceeds the threshold for a target time period, the network or computing resource of the container needs to be scaled. If the initial container outlet bandwidth exceeds the threshold for a target time period, the network or computing resource of the container needs to be scaled. If the CPU utilization exceeds the CPU scaling threshold, it indicates that the computing resource is insufficient, and the CPU needs to be scaled. If the resident memory usage rate exceeds the resident memory scaling threshold, it indicates that the memory resource is insufficient, and the memory needs to be scaled.

[0315] After the coordination management, a cooling period will be entered, during which no evaluation will be carried out to avoid frequent updates of container cluster indicators from causing severe fluctuations in system services. Resource scheduling: mainly responsible for allocating and scheduling the underlying resources of the container cluster. When the cluster needs to be expanded or shrunk, resource scheduling will create or destroy containers according to the coordination decision. As the number of further applications continues to increase, the demand for computing resources also increases, and the amount of resources required by the terminal needs to be analyzed and evaluated with the amount of resources that the terminal can actually provide.

[0316] A service capability evaluation algorithm for the terminal is constructed to calculate the required resource amount of the to-be-scheduled container and the residual resource amount evaluation score of the terminal. The evaluation rule is:

[0317] ;

[0318] Wherein, DC, DM, DN are the required resource amount of the to-be-scheduled container, CapC, CapM, CapN are the residual resource amount, and int() is the integer function.

[0319] After evaluation, the to-be-scheduled container image parameters are pre-paired with the corresponding hardware resources, the working directory corresponding to the container is created, and the necessary environment variables and parameters of the container are generated.

[0320] Optionally, at least one container with a CPU utilization rate, a resident memory occupancy ratio, and a transmission bit rate occupancy ratio greater than a corresponding threshold value is taken as a target container, including:

[0321] The core type corresponding to the target processing type is obtained.

[0322] In this embodiment, the core type includes a lightweight core, a basic core, and a real-time core.

[0323] In this embodiment, the lightweight core is a CPU core designed for low-load and burst lightweight tasks. The resource allocation priority is low, and the resources are shared with other tenants, and there is resource competition. The basic core is a CPU core for medium-load and high-stability conventional tasks. Shared resources and protected resources are provided, the protected resources are isolated from other tenants, and the performance fluctuations caused by resource competition are reduced. The real-time core is a CPU core optimized for high-priority, low-latency, and strong real-time tasks. Protected resources are allocated, completely isolated from other tenants, and ensure that computing resources are available at any time.

[0324] In this embodiment, a correspondence between the processing type and the core type is established in advance, and the correspondence is queried to obtain the core type corresponding to the target processing type.

[0325] Filter the containers in the container cluster based on the core type corresponding to the target processing type to obtain an initial container set.

[0326] It should be noted that the containers can be filtered according to the resource isolation of the core type, for example, a lightweight core is suitable for containers without strong resource isolation requirements; a basic core is suitable for containers with medium isolation requirements; and a real-time core is suitable for containers with strong isolation requirements.

[0327] At least one of the CPU utilization, resident memory occupancy, and transmission bit rate occupancy of the container in the initial container set is greater than the corresponding threshold value.

[0328] Optionally, the method further comprises:

[0329] According to the core type corresponding to the target processing type, the resource type corresponding to the target container is determined.

[0330] In this embodiment, if the core type corresponding to the target processing type is a lightweight core, the resource type corresponding to the target container is a shared resource; if the core type corresponding to the target processing type is a basic core, the resource type corresponding to the target container includes a shared resource and a protected resource; and if the core type corresponding to the target processing type is a real-time core, the resource type corresponding to the target container is a protected resource.

[0331] Based on the resource type corresponding to the target container, resources are allocated to the target container.

[0332] Optionally, according to the core type corresponding to the target processing type, the resource type corresponding to the target container is determined, comprising:

[0333] If the core type corresponding to the target processing type is a lightweight core, the resource type corresponding to the target container is a shared resource;

[0334] If the core type corresponding to the target processing type is a basic core, the resource type corresponding to the target container includes a shared resource and a protected resource;

[0335] If the core type corresponding to the target processing type is a real-time core, the resource type corresponding to the target container is a protected resource.

[0336] It should be noted that DC, DM, and DN are resource amounts required by the to-be-scheduled containers, which can be analyzed and predicted through Vc, Vm, and Vn. CapC, CapM, and CapN are residual resource amounts, representing the resource amounts that can be used under the capability of the operating system. In view of the limited resource amount of the power intelligent terminal and the diversity characteristics of real-time and non-real-time tasks, a hierarchical sharing and mutual exclusion mechanism is proposed to maximize the resource amount.

[0337] There are two types of computing resource control signals, binary semaphore and counting semaphore. Binary semaphore can only take 0 and 1. This semaphore is used to protect resources, ensuring that only one process or thread can access shared resources at the same time, avoiding data inconsistency. Counting semaphore can have an initial value and can be increased or decreased. This semaphore is used to control the access permission of multiple processes or threads, allowing a certain number of processes to access shared resources at the same time.

[0338] In this embodiment, as shown in Figure 7 , combined with the characteristics of power system business, the embodiment defines three states of light kernel, basic kernel and real-time kernel. Light kernel uses counting semaphore mode, shared resource area, basic kernel uses two modes, and real-time kernel uses binary semaphore to protect resource area.

[0339] In this embodiment, the running mode of shared resources is as follows: loading the context of process A, executing process A, saving the context of process A, loading the context of process B, executing process B, and saving the context of process B. The running mode of protected resources is as follows: loading the context of process A, executing process A, loading the context of process B, and executing process B.

[0340] In this embodiment, the shared and mutual exclusion protection algorithm flow is as follows:

[0341] Indicates whether process i is resource protected, shared state , protected state .

[0342] Indicates the ordering of process i in sharing the resource.

[0343] N: total number of processes.

[0344] When a process needs to be protected by resources , the system allocates to the protected resource area. When accesses shared resources, a value is allocated, and the waiting is ordered. When is greater than the current value, the process is blocked until the value of the semaphore is less than other processes, and process i is awakened. When a process finishes using shared resources, it performs an exit operation, =0.

[0345] The steps of entering the shared area are as follows:

[0346] Ordering phase:

[0347] .

[0348] Waiting phase:

[0349] : wait Execution is complete.

[0350] : The thread with a larger thread ID has a lower priority.

[0351] Enter the critical section:

[0352] When the above waiting condition is met, the thread i enters the critical section.

[0353] Exit the critical section:

[0354] After the thread i completes the critical section operation, the resource needs to be released, = 0.

[0355] The shared and mutual exclusion mechanism can achieve the best response matching for the task type, and can also mine the best computing power of the terminal CapC, CapM, and CapN are the remaining resource quantities, and the formula is as follows:

[0356] Cap remaining = Cap unit * ∑ + Cap unit * max( , , …, ).

[0357] In the case of meeting the resource quantity of the real-time task , by expanding the shared resource quantity , the resource can be fully utilized and mined. Cap is the smallest unit of the unit resource.

[0358] The embodiment proposes a task arrangement algorithm based on virtual utilization rate, a plurality of virtual containers for meeting task type execution request, and then combines the capacity calculation characteristics to realize a task arrangement and distribution method, while guaranteeing load balancing, the task is quickly executed.

[0359] The embodiment proposes a shared and mutual exclusion mechanism of computing resource grading, proposes a logic algorithm of computing resource control signal according to the grading response requirement of the task, realizes the sharing and mutual exclusion of the resource, and maximally mines the computing power resource of the system.

[0360] S530, based on the target container, the data processing method described in the above embodiment is executed.

[0361] In one specific example, the operating system architecture is as shown in Figure 8 , and the operating system architecture includes a system interface module, a kernel module, a component module and a user module. The operating system provided by the embodiment has the characteristics of compatibility, light weight, multi-tasking and resource sharing.

[0362] The system interface module is built-in heterogeneous architecture kernel instruction code, contains multiple instruction set codes, automatically selects at runtime, supports multiple architectures, provides more choices and flexibility, and saves a lot of time and resources without redesigning and optimizing the model for each chip platform.

[0363] The kernel module supports lightweight cores, functional cores and real-time cores, can be adapted to various devices, and meets different requirements of computing power and timeliness of various power distribution services.

[0364] The component module provides an accelerated mapping control algorithm, a task scheduling algorithm, and a computing resource sharing and mutual exclusion function, meets the high real-time, high reliability and high performance requirements of the upper application, and realizes flexible business logic management and resource allocation.

[0365] The user module provides business function support and application development environment, and facilitates business program development.

[0366] The heterogeneous hardware infrastructure includes an Advanced RISC Machine (ARM) and a Reduced Instruction Set Computer-Fifth Generation (RISC-V).

[0367] The operating system optimizes the selection of the instruction set encapsulation design of the intelligent terminal hardware at the bottom, and shields the complexity of the heterogeneous computing. The upper layer establishes a set of reusable and extensible tools, services and frameworks to provide sufficient computing power support and algorithm guarantee for the differentiated needs of the business.

[0368] The target container determination method provided in the embodiment is suitable for a lightweight operating system of a power intelligent terminal, supports multiple hardware architectures through a compatible kernel code library and a lightweight kernel call based on a kernel architecture; based on a predicted interaction algorithm, multi-modal rapid extraction and acceleration and task rapid mapping execution are realized; through a task and resource scheduling algorithm and a hardware resource sharing and mutual exclusion mechanism, the sharing and optimized scheduling functions of the hardware resources are realized, the computing power of the system is fully tapped, and the implementation guarantee of multi-task is realized.

[0369] The technical scheme provided by the embodiment is used for dynamic loading of real-time tasks and non-real-time tasks of power supply services, and a multi-type resource utilization rate priority allocation method is proposed for multi-core allocation in a multi-core processor. A plurality of virtual containers for meeting task type execution requests are constructed, and then a task allocation method is proposed in combination with capacity calculation characteristics, the core of which is to dynamically allocate the accessible resource amount of the container according to the utilization rate of the processor core and other resources required by the current task. A service capability evaluation algorithm for the terminal is constructed, the core of which is to provide prediction for the further service capability of the terminal according to the processor core and other resources required by the future task.

[0370] Embodiment three

[0371] Figure 9 A structural schematic diagram of a data processing device provided by the embodiment of the application. The embodiment can be applied to the case of data processing. The device can be realized in the manner of software and / or hardware. The device can be integrated in any device providing data processing functions, for example, can be configured in an embedded system, as shown in the figure. Figure 9 The data processing device specifically includes a first acquisition module 910, a first determination module 920, a second determination module 930, a third determination module 940, and an action execution module 950.

[0372] The first acquisition module is configured to acquire multi-modal data and a target data set. The multi-modal data includes acquisition data of at least two modalities. The target data set includes a plurality of groups of target data and an action corresponding to each group of target data. Each group of target data includes target data of each modality.

[0373] The first determination module is configured to determine a correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data.

[0374] The second determination module is configured to determine a fusion correlation coefficient of the multi-modal data and each group of target data according to the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data and a weight of each modality.

[0375] The third determination module is configured to determine a target action corresponding to the multi-modal data according to the fusion correlation coefficient of the multi-modal data and each group of target data.

[0376] The action execution module is configured to execute the target action corresponding to the multi-modal data.

[0377] Optionally, the third determination module is specifically configured to:

[0378] If the fusion correlation coefficient of the multi-modal data and at least one group of target data is greater than the correlation coefficient threshold, a target action corresponding to the multi-modal data is determined according to at least one group of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold.

[0379] If the fusion correlation coefficient of the multi-modal data and each group of target data is less than or equal to the correlation coefficient threshold, the multi-modal data is input into a target model to obtain a target action corresponding to the multi-modal data, wherein the target model sequentially includes a state perceiver, a learner and an action selector from the input to the output.

[0380] Optionally, the third determining module is specifically configured to:

[0381] If there is a group of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold, an action corresponding to the group of target data is taken as a target action corresponding to the multi-modal data.

[0382] If there are multiple groups of target data whose fusion correlation coefficient with the multi-modal data is greater than the correlation coefficient threshold, an action corresponding to a group of target data whose fusion correlation coefficient with the multi-modal data is the largest in the target data set is taken as a target action corresponding to the multi-modal data.

[0383] Optionally, the first determining module is specifically configured to:

[0384] Determine the value of each dimension of the collected data of each modality and the dimension vector.

[0385] Determine the value of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data.

[0386] According to the value of each dimension of the collected data of each modality, the dimension vector of the collected data of each modality, the value of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data, determine the correlation coefficient of the collected data of each modality and the target data of each modality in each group of target data.

[0387] Optionally, the first determining module is specifically configured to:

[0388] Determine the covariance matrix corresponding to the collected data of each modality.

[0389] According to the covariance matrix corresponding to the collected data of each modality, determine the value of each dimension of the collected data of each modality and the dimension vector.

[0390] Optionally, the first determining module is specifically configured to:

[0391] Process the collected data of each modality to obtain a consistency matrix of the collected data of each modality.

[0392] determine a covariance matrix corresponding to the acquisition data of each modality based on the consistency matrix of the acquisition data of each modality.

[0393] Optionally, the first determining module is specifically configured to:

[0394] obtain the mean and the variance of the acquisition data of each modality;

[0395] determine the consistency matrix of the acquisition data of each modality according to the acquisition data of each modality, the mean of the acquisition data of each modality, and the variance of the acquisition data of each modality.

[0396] Optionally, the first determining module is specifically configured to:

[0397] take the difference between the acquisition data of each modality and the mean as the difference of each modality;

[0398] take the ratio of the difference of each modality and the variance as the consistency matrix of the acquisition data of each modality.

[0399] Optionally, the first determining module is specifically configured to:

[0400] obtain the number of acquisition data types of each modality;

[0401] determine the covariance matrix corresponding to the acquisition data of each modality according to the number of acquisition data types of each modality and the consistency matrix of the acquisition data of each modality.

[0402] Optionally, the first determining module is specifically configured to:

[0403] take the product of the transpose of the consistency matrix of the acquisition data of each modality and the consistency matrix of the acquisition data of each modality as a first matrix;

[0404] take the ratio of the first matrix and the number of acquisition data types of each modality as the covariance matrix corresponding to the acquisition data of each modality.

[0405] Optionally, the first determining module is specifically configured to:

[0406] determine a vector index of the acquisition data of each modality according to the value of each dimension of the acquisition data of each modality and the dimension vector of the acquisition data of each modality;

[0407] determine a vector index of the target data of each modality in each group of target data according to the value of each dimension of the target data in each group of target data and the dimension vector of the target data in each group of target data;

[0408] determine the correlation coefficient of the acquisition data of each modality and the target data of each modality in each group of target data according to the vector index of the acquisition data of each modality and the vector index of the target data of each modality in each group of target data.

[0409] Optionally, the first determining module is specifically used for:

[0410] The ratio of the vector exponent of the acquired data of each modality to the vector exponent of the target data of each modality in each group of target data is used as the correlation coefficient between the acquired data of each modality and the target data of each modality in each group of target data.

[0411] Optionally, the second determining module is specifically used for:

[0412] Obtain the matrix proportion of each modality;

[0413] The weights of each mode are determined based on the matrix proportions of each mode.

[0414] Based on the correlation coefficients between the collected data of each modality and the target data of each modality in each group of target data, and the weights of each modality, a weighted sum is performed to obtain the fusion correlation coefficient between the multimodal data and each group of target data.

[0415] Optionally, the third determining module is specifically used for:

[0416] The state sensor encodes the multimodal data into a state feature vector;

[0417] The learner updates the parameters of the policy model based on the state feature vector and the reward function.

[0418] The action selector selects the target action corresponding to the multimodal data from the action space based on the updated policy model.

[0419] Optionally, the reward function is:

[0420] ;

[0421] in, Let A be the reward function, and A be the action space. For the reward vector, In order to take action The reward value obtained at that time Let s be the k-th action in the action space, and s be the current state. The next state after the action is performed. This represents the change in state.

[0422] Optional, multimodal data includes: image data, text data, and audio data.

[0423] Optional, also includes:

[0424] The instruction set collection acquisition module is configured to acquire an instruction set collection, wherein the instruction set collection comprises a plurality of versions of instruction sets corresponding to a plurality of hardware architectures;

[0425] The screening module is configured to screen the instruction set collection based on the overhead time of the current hardware architecture executing the respective versions of instruction sets to obtain a target instruction set.

[0426] The burning module is configured to burn the target instruction set to the current hardware architecture.

[0427] Optionally, the screening module is specifically configured to:

[0428] determine the mean value of the overhead time, wherein the overhead time is the overhead time of the current hardware architecture executing the respective versions of instruction sets;

[0429] the target version of instruction set with a ratio to the mean value of the overhead time less than or equal to a set threshold value is taken as the target instruction set.

[0430] Optionally, the overhead time of the current hardware architecture executing the respective versions of instruction sets is the time of the current hardware architecture executing the respective versions of instruction sets.

[0431] The product described above can execute the method provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0432] Embodiment Four

[0433] Figure 10 A structural schematic diagram of a target container determination apparatus provided by an embodiment of the application is shown. The embodiment can be applicable to the case of target container determination. The apparatus can be realized in the form of software and / or hardware. The apparatus can be integrated in any device providing the function of target container determination, such as a server. Figure 10 As shown in the figure, the target container determination apparatus specifically comprises a second acquisition module 1010, a target container determination module 1020 and a method execution module 1030.

[0434] The second acquisition module is configured to acquire performance parameters of each container in a container cluster, wherein the performance parameters comprise CPU utilization, resident memory occupancy and transmission bit rate occupancy.

[0435] The target container determination module is configured to take a container with at least one of the CPU utilization, the resident memory occupancy and the transmission bit rate occupancy greater than a corresponding threshold value as a target container.

[0436] The method execution module is configured to execute the data processing method described in the above embodiments based on the target container.

[0437] Optionally, the target container determination module is specifically configured to:

[0438] determine the container as an initial container if at least one of the CPU utilization ratio, the resident memory ratio and the transmission bit rate ratio is greater than a corresponding threshold value;

[0439] scale the initial container to obtain a target container.

[0440] Optionally, the target container determination module is specifically configured to:

[0441] obtain a target processing type, wherein the target processing type is a processing type corresponding to the data processing method in any of the embodiments;

[0442] determine a CPU utilization ratio threshold value, a resident memory ratio threshold value and a transmission bit rate ratio threshold value corresponding to the target processing type;

[0443] determine the container as a target container if at least one of the following conditions is met:

[0444] the CPU utilization ratio of the container is greater than the CPU utilization ratio threshold value;

[0445] the resident memory ratio of the container is greater than the resident memory ratio threshold value;

[0446] the transmission bit rate ratio of the container is greater than the transmission bit rate ratio threshold value.

[0447] Optionally, the method further comprises:

[0448] a third obtaining module configured to obtain a contraction number, a CPU utilization ratio contraction threshold value, a resident memory ratio contraction threshold value and a transmission bit rate ratio contraction threshold value;

[0449] a to-be-contracted container determination module configured to filter each container in a container cluster based on the contraction number, the CPU utilization ratio contraction threshold value, the resident memory ratio contraction threshold value and the transmission bit rate ratio contraction threshold value to obtain a to-be-contracted container;

[0450] a contraction module configured to contract the to-be-contracted container.

[0451] Optionally, the to-be-contracted container determination module is specifically configured to:

[0452] determine a container in the container cluster as a candidate container if the CPU utilization ratio of the container is less than the CPU utilization ratio contraction threshold value, the resident memory ratio of the container is less than the resident memory ratio contraction threshold value and the transmission bit rate ratio of the container is less than the transmission bit rate ratio contraction threshold value;

[0453] determine the candidate container corresponding to the contraction number as the to-be-contracted container.

[0454] Optionally, the target container determination module is specifically configured to:

[0455] acquire historical CPU utilization and historical resident memory occupancy of the initial container;

[0456] determine a CPU expansion threshold according to the historical CPU utilization of the initial container;

[0457] determine a resident memory expansion threshold according to the historical resident memory occupancy of the initial container;

[0458] acquire initial container ingress bandwidth and / or egress bandwidth;

[0459] expand the initial container according to the initial container ingress bandwidth and / or egress bandwidth, the CPU expansion threshold and the resident memory expansion threshold to obtain a target container.

[0460] Optionally, the target container determination module is specifically configured to:

[0461] acquire a core type corresponding to the target processing type, wherein the core type includes a lightweight core, a basic core and a real-time core;

[0462] filter containers in a container cluster based on the core type corresponding to the target processing type to obtain an initial container set;

[0463] take a container in the initial container set as a target container if at least one of CPU utilization, resident memory occupancy and transmission bit rate occupancy of the container is greater than a corresponding threshold.

[0464] Optionally, the method further comprises:

[0465] a resource type determination module corresponding to the target container, configured to determine a resource type corresponding to the target container according to the core type corresponding to the target processing type;

[0466] a resource allocation module, configured to allocate resources for the target container based on the resource type corresponding to the target container.

[0467] Optionally, the resource type determination module corresponding to the target container is specifically configured to:

[0468] if the core type corresponding to the target processing type is a lightweight core, the resource type corresponding to the target container is shared resources;

[0469] if the core type corresponding to the target processing type is a basic core, the resource type corresponding to the target container includes shared resources and protection resources;

[0470] if the core type corresponding to the target processing type is a real-time core, the resource type corresponding to the target container is protection resources.

[0471] The above product can perform the method provided by any embodiment of the application, has the corresponding function module and beneficial effects of performing the method.

[0472] Embodiment five

[0473] Figure 11 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the applications described and / or claimed in this document.

[0474] As shown in Figure 11 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected in communication with the at least one processor 11, where the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0475] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a speaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0476] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as a data processing method or a target container determination method.

[0477] In some embodiments, the data processing method or the target container determination method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the data processing method or the target container determination method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the data processing method or the target container determination method by any other suitable means, such as by means of firmware.

[0478] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0479] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a machine or a remote machine or a server.

[0480] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0481] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0482] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0483] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are a host product in the cloud computing service system to solve the defects of great management difficulty and weak business scalability in traditional physical hosts and VPS services.

[0484] It should be understood that the various forms of flow shown above can be reordered, additional steps added, or steps deleted. For example, the steps described in the present application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and the present application is not limited herein.

[0485] The embodiments of the present application further provide a computer program product comprising a computer program which, when executed by a processor, implements the data processing method or the target container determination method according to any of the embodiments of the present application.

[0486] The computer program product, in implementation, can be written in one or more programming languages or combinations thereof to implement computer program codes for performing the operations of the present application, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program codes can be executed entirely on a user computer, partially on a user computer, as an independent software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0487] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A data processing method, characterized in that, Performed by an embedded system, the method includes: Acquire multimodal data and target data sets, wherein the multimodal data includes: collected data from at least two modalities, and the target data set includes: multiple sets of target data and actions corresponding to each set of target data, and each set of target data includes: target data for each modality; Determine the correlation coefficient between the collected data of each modality and the target data of each modality in each set of target data; Based on the correlation coefficients between the acquired data of each modality and the target data of each modality in each group of target data, and the weights of each modality, the fusion correlation coefficient between the multimodal data and each group of target data is determined. Based on the correlation coefficient between the multimodal data and each set of target data, the target action corresponding to the multimodal data is determined. Execute the target action corresponding to the multimodal data; Based on the fusion correlation coefficient between multimodal data and each set of target data, the target action corresponding to the multimodal data is determined, including: if there is a set of target data whose fusion correlation coefficient with the multimodal data is greater than the correlation coefficient threshold, then the action corresponding to that set of target data is taken as the target action corresponding to the multimodal data; if there are multiple sets of target data whose fusion correlation coefficient with the multimodal data is greater than the correlation coefficient threshold, then the action corresponding to the set of target data with the largest fusion correlation coefficient with the multimodal data in the target data set is taken as the target action corresponding to the multimodal data; if the fusion correlation coefficient between the multimodal data and each set of target data is less than or equal to the correlation coefficient threshold, then the multimodal data is input into the target model to obtain the target action corresponding to the multimodal data, wherein the target model includes, from input to output, a state perceiver, a learner, and an action selector in sequence; Multimodal data includes image data, text data, and audio data.

2. The method according to claim 1, characterized in that, Determine the correlation coefficient between the acquired data of each modality and the target data of each modality in each set of target data, including: Determine the values ​​and dimension vectors of each dimension of the acquired data for each modality; Determine the values ​​and dimension vectors of each dimension of the target data in each set of target data; Based on the values ​​of each dimension of the collected data for each modality, the dimension vector of the collected data for each modality, the values ​​of each dimension of the target data in each group of target data, and the dimension vector of the target data in each group of target data, the correlation coefficient between the collected data for each modality and the target data of each modality in each group of target data is determined.

3. The method according to claim 2, characterized in that, Determine the values ​​and dimension vectors of each dimension of the acquired data for each modality, including: Determine the covariance matrix corresponding to the acquired data for each mode; Based on the covariance matrix corresponding to the acquired data of each modality, determine the values ​​and dimension vectors of each dimension of the acquired data of each modality.

4. The method according to claim 3, characterized in that, Determine the covariance matrix corresponding to the acquired data for each mode, including: The collected data for each modality are processed to obtain the consistency matrix of the collected data for each modality; Based on the consistency matrix of the acquired data for each modality, the covariance matrix corresponding to the acquired data for each modality is determined.

5. The method according to claim 4, characterized in that, The acquired data for each modality is processed to obtain the consistency matrix of the acquired data for each modality, including: Obtain the mean and variance of the collected data for each modality; Based on the collected data of each modality, the mean of the collected data of each modality, and the variance of the collected data of each modality, the consistency matrix of the collected data of each modality is determined.

6. The method according to claim 5, characterized in that, Based on the collected data for each modality, the mean of the collected data for each modality, and the variance of the collected data for each modality, a consistency matrix for the collected data for each modality is determined, including: The difference between the collected data of each modality and the mean is taken as the difference between each modality; The ratio of the difference to the variance of each modality is used as the consistency matrix of the collected data for each modality.

7. The method according to claim 4, characterized in that, Based on the consistency matrix of the acquired data for each modality, the covariance matrix corresponding to the acquired data for each modality is determined, including: Obtain the number of data types acquired for each modality; Based on the number of data types collected in each modality and the consistency matrix of the collected data in each modality, the covariance matrix corresponding to the collected data in each modality is determined.

8. The method according to claim 7, characterized in that, Based on the number of data types collected in each modality and the consistency matrix of the collected data in each modality, the covariance matrix corresponding to the collected data in each modality is determined, including: The first matrix is ​​the product of the transpose of the consistency matrix of the acquired data for each modality and the consistency matrix of the acquired data for each modality. The ratio of the first matrix to the number of data types acquired in each modality is used as the covariance matrix corresponding to the acquired data of each modality.

9. The method according to claim 2, characterized in that, Based on the values ​​of each dimension of the acquired data for each modality, the dimension vector of the acquired data for each modality, the values ​​of each dimension of the target data in each set of target data, and the dimension vector of the target data in each set of target data, the correlation coefficient between the acquired data for each modality and the target data for each modality in each set of target data is determined, including: Based on the values ​​of each dimension of the acquired data for each modality and the dimension vector of the acquired data for each modality, determine the vector exponent of the acquired data for each modality; Based on the values ​​of each dimension of the target data in each set of target data and the dimension vector of the target data in each set of target data, determine the vector exponent of the target data for each modality in each set of target data; Based on the vector exponents of the acquired data for each modality and the vector exponents of the target data for each modality in each set of target data, the correlation coefficient between the acquired data for each modality and the target data for each modality in each set of target data is determined.

10. The method according to claim 9, characterized in that, Based on the vector exponents of the acquired data for each modality and the vector exponents of the target data for each modality in each set of target data, the correlation coefficient between the acquired data for each modality and the target data for each modality in each set of target data is determined, including: The ratio of the vector exponent of the acquired data of each modality to the vector exponent of the target data of each modality in each group of target data is used as the correlation coefficient between the acquired data of each modality and the target data of each modality in each group of target data.

11. The method according to claim 1, characterized in that, Based on the correlation coefficients between the acquired data of each modality and the target data of each modality in each set of target data, and the weights of each modality, the fusion correlation coefficients between the multimodal data and each set of target data are determined, including: Obtain the matrix proportion of each modality; The weights of each mode are determined based on the matrix proportions of each mode. Based on the correlation coefficients between the collected data of each modality and the target data of each modality in each group of target data, and the weights of each modality, a weighted sum is performed to obtain the fusion correlation coefficient between the multimodal data and each group of target data.

12. The method according to claim 1, characterized in that, The multimodal data is input into the target model to obtain the target action corresponding to the multimodal data, including: The state sensor encodes the multimodal data into a state feature vector; The learner updates the parameters of the policy model based on the state feature vector and the reward function. The action selector selects the target action corresponding to the multimodal data from the action space based on the updated policy model.

13. The method according to claim 12, characterized in that, The reward function is: ； in, Let A be the reward function, and A be the action space. For the reward vector, In order to take action The reward value obtained at that time Let s be the k-th action in the action space, and s be the current state. The next state after the action is performed. This represents the change in state.

14. The method according to claim 1, characterized in that, Also includes: Obtain an instruction set set, wherein the instruction set set includes: multiple versions of instruction sets corresponding to multiple hardware architectures; Based on the overhead time of executing the corresponding instruction sets of the current hardware architecture, the instruction set set is filtered to obtain the target instruction set; The target instruction set is burned into the current hardware architecture.

15. The method according to claim 14, characterized in that, Based on the overhead and execution time of various versions of the instruction set corresponding to the current hardware architecture, the instruction set set is filtered to obtain the target instruction set, including: Determine the average overhead time, where the overhead time is the overhead time for the current hardware architecture to execute the corresponding version of the instruction set; The instruction set of the target version whose ratio to the average overhead time is less than or equal to a set threshold is taken as the target instruction set.

16. The method according to claim 15, characterized in that, The overhead time for the current hardware architecture to execute the corresponding version of the instruction set is the time required for the current hardware architecture to execute the corresponding version of the instruction set.

17. A method for determining a target container, characterized in that, The method includes: Obtain the performance parameters of each container in the container cluster, wherein the performance parameters include: CPU utilization, resident memory percentage, and transmission bit rate percentage; Containers whose CPU utilization, resident memory percentage, and transmission bit rate percentage are all greater than their corresponding thresholds are selected as target containers. The data processing method of any one of claims 1-16 is performed based on the target container.

18. The method according to claim 17, characterized in that, Containers whose CPU utilization, resident memory percentage, and transfer bit rate percentage are at least greater than their corresponding thresholds are designated as target containers, including: Containers whose CPU utilization, resident memory percentage, and transmission bit rate percentage are greater than their corresponding thresholds are used as initial containers. The initial container is expanded to obtain the target container.

19. The method according to claim 17, characterized in that, Containers whose CPU utilization, resident memory percentage, and transfer bit rate percentage are at least greater than their corresponding thresholds are designated as target containers, including: Obtain the target processing type, wherein the target processing type is the processing type corresponding to the data processing method of any one of claims 1-16; Determine the CPU utilization threshold, resident memory percentage threshold, and transmission bit rate percentage threshold corresponding to the target processing type; Containers that satisfy at least one of the following conditions will be used as target containers: The container's CPU utilization is greater than the CPU utilization threshold; The container's resident memory percentage is greater than the resident memory percentage threshold; The percentage of the container's transmission bit rate is greater than the threshold for the percentage of transmission bit rate.

20. The method according to claim 17, characterized in that, Also includes: Obtain the shrinkage quantity, CPU utilization shrinkage threshold, resident memory percentage shrinkage threshold, and transmission bit rate percentage shrinkage threshold; The containers in the container cluster are screened based on the number of containers to be shrunk, the CPU utilization shrinkage threshold, the resident memory ratio shrinkage threshold, and the transmission bit rate ratio shrinkage threshold to obtain the containers to be shrunk. Shrink the container to be shrunk.

21. The method according to claim 20, characterized in that, Based on the shrinking quantity, CPU utilization shrinking threshold, resident memory percentage shrinking threshold, and transmission bit rate percentage shrinking threshold, containers in the container cluster are screened to obtain the containers to be shrunk, including: Containers in the container cluster with CPU utilization lower than the CPU utilization shrinkage threshold, resident memory percentage lower than the resident memory percentage shrinkage threshold, and transmission bit rate percentage lower than the transmission bit rate shrinkage threshold are selected as candidate containers. The candidate container corresponding to the shrinkage quantity is selected as the container to be shrunk.

22. The method according to claim 18, characterized in that, The initial container is expanded to obtain the target container, which includes: Obtain the historical CPU utilization and historical resident memory percentage of the initial container; The CPU expansion threshold is determined based on the historical CPU utilization of the initial container; The resident memory expansion threshold is determined based on the historical resident memory percentage of the initial container. Obtain the initial container ingress bandwidth and / or egress bandwidth; Based on the initial container ingress bandwidth and / or egress bandwidth, CPU expansion threshold, and resident memory expansion threshold, the initial container is expanded to obtain the target container.

23. The method according to claim 19, characterized in that, Containers whose CPU utilization, resident memory percentage, and transfer bit rate percentage are at least greater than their corresponding thresholds are designated as target containers, including: Obtain the core type corresponding to the target processing type, wherein the core type includes: lightweight core, basic core, and real-time core; Based on the kernel type corresponding to the target processing type, the containers in the container cluster are filtered to obtain an initial container set; Containers in the initial container set whose CPU utilization, resident memory percentage, and transmission bit rate percentage are greater than their corresponding thresholds are selected as target containers.

24. The method according to claim 23, characterized in that, Also includes: Based on the kernel type corresponding to the target processing type, determine the resource type corresponding to the target container; Based on the resource type corresponding to the target container, resources are allocated to the target container.

25. The method according to claim 24, characterized in that, Based on the kernel type corresponding to the target processing type, the resource type corresponding to the target container is determined, including: If the kernel type corresponding to the target processing type is a lightweight kernel, then the resource type corresponding to the target container is a shared resource; If the kernel type corresponding to the target processing type is a basic kernel, then the resource type corresponding to the target container includes shared resources and protected resources; If the kernel type corresponding to the target processing type is a real-time kernel, then the resource type corresponding to the target container is a protected resource.

26. A data processing apparatus, characterized in that, Configured in an embedded system, the device includes: The first acquisition module is used to acquire multimodal data and a target data set, wherein the multimodal data includes: acquisition data of at least two modalities, and the target data set includes: multiple sets of target data and actions corresponding to each set of target data, and each set of target data includes: target data of each modality; The first determining module is used to determine the correlation coefficient between the collected data of each modality and the target data of each modality in each group of target data; The second determining module is used to determine the fusion correlation coefficient between the multimodal data and each set of target data based on the correlation coefficient between the acquired data of each modality and the target data of each modality in each set of target data and the weight of each modality. The third determination module is used to determine the target action corresponding to the multimodal data based on the fusion correlation coefficient of the multimodal data and each set of target data; The action execution module is used to execute the target action corresponding to the multimodal data; The third determining module is specifically used for: If there exists a set of target data whose fusion correlation coefficient with the multimodal data is greater than the correlation coefficient threshold, then the action corresponding to that set of target data is taken as the target action corresponding to the multimodal data. If there exist multiple sets of target data whose fusion correlation coefficient with the multimodal data is greater than the correlation coefficient threshold, then the action corresponding to the set of target data with the largest fusion correlation coefficient with the multimodal data in the target data set is taken as the target action corresponding to the multimodal data. If the fusion correlation coefficients of the multimodal data and each set of target data are less than or equal to the correlation coefficient threshold, then the multimodal data is input into the target model to obtain the target action corresponding to the multimodal data. The target model, from input to output, includes: a state perceiver, a learner, and an action selector. Multimodal data includes image data, text data, and audio data.

27. A target container determining device, characterized in that, The device includes: The second acquisition module is used to acquire the performance parameters of each container in the container cluster, wherein the performance parameters include: CPU utilization, resident memory percentage and transmission bit rate percentage. The target container determination module is used to identify containers whose CPU utilization, resident memory percentage, and transmission bit rate percentage are all greater than their corresponding thresholds as target containers. A method execution module is configured to execute the data processing method of any one of claims 1-16 based on the target container.

28. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data processing method of any one of claims 1-16, or the target container determination method of any one of claims 17-25.

29. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data processing method of any one of claims 1-16, or the target container determination method of any one of claims 17-25.

30. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data processing method according to any one of claims 1-16, or the target container determination method according to any one of claims 17-25.

Images