Multilevel feature multicast transmission method, device, equipment, storage medium and product

Abstract

The application discloses a multi-level feature multicast transmission method and device, equipment, storage medium and product, and relates to the technical field of semantic communication. The method comprises the following steps: acquiring multi-level features according to a target picture, the multi-level features comprising semantic features; determining semantic errors of different levels according to the semantic features; and determining the communication resources of data receiving equipment in the current level according to target features of the target picture. The receiving capabilities of different users are acquired, so that reasonable resource allocation can be performed when subsequent semantic communication is performed for multiple users. By establishing an optimization problem based on the communication resources of the data receiving equipment of different levels and the semantic errors of different levels, an optimal solution is obtained. Furthermore, the target features are multicast transmitted based on the optimal solution, the communication resource overhead in the multi-user scenario is reduced, the requirements of different mobile agents are met in information transmission, and the transmission delay and resource consumption are reduced as a whole.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, and particularly relates to a multi-level feature multicast transmission method, device, equipment, storage medium and product. BACKGROUND

[0002] With the rapid development of mobile communication and Internet technology, the demand for high-speed and low-latency wireless access has increased dramatically, and the traditional communication system has approached the Shannon limit. In the future 6G typical intelligent scene, there is a remote zero-sample semantic understanding problem. For example, in the automatic driving scene, the unmanned vehicle often needs to rely on wireless communication with the remote vehicle or base station to obtain the remote traffic situation in advance, so as to optimize the route planning. However, there are often new conditions such as emergency accidents and new vehicles on the road in road traffic. These scene pictures often cannot be obtained in advance and are invisible in the training process, which belong to unknown categories, that is, the automatic vehicle faces the remote zero-sample image recognition problem.

[0003] At present, when the server transmits features to multiple mobile agents, the communication resource overhead is large.

[0004] Therefore, how to reduce the communication resource overhead in the multi-user scenario is a problem to be solved at present. SUMMARY

[0005] The main purpose of the present application is to provide a multi-level feature multicast transmission method, device, equipment, storage medium and product, which aims to solve the technical problem of large communication resource overhead in the multi-user scenario.

[0006] To achieve the above purpose, the present application provides a multi-level feature multicast transmission method, which comprises:

[0007] Obtaining multi-level features according to a target picture, wherein the multi-level features comprise semantic features;

[0008] Determining semantic errors of different levels according to the semantic features, and determining the communication resources of the data receiving equipment in the current level according to the target features of the target picture;

[0009] Based on the communication resources of the data receiving equipment of different levels and the semantic errors of different levels, an optimization problem is established to obtain an optimal solution;

[0010] Based on the optimal solution, the target features are multicast transmitted.

[0011] In an embodiment, the step of establishing an optimization problem based on the semantic errors of different levels and the communication resources of the data receiving equipment of different levels to obtain an optimal solution comprises:

[0012] determining semantic loss corresponding to different levels according to the semantic error of different levels;

[0013] determining multicast delay corresponding to different levels according to the communication resource of data receiving device in different levels;

[0014] establishing an optimization problem according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels, and obtaining an optimal solution.

[0015] In an embodiment, the step of establishing an optimization problem according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels, and obtaining an optimal solution, comprises:

[0016] establishing a multi-level multicast selection strategy, a bandwidth allocation strategy and a power allocation strategy according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels;

[0017] minimizing semantic loss and average maximum multicast delay by optimizing the multi-level feature multicast strategy, the multicast bandwidth allocation strategy and the power allocation strategy, and obtaining an optimal solution.

[0018] In an embodiment, the step of minimizing semantic loss and average maximum multicast delay by optimizing the multi-level feature multicast strategy, the multicast bandwidth allocation strategy and the power allocation strategy, and obtaining an optimal solution, comprises:

[0019] jointly optimizing a corresponding bandwidth allocation strategy and a corresponding power allocation strategy under any given multicast grouping strategy, to obtain an optimal bandwidth allocation strategy and an optimal power allocation strategy;

[0020] obtaining a multicast grouping optimal strategy based on the optimal bandwidth allocation strategy and the optimal power allocation strategy.

[0021] In an embodiment, the step of obtaining a multicast grouping optimal strategy based on the optimal bandwidth allocation strategy and the optimal power allocation strategy, comprises:

[0022] generating all multicast grouping strategies based on the optimal bandwidth allocation strategy and the optimal power allocation strategy;

[0023] searching for a minimum value of semantic loss and average maximum multicast delay in all multicast grouping strategies, and the multicast grouping strategy corresponding to the minimum value is the multicast grouping optimal strategy.

[0024] In an embodiment, the multi-level features include visual features, latent low-dimensional intermediate features and semantic features, and the step of obtaining multi-level features according to a target image comprises:

[0025] extract visual features of the target picture, and project the visual features into a latent low-dimensional intermediate space to obtain latent low-dimensional intermediate features;

[0026] project the latent low-dimensional intermediate features into a semantic feature space based on a multi-level semantic encoder to obtain the semantic features.

[0027] In addition, to achieve the above object, the present application further provides a multi-level feature multicast transmission device, which comprises:

[0028] a feature acquisition module configured to acquire multi-level features from a target picture, the multi-level features comprising semantic features;

[0029] a transmission resource module configured to determine semantic errors of different levels according to the semantic features, and determine communication resources of data receiving devices in a current level according to target features of the target picture;

[0030] a transmission optimization module configured to establish an optimization problem based on the semantic errors of different levels and the communication resources of data receiving devices in different levels, and obtain an optimal solution;

[0031] a multicast transmission module configured to perform multicast transmission on the target features based on the optimal solution.

[0032] In addition, to achieve the above object, the present application further provides a multi-level feature multicast transmission device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-level feature multicast transmission method as described above.

[0033] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and has a computer program stored thereon, the computer program being executable by a processor to implement the steps of the multi-level feature multicast transmission method as described above.

[0034] In addition, to achieve the above object, the present application further provides a computer program product, which comprises a computer program, the computer program being executable by a processor to implement the steps of the multi-level feature multicast transmission method as described above.

[0035] The one or more technical solutions provided by the present application have at least the following technical effects:

[0036] According to the target picture, multi-level features are obtained, the multi-level features include semantic features, by determining semantic errors of different levels according to the semantic features, and determining the communication resources of the data receiving device in the current level according to the target features of the target picture, the receiving capabilities of different users are obtained, which is convenient for subsequent reasonable resource allocation when facing multi-user semantic communication. By establishing an optimization problem based on the communication resources of the data receiving device of different levels and the semantic errors of different levels, the optimal solution is obtained, further, based on the optimal solution, the target features are multicast transmitted, which reduces the communication resource overhead in the multi-user scenario, meets the needs of different mobile agents in information transmission, and reduces the transmission delay and resource consumption as a whole, which provides strong technical support for realizing intelligent and simple communication. BRIEF DESCRIPTION OF DRAWINGS

[0037] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor.

[0039] Figure 1 The MEC system schematic diagram of the embodiments of the present application;

[0040] Figure 2 The flowchart of the first embodiment of the multi-level feature multicast transmission method of the present application;

[0041] Figure 3 The flowchart of the second embodiment of the multi-level feature multicast transmission method of the present application;

[0042] Figure 4 The multi-level feature multistage feature extractor construction schematic diagram of the embodiments of the present application;

[0043] Figure 5 The module structure schematic diagram of the multi-level feature multicast transmission device of the embodiments of the present application;

[0044] Figure 6 The device structure schematic diagram of the hardware running environment involved in the multi-level feature multicast transmission method in the embodiments of the present application.

[0045] The purpose implementation, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0047] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0048] The present application proposes a new edge server (Multi-Access Edge Computing, MEC) system, as shown in Figure 1 The MEC system is composed of a single MEC server and multiple mobile agents, each agent is endowed with a single multi-level feature extractor and a semantic knowledge base (Semantic Knowledge Base, SKB). The is represented as the number of mobile agents, and is represented as a set of mobile agents. The MEC server and the mobile agent both have a semantic knowledge base (SKB) and a multi-level semantic codec. The SKB is defined as a set of semantic vectors corresponding to some image categories, and the multi-level semantic codec describes the mapping relationship between the dimension size, classification accuracy, and semantic loss of different levels.

[0049] Taking image transmission as an example, assuming that the MEC server perceives a certain image sample, sends it to the mobile agent, and the mobile agent needs to identify the specific semantic description of the image, which is called a remote image recognition task. Since the MEC server and different mobile agents may have different resource conditions such as the size of the semantic knowledge base, it is impossible to achieve that the information sent by the MEC to the mobile agent each time is the estimated category information with extremely high compression rate. Therefore, research on multi-level feature extraction, transmission and resource allocation for multi-user transmission is carried out. The present application mainly focuses on the downlink transmission in the multi-user scenario.

[0050] Based on this, the present application embodiment provides a multi-level feature multicast transmission method, referring to Figure 2 , Figure 2 is a flowchart of the first embodiment of the multi-level feature multicast transmission method of the present application.

[0051] In this embodiment, the multi-level feature multicast transmission method comprises steps S10-S40:

[0052] Step S10, obtaining multi-level features according to a target picture.

[0053] It should be noted that the target picture can be understood as a picture input into the MEC system, and the multi-level features can be understood as the characteristics exhibited by the picture at different processing stages or different abstraction levels. For example, a multi-feature extractor can be used to obtain the multi-level features of the target picture.

[0054] Step S20, determine the semantic error of different levels according to the multi-level features, and determine the communication resources of the data receiving device in the current level according to the target features of the target picture.

[0055] It should be noted that the semantic error can be understood as the loss of picture information due to compression, noise and other factors during transmission or processing, especially the loss of key information affecting the understanding of picture content, and the data receiving device can be a plurality of mobile intelligent agents, and the communication resources of the data receiving device can be the network resources required during data transmission, including but not limited to bandwidth, delay, transmission power, etc.

[0056] Step S30, based on the communication resources of the data receiving device of different levels and the semantic error of different levels, an optimization problem is established to obtain an optimal solution.

[0057] For example, the communication resources of the data receiving device of different levels can be optimized under the condition of ensuring the minimum semantic error, and a corresponding target function is established to make the target function reach the minimum (or maximum) solution.

[0058] Step S40, multicast transmission based on the optimal solution to the target features.

[0059] It should be noted that the target features, i.e. certain features of the image samples finally determined to be sent at the MEC, are at least one of the estimated categories of the target picture, the semantic vector corresponding to the minimum semantic error in the first semantic knowledge base, the semantic features of the target picture, the latent low-dimensional intermediate features, and the visual features. Multicast transmission is a communication method for efficient data transmission in computer networks, which allows a sender to send data to a specific group of receivers simultaneously.

[0060] In this embodiment, multi-level features are obtained according to the target picture, which include semantic features. By determining the semantic error of different levels according to the semantic features, and determining the communication resources of the data receiving device in the current level according to the target features of the target picture, the receiving ability of different users is obtained, which facilitates reasonable resource allocation when subsequent semantic communication is performed for multiple users. By establishing an optimization problem based on the communication resources of the data receiving device of different levels and the semantic error of different levels, an optimal solution is obtained, and further, multicast transmission is performed based on the optimal solution to the target features, which reduces the communication resource overhead in the multi-user scenario. In information transmission, the needs of different mobile agents are met, and the transmission delay and resource consumption are reduced as a whole, which provides strong technical support for realizing intelligent communication.

[0061] Reference Figure 3 , Figure 3 is a flowchart of the second embodiment of the multi-level feature multicast transmission method of the present application, based on the aboveFigure 2 The first embodiment is shown, and a second embodiment of the multi-level feature multicast transmission method of the present application is proposed.

[0062] In the second embodiment, the step S10 comprises:

[0063] Step S101, extracting the visual features of the target picture, and projecting the visual features into a latent low-dimensional intermediate space to obtain latent low-dimensional intermediate features.

[0064] It should be noted that the visual features can be attributes automatically extracted from the picture that can describe the content of the picture, such as the color, texture, shape, edge, etc. of the picture. The latent low-dimensional intermediate space can be a mathematical space with a lower dimension than the original data space, used to store the feature representation after dimension reduction. The latent low-dimensional intermediate features can be understood as the feature representation obtained after dimension reduction, which encodes the key information of the original picture in a lower dimension.

[0065] Step S102, projecting the latent low-dimensional intermediate features into a semantic feature space based on a multi-level semantic encoder to obtain semantic features.

[0066] It should be noted that the multi-level semantic encoder can be understood as a table model structure, which usually contains multiple levels or layers, and each level is responsible for extracting semantic information of different levels. The semantic feature space is used for further processing of the latent low-dimensional intermediate feature information. The semantic features focus more on describing the abstract and high-level meaning of the picture content, such as the category, attribute, relationship, etc. of the object.

[0067] Based on the visual and semantic auto-encoder (AE), given any image sample The multi-level feature extractor used is as follows.

[0068] ① Level 1, visual features:

[0069] A pre-trained large-scale CNN (e.g. GoogleNet, ResNet, etc.) is used to project the input image sample into visual features, i.e. where denotes the pre-trained large-scale CNN, denotes the dimension of the visual features.

[0070] ② Level 2, latent low-dimensional intermediate features:

[0071] The visual encoder projects the visual features into the space of the latent low-dimensional intermediate features, i.e. where denotes the visual encoder, denotes the dimension of the intermediate features.

[0072] ③ Level 3, semantic features:

[0073] The semantic decoder projects the latent low-dimensional intermediate feature into the space of semantic features, i.e., where denotes the semantic decoder, denotes the dimension of the semantic feature.

[0074] ④ Level 4, estimate the class label:

[0075] The class of a sample is estimated by measuring the semantic similarity of the projected semantic representation to the class prototypes in the SKB, i.e., where denotes the set of classes (class samples seen during training), denotes the semantic vector of class .

[0076] In this embodiment, by extracting the visual features of the target picture, the visual information of the target picture can be obtained, by projecting the visual features into a latent low-dimensional intermediate space, the dimension of the feature vector is reduced, the calculation efficiency is improved, and a latent low-dimensional intermediate feature is obtained. Further, based on the multi-level semantic encoder, the latent low-dimensional intermediate feature is projected into the semantic feature space to obtain the semantic feature, which can capture and encode the complex semantic information in the picture, so that the picture content can be more accurately understood and analyzed, thereby providing more personalized and intelligent services for users.

[0077] Referring to Figure 4 , Figure 4 is a flowchart of a third embodiment of the multi-level feature multicast transmission method of the present application, based on the second embodiment shown in Figure 3 , the third embodiment of the multi-level feature multicast transmission method of the present application is proposed.

[0078] In the third embodiment, the step S30 comprises:

[0079] Step S301, according to the semantic error of different levels to determine the corresponding semantic loss of different levels.

[0080] For example, let denote whether the k th user is in the l th multicast group.

[0081] For the first level, the visual feature multicast, the semantic loss can be represented as:

[0082] When , the visual feature vector of the sample n (represented as ) is multicast from the MEC server to the set the mobile agents in the set i.e., all the agents using level 1 transmission. Then, each agent estimates the class based on its multi-level feature extractor and its SKB, denoted as equation (1):

[0083]

[0084] In equation (1), denotes the class estimated by each agent in level 1 based on its multi-level feature extractor and its SKB, denotes the knowledge base of the k th user, denotes the semantic knowledge vector corresponding to the c th class, denotes the semantic decoder of the k th user, denotes the visual encoder of the k th user, denotes the visual feature vector of the n th sample, F denotes the Frobenius norm, n denotes the n th sample, k denotes the k th user.

[0085] The agent The corresponding semantic loss is denoted as equation (2):

[0086]

[0087] where, denotes the semantic loss of the agent The maximum semantic loss occurring among the mobile agents in the set is denoted as equation (3)

[0088]

[0089] where, denotes the maximum semantic loss occurring among the mobile agents in the set For level 2, latent low-dimensional intermediate feature multicasting, the semantic loss can be denoted as:

[0090] When the intermediate feature of the sample denoted as is multicasted from the MEC server to the mobile agents within the set at this time i.e., all the agents using level 2 transmission. Then, each agent The class representation is estimated based on its semantic decoder as formula (4):

[0091]

[0092] In formula (4), represents the class estimated by the kth agent based on its multi-level feature extractor and its SKB at level 2, represents the knowledge base of the kth user, k represents the semantic knowledge vector corresponding to the kth class, represents the semantic decoder of the kth user, c represents the visual encoder of the kth user, represents the visual feature vector of the kth sample, k represents the Frobenius norm, represents the kth sample, represents the kth user. n F The corresponding semantic loss is represented as formula (5): n n k k wherein,

[0093] represents the semantic loss of the agent The maximum semantic loss occurring in the mobile agents in the set

[0094] is represented as formula (6):

[0095] wherein, represents the maximum semantic loss occurring in the mobile agents in the set

[0096]

[0097] wherein, represents the maximum semantic loss occurring in the mobile agents in the set

[0098] For the 3rd level, the extracted semantic features (directly obtained from the image) are multicast, and the semantic loss can be represented as:

[0099] When , the semantic feature vector of the sample (i.e., ) is multicast from the MEC server to the mobile agents in the set . Then, each agent estimates the class based on its SKB as formula (7):

[0100] ​​​​​

[0101] In equation (7), This indicates that each agent at level 3 belongs to a category based on its multi-level feature extractor and its SKB estimate. Indicates the first k A user's knowledge base Indicates the corresponding number c Semantic knowledge vectors for each category, Indicates the corresponding number m The semantic knowledge vectors of each category are represented by the semantic loss expressed in formula (8):

[0102]

[0103] in, Indicates agent The corresponding semantic loss, Indicates the corresponding number c Semantic knowledge vectors for each category, Indicates the first M Semantic decoder for each user Indicates the first M A user's visual encoder Indicates the first n The visual feature vector of each sample, F Describe the Frobenius norm. n Indicates the first n One sample, M Indicates the first M One user.

[0104] For Level 4, the semantic loss of the estimated semantic features (the semantic features with the highest similarity to the direct semantic features of the image found from the sender's SKB) multicast can be expressed as:

[0105] when At that time, the MEC server first estimates the category of the image sample, i.e., formula (9):

[0106]

[0107] In equation (9), This indicates that each agent at level 4 is classified based on its multi-level feature extractor and its SKB estimate, and then the corresponding semantic vector is generated. Multicast to Collection The mobile agent within. The corresponding semantic loss is given by formula (10):

[0108]

[0109] in, Indicates agent The corresponding semantic loss, in the set The maximum semantic loss occurring in the mobile agent within the scope is due to Provided.

[0110] For level 5, estimating category label multicast, the semantic loss can be expressed as:

[0111] when At that time, the MEC server first estimates the category of the image sample, i.e., formula (9), and then calculates the estimated category label. Multicast to Collection The mobile agent within the system. To ensure that the corresponding semantic features are stored in the mobile agent's SKB, the following must be satisfied: , The corresponding semantic loss is the same as that of level 4, expressed as formula (11):

[0112]

[0113] in, Indicates agent The corresponding semantic loss, in the set The maximum semantic loss occurring in the mobile agent within the scope is due to Provided.

[0114] Step S302: Determine the multicast delay corresponding to different levels based on the communication resources of the data receiving devices in different levels.

[0115] For example, based on step S301, the multicast delays corresponding to different levels are represented as follows. The multicast delay caused by Level 1 transmission (represented as...) The formula for calculating the multicast delay of Level 1 is expressed as formula (12):

[0116]

[0117] in, This indicates Level 1 multicast latency. Represents the dimensions of visual features. Q This represents the number of quantization bits per dimension of data, and the transmission rate. Represented as formula (13):

[0118]

[0119] In formula (13), This indicates the first-level transmission rate. This indicates the allocation of multicast bandwidth at layer l. This indicates the allocation of multicast power for the l-th layer. This indicates the connection from the MEC server to the proxy. Channel power gain, denotes the noise power spectral density.

[0120] denotes the multicast delay caused by the 2nd level transmission (denoted as ), the 2nd level multicast delay calculation formula is denoted as formula (14):

[0121]

[0122] wherein, denotes the 2nd level multicast delay, denotes the potential feature dimension, Q denotes the number of quantization bits for each dimension of data, the transmission rate is denoted as formula (15):

[0123]

[0124] wherein, denotes the 3rd level transmission rate, denotes the allocated 2nd layer multicast bandwidth, denotes the allocated 2nd layer multicast power, the multicast delay caused by the 3rd level transmission (denoted as ), the 3rd level multicast delay calculation formula is denoted as formula (16):

[0125]

[0126] in formula (16), denotes the 3rd level multicast delay, denotes the semantic feature dimension, the transmission rate is denoted as formula (17):

[0127]

[0128] wherein, denotes the 4th level transmission rate, denotes the allocated 3rd layer multicast bandwidth, denotes the allocated 3rd layer multicast power, the multicast delay caused by the 4th level transmission (denoted as ), the 4th level multicast delay calculation formula is denoted as formula (18):

[0129]

[0130] in formula (18), denotes the 4th level multicast delay, the transmission rate is denoted as formula (19):

[0131]

[0132] Multicast delay caused by the 5th level transmission (denoted as ), the 5th level multicast delay calculation formula is denoted as formula (20):

[0133]

[0134] In formula (20), denotes the 5th level multicast delay, the transmission rate is denoted as formula (21):

[0135]

[0136] wherein, denotes the 5th level transmission rate, denotes the allocated 5th layer multicast bandwidth, denotes the allocated 5th layer multicast power, denotes the channel power gain from the MEC server to the proxy , and denotes the noise power spectral density.

[0137] Step S303, according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels, an optimization problem is established, and an optimal solution is obtained.

[0138] It should be noted that step S303 includes: according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels, a multi-level multicast selection strategy, a bandwidth allocation strategy and a power allocation strategy are established; by optimizing the multi-level feature multicast strategy, the multicast bandwidth allocation strategy, and the power allocation strategy, the semantic loss and the average maximum multicast delay are minimized, and an optimal solution is obtained.

[0139] For example, according to steps S301 and S302, the average semantic loss of the entire MEC system is denoted as formula (22):

[0140]

[0141] In formula (22), N denotes the number of samples, denotes the semantic error caused by the transmission of the i-th picture sample of the i-th user to the i-th level feature, k denotes the indication variable whether the i-th user is in the multicast user group of the i-th level, if n , it indicates that the i-th user is in the multicast user group of the i-th level, otherwise, l k l k l ​​​​​​​The average maximum multicast delay consumption is expressed as equation (23):

[0142]

[0143] In equation (23), represents the feature dimension, Q represents the number of quantization bits for each dimension of data, is expressed as the downlink rate, specifically expressed as equation (24):

[0144]

[0145] wherein, represents the allocation of the first l layer multicast bandwidth, represents the allocation of the first l layer multicast power, represents the channel gain of the k user, is the noise power spectral density.

[0146] In equation (23), , , , and .

[0147] The multi-layer feature multicast strategy can be expressed as equation (25):

[0148]

[0149] In equation (25), represents the multi-layer feature multicast strategy, the multicast bandwidth allocation strategy can be expressed as equation (26):

[0150]

[0151] In equation (26), represents the multicast bandwidth allocation strategy, the power allocation strategy can be expressed as equation (27):

[0152]

[0153] In equation (27), represents the power allocation strategy, on this basis, the semantic loss and the average maximum multicast delay are minimized.

[0154] It should be noted that the step of obtaining the optimal solution by optimizing the multi-level feature multicast strategy, the multicast bandwidth allocation strategy, and the power allocation strategy to minimize the semantic loss and the average maximum multicast delay includes: under any given multicast grouping strategy, jointly optimizing the corresponding bandwidth allocation strategy and the corresponding power allocation strategy to obtain the optimal bandwidth allocation strategy and the optimal power allocation strategy; and obtaining the optimal multicast grouping strategy based on the optimal bandwidth allocation strategy and the optimal power allocation strategy.

[0155] For example, the optimization problem can be modeled as P1, which can be represented by formulas (28) to (29).

[0156]

[0157] The constraint condition of P1 is represented by formulas (30) to (34).

[0158]

[0159] The optimal multi-level multicast selection strategy of problem (P1) is represented by formula (36).

[0160]

[0161] The bandwidth allocation strategy and the power allocation strategy are represented by formula (37).

[0162]

[0163] The power allocation strategy is represented by formula (38).

[0164]

[0165] It should be noted that the meanings of the characters involved in formulas (28) to (38) have been specifically explained in the above formulas, and will not be repeated here. The step of obtaining the optimal multicast grouping strategy based on the optimal bandwidth allocation strategy and the optimal power allocation strategy includes: generating all multicast grouping strategies based on the optimal bandwidth allocation strategy and the optimal power allocation strategy; searching for the minimum value of the semantic loss and the average maximum multicast delay in all multicast grouping strategies, and the multicast grouping strategy corresponding to the minimum value is the optimal multicast grouping strategy.

[0166] For example, the optimal solution of problem (P1) can be given based on brute force search. First, under any given multicast grouping strategy , the bandwidth allocation strategy and the power allocation strategy are jointly optimized. Then, the optimal grouping strategy is found by brute force search. In any given The problem (P1) is equivalently transformed into problem (P2) below, which can be represented as equation (39).

[0167]

[0168] wherein, denotes the kth-order multicast delay, n P2 constraint is represented as equation (40), equations (31), (32), (35), and equation (41).

[0169]

[0170] wherein, ， ， .

[0171] The algorithm for solving problem (P2) is summarized in Algorithm 1. Wherein, denotes the dual variable associated with constraint (40), β1 denotes the dual variable associated with constraint (31) and constraint (32) respectively. β2 It should be noted that given equations (42) to (45) are used for Algorithm 1.

[0172]

[0173] wherein, Algorithm 1 can include: initializing

[0174] , , , ; According to given

[0175] , , , obtain , and from equations (42), (43) and (44), wherein, denotes the kth-order multicast delay under the condition that the input is , , , denotes the kth-order multicast delay under the condition that the input is n , denotes the kth-order multicast delay under the condition that the input is , , denotes the kth-order multicast delay under the condition that the input is l , denotes the kth-order multicast delay under the condition that the input is , , Conditional allocation of the first l Layer multicast power.

[0176] Computing About , , The sub-gradient of the sub-gradient is given as follows:

[0177]

[0178] According to the ellipse method, update , , , return according to the given , , From equations (42), (43) and (44) ， and Step, until , , Within the specified range of accuracy convergence;

[0179] Set (g)= From equation (45) And set equation (49)

[0180]

[0181] In equation (49), Indicates that the optimal power allocation is found in a multi-user communication system, while considering the constraints of power and bandwidth.

[0182] It can be understood that by obtaining the optimal bandwidth allocation strategy And the optimal power allocation strategy Then through the brute force search of the multicast group optimal strategy The optimal solution of problem (P1) is finally obtained.

[0183] The embodiment determines the semantic loss corresponding to different levels according to the semantic errors of different levels, determines the multicast delay corresponding to different levels according to the communication resources of the data receiving device in different levels, establishes an optimization problem according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels, and obtains the process of the optimal solution, so that the whole MEC system has efficient target identification and classification performance, meets the needs of different mobile agents in information transmission, reduces the transmission delay and resource consumption as a whole, and realizes efficient transmission of multi-level features.

[0184] In an embodiment, based on the first embodiment described above, the specific multicast transmission step can be as follows. For example, it is assumed that there are image samples sent to mobile agents by the MEC server, denoted as Each mobile agent needs all the image samples, so multicast is considered at the MEC server. Based on the multi-level feature encoder for multi-level processing of source information (here, taking picture information as an example), for each image sample to be transmitted, there are five transmission level types (features) available at the MEC. By obtaining the multi-level features, the semantic error is obtained. Among them, the visual features of the target picture can be extracted, and the visual features are projected into a latent low-dimensional intermediate space to obtain latent low-dimensional intermediate features. Based on the multi-level semantic encoder, the latent low-dimensional intermediate features are projected into the semantic feature space to obtain the semantic features. Based on the first semantic knowledge base (SKB at the MEC server), the semantic features are classified to obtain the semantic error, and the semantic vector corresponding to the minimum semantic error can also be obtained. Among them, the first semantic knowledge base is a knowledge base maintained at the MEC, used to store the semantic vectors of each sample image. Further, the target feature is confirmed, the MEC uses the multi-level semantic encoder to send the final target feature to the receiving end through the channel, and in this process, the communication resources are optimized, the bandwidth is reasonably allocated, the bandwidth consumption is reduced, and the transmission delay is reduced. The receiving end decodes the semantic information feature through the multi-level semantic decoder, matches with its own semantic knowledge base, obtains the semantic to be transmitted, and completes the semantic communication process.

[0185] Wherein, it can be determined according to the size relationship between the semantic error and the set error threshold value which feature is the target feature of the current target picture, and the interactive feedback action of the receiving end and the MEC server. For example, when the semantic error is less than the preset threshold value, the estimated category information corresponding to the semantic error is taken as the target feature of the current target picture, and is sent to the data receiving end; the data receiving end feeds back the identification result to the MEC, such as the identification result is that the estimated category semantic vector corresponding to the received estimated category identifier does not exist in the second semantic knowledge base (SKB of the mobile agent end) of the receiving end, then the MEC needs to resend the estimated category semantic vector as the target feature of the current target picture to the data receiving end. Similarly, in the sending decision of each level, the corresponding threshold value decision is first carried out, and the sending of the current level is carried out if it meets the threshold value requirement; if it does not meet, it is returned to the sending decision of the previous level, and similar decisions are carried out again to finally determine the transmission strategy.

[0186] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the multi-level feature multicast transmission method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0187] The present application also provides a multi-level feature multicast transmission device, please refer to Figure 5 , the multi-level feature multicast transmission device comprises:

[0188] The feature acquisition module 10 is configured to acquire multi-level features according to the target picture;

[0189] The transmission resource module 20 is configured to determine the semantic errors of different levels according to the multi-level features, and determine the communication resources of the data receiving device in the current level according to the target feature of the target picture;

[0190] The transmission optimization module 30 is configured to establish an optimization problem based on the semantic errors of different levels and the communication resources of the data receiving device of different levels, and obtain an optimal solution;

[0191] The multicast transmission module 40 is configured to multicast the target feature based on the optimal solution.

[0192] The multi-level feature multicast transmission device provided by the present application adopts the multi-level feature multicast transmission method in the above embodiment, which can solve the technical problem of large communication resource overhead in a multi-user scenario. Compared with the prior art, the multi-level feature multicast transmission device provided by the present application has the same beneficial effects as the multi-level feature multicast transmission method provided by the above embodiment, and the other technical features in the multi-level feature multicast transmission device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0193] The application provides a multi-level feature multicast transmission device, comprising: at least one processor; and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the multi-level feature multicast transmission method in the above embodiment one.

[0194] Reference will now be made to the following description Figure 6 which shows a structural schematic diagram of a multi-level feature multicast transmission device suitable for implementing the embodiments of the application. The multi-level feature multicast transmission device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 6 The multi-level feature multicast transmission device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.

[0195] As shown in Figure 6 , the multi-level feature multicast transmission device can include a processing apparatus 1001 (for example, a central processor, a graphics processor, or the like) which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage apparatus 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the multi-level feature multicast transmission device are also stored. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input apparatus 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, and the like; an output apparatus 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, and the like; and a communication apparatus 1009. The communication apparatus 1009 can allow the multi-level feature multicast transmission device to perform wireless or wired communication with other devices to exchange data. Although Figure 6The multi-tiered feature multicast transmission device is shown with various systems, but it should be understood that not all of the illustrated systems are required to be implemented or present. More or fewer systems can alternatively be implemented or present.

[0196] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0197] The multi-tiered feature multicast transmission device provided by the present application adopts the multi-tiered feature multicast transmission method in the above-mentioned embodiments, and can solve the technical problem of large communication resource overhead in a multi-user scenario. Compared with the prior art, the multi-tiered feature multicast transmission device provided by the present application has the same beneficial effects as the multi-tiered feature multicast transmission method provided by the above-mentioned embodiments, and other technical features in the multi-tiered feature multicast transmission device are the same as the features disclosed in the previous embodiment method, which will not be described here.

[0198] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0199] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0200] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer program) for executing the multi-tiered feature multicast transmission method in the above-mentioned embodiments.

[0201] The computer readable storage medium provided in the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0202] The computer readable storage medium described above may be contained in a multi-level feature multicast transmission device, or may exist separately without being assembled into the multi-level feature multicast transmission device.

[0203] The computer readable storage medium described above carries one or more programs, which, when executed by the multi-level feature multicast transmission device, cause the multi-level feature multicast transmission device to: acquire multi-level features according to a target picture; determine semantic errors of different levels according to the multi-level features, and determine communication resources of a data receiving device in a current level according to a target feature of the target picture; establish an optimization problem based on the communication resources of the data receiving device of different levels and the semantic errors of different levels, and obtain an optimal solution; and multicast the target feature based on the optimal solution.

[0204] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0205] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0206] The modules involved in the embodiments of the present application can be implemented in software or hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0207] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the above-mentioned multi-level feature multicast transmission method, and can solve the technical problem of large communication resource overhead in a multi-user scenario. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the multi-level feature multicast transmission method provided by the above-mentioned embodiments, and will not be described here.

[0208] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-level feature multicast transmission method described above.

[0209] The computer program product provided in this application can solve the technical problem of high communication resource overhead in multi-user scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multi-level feature multicast transmission method provided in the above embodiments, and will not be repeated here.

[0210] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method of multi-tier feature multicast transmission, the method comprising: The method comprises: According to the target picture, a multi-level feature is obtained, and the multi-level feature comprises a semantic feature; According to the semantic feature, a semantic error of different levels is determined, and a communication resource of a data receiving device in a current level is determined according to a target feature of the target picture; According to the semantic error of different levels, a semantic loss corresponding to different levels is determined; According to the communication resource of the data receiving device in different levels, a multicast delay corresponding to different levels is determined; According to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels, a multi-level multicast selection strategy, a bandwidth allocation strategy and a power allocation strategy are established; Under any given multicast grouping strategy, a corresponding bandwidth allocation strategy and a corresponding power allocation strategy are jointly optimized to obtain an optimal bandwidth allocation strategy and an optimal power allocation strategy; Based on the optimal bandwidth allocation strategy and the optimal power allocation strategy, all multicast grouping strategies are generated; In all the multicast grouping strategies, a minimum value of the semantic loss and the average maximum multicast delay is searched, and the multicast grouping strategy corresponding to the minimum value is the optimal multicast grouping strategy, and an optimal solution is obtained; Based on the optimal solution, the target feature is multicast transmitted.

2. The method of claim 1, wherein, The multi-level feature further comprises a visual feature and a latent low-dimensional intermediate feature, and the step of obtaining the multi-level feature according to the target picture comprises: The visual feature of the target picture is extracted, and the visual feature is projected to a latent low-dimensional intermediate space to obtain the latent low-dimensional intermediate feature; Based on a multi-level semantic encoder, the latent low-dimensional intermediate feature is projected to a semantic feature space to obtain the semantic feature.

3. An apparatus for multi-tier feature multicast transmission, the apparatus comprising: The device comprises: A feature acquisition module is configured to obtain a multi-level feature according to a target picture, and the multi-level feature comprises a semantic feature; A transmission resource module is configured to determine a semantic loss corresponding to different levels according to a semantic error of different levels, and determine a multicast delay corresponding to different levels according to a communication resource of a data receiving device in different levels; A transmission optimization module is configured to establish an optimization problem based on the semantic error of different levels and the communication resource of the data receiving device in different levels, and obtain an optimal solution; A multicast transmission module is configured to multicast transmit a target feature based on the optimal solution; The transmission resource module is further configured to establish a multi-level multicast selection strategy, a bandwidth allocation strategy and a power allocation strategy according to the semantic loss corresponding to different levels and the multicast delay corresponding to different levels; The transmission resource module is further configured to, under any given multicast grouping strategy, jointly optimize a corresponding bandwidth allocation strategy and a corresponding power allocation strategy to obtain an optimal bandwidth allocation strategy and an optimal power allocation strategy; The transmission resource module is further configured to generate all multicast grouping strategies based on the optimal bandwidth allocation strategy and the optimal power allocation strategy, and search for a minimum value of the semantic loss and the average maximum multicast delay in all the multicast grouping strategies, and the multicast grouping strategy corresponding to the minimum value is the optimal multicast grouping strategy, and an optimal solution is obtained.

4. A multi-tiered feature multicast transmission device, comprising: The device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the multi-level feature multicast transmission method according to any one of claims 1 to 2.

5. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the multi-level feature multicast transmission method according to any one of claims 1 to 2.

6. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the multi-level feature multicast transmission method according to any one of claims 1 to 2.

Images