Feature data transmission method and related equipment

Abstract

The invention relates to the field of artificial intelligence. The invention particularly relates to a feature data transmission method and related equipment. The method comprises the steps that a server receives semantic verification information of feature data F from an edge device and t0 sets of coding results, and the t0 sets of coding results are obtained by coding the feature data F through a sub-coder; t0 is the number of initially transmitted data groups of the edge device; the server decodes the received t0 groups of coding results by using a sub-decoder to obtain first reconstruction feature data; the server uses a semantic verifier to calculate a first similarity based on the semantic verification information, the channel state information and the first reconstruction feature data, wherein the first similarity is the similarity between the first reconstruction feature data and the feature data F; and the server sends indication information for indicating whether the edge device retransmits the feature data F to the edge device based on the first similarity and a similarity threshold. The embodiment of the invention is beneficial to improving the accuracy of a result for deciding whether to retransmit.

Description

Technical Field

[0001] The present application relates to the field of AI, and in particular to a feature data transmission method and related equipment. Background Art

[0002] With the continuous advancement of deep learning technology, neural network models have gradually become larger and more complex in order to achieve better performance. However, this has also brought a new problem: the model has higher and higher requirements for the computing resources of the running device, which has brought great challenges to the operation of the device.

[0003] In order to solve the above problems, the concept of end-cloud collaboration has been proposed. In the end-cloud collaboration scenario, the complex model is divided into two parts and deployed on both the edge device side and the cloud server side to reduce the computing overhead of the edge device side. The network model on the edge device side performs preliminary processing and feature extraction on the collected information, and then sends the extracted intermediate features to the cloud side. The cloud side uses the received intermediate features to run the downstream neural network task to obtain the results. The current method is to combine the joint source-channel coding (JSCC) model based on convolutional neural networks (CNN) with the hybrid automatic repeat request (HARQ) technology of traditional communication for data transmission. After each data transmission, the cloud-side device will make a decision based on the reconstructed data, and feedback the ACK / NACK flag to the end-side device to let the end-side device decide whether to retransmit. However, the result of making decisions based only on reconstructed data is low in accuracy. Summary of the invention

[0004] The embodiments of the present application provide a characteristic data transmission method and related equipment, and the use of the embodiments of the present application is conducive to improving the accuracy of the result of deciding whether to retransmit.

[0005] In a first aspect, an embodiment of the present application provides a method for transmitting characteristic data, which is applied to a server. The method includes:

[0006] The server receives semantic verification information and t0 group encoding results of feature data F from the edge device, wherein t0 group encoding results are obtained by encoding the feature data F using a sub-encoder; t0 is the number of initially transmitted data groups of the edge device, and t0 is an integer greater than 0; the server decodes the received t0 group encoding results using a sub-decoder to obtain first reconstructed feature data; the server uses a semantic verifier to calculate a first similarity based on the semantic verification information, channel state information and the first reconstructed feature data, and the first similarity is the similarity between the first reconstructed feature data and the feature data F; the server sends indication information to the edge device based on the first similarity and the similarity threshold, and the indication information is used to indicate whether the edge device retransmits the feature data F.

[0007] It should be pointed out here that the channel state information refers to the state information of the channel used for data transmission between the edge device and the server.

[0008] Among them, the sub-encoder and sub-decoder are implemented based on a spiking neural network (SNN).

[0009] When determining the similarity between the reconstructed feature data and the feature data F, the channel state information and the semantic verification information are introduced to improve the accuracy of the determined similarity, which is beneficial to the accuracy of the decision result. In the existing scheme, the encoder and the decoder are implemented based on, for example, a convolutional neural network. In order to obtain different encoding results of the feature data, a plurality of encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side, which will increase the storage overhead and training overhead of the encoder and the decoder. In the scheme of the present application, the sub-encoder and the sub-decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is used each time the encoding and decoding is performed, which makes the encoding and decoding results obtained each time different, and there is no need to set sub-codecs with different parameters. The encoding side only needs to store a set of parameters of the sub-encoder, and the decoding side only needs to store a set of parameters of the sub-decoder, which greatly saves the overhead of training the codec and the overhead of storing the codec.

[0010] In combination with the first aspect, in one possible implementation, when the first similarity is not greater than the similarity threshold, the indication information is used to instruct the edge device to retransmit the feature data F; when the first similarity is greater than the similarity threshold, the indication information is used to instruct the edge device not to retransmit the feature data F.

[0011] Furthermore, the indication information used to instruct the edge device to retransmit the characteristic data F is also used to indicate the transmission resources used to retransmit the characteristic data F.

[0012] In combination with the first aspect, in a possible implementation, the semantic verifier includes a first feature extraction module and a similarity calculation module, and the semantic verifier is used to calculate the first similarity based on the semantic verification information, the channel state information and the first reconstructed feature data, including:

[0013] The server extracts features from the first reconstructed feature data through a first feature extraction module to obtain a first feature vector; the server fuses the first feature vector, semantic verification information and channel state information to obtain a second feature vector; the server processes the second feature vector through a similarity calculation module to obtain a first similarity.

[0014] In combination with the first aspect, in a possible implementation, the method of the present application further includes:

[0015] The server allocates transmission resources to the edge device, and the number of initial transmission data groups is determined by the edge device based on the allocated transmission resources.

[0016] In combination with the first aspect, in a possible implementation, the method of the present application further includes:

[0017] The server trains to obtain a first semantic verifier;

[0018] The server trains a first semantic checker, including:

[0019] The server receives the first encoding result and the semantic verification information of the first feature data from the edge device, the first encoding result is obtained by the edge device encoding the first feature data using the trained first sub-encoder, and the first semantic verification information is obtained by the edge device processing the first feature data using the intermediate semantic verification extractor; the first feature data is obtained by the edge device extracting features from the first input data using the first feature extraction network; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server;

[0020] The server processes the received first encoding result according to the first sub-decoder to obtain third reconstructed feature data; the server uses the intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information to obtain the similarity between the third reconstructed feature data and the first feature data; the server calculates the first loss value based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the server reversely transmits the first gradient information based on the first loss value to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier whose weight parameters were adjusted during the last training;

[0021] The server sends first gradient information to the edge device, so that the edge device adjusts a weight parameter of the intermediate semantic check extractor based on the first gradient information;

[0022] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic check extractor converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic check extractor converge, the intermediate semantic verifier obtained is the first semantic verifier.

[0023] In combination with the first aspect, in a possible implementation, the method of the present application further includes:

[0024] The server trains to obtain a first sub-decoder;

[0025] The server trains to obtain a first sub-decoder, including:

[0026] The server receives a second encoding result from the edge device, where the second encoding result is obtained by the edge device encoding the second feature data using the first intermediate sub-encoder; the second feature data is obtained by the edge device extracting features from the second input data using the intermediate feature extraction network; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the last training;

[0027] The server processes the received second encoding result using the first intermediate sub-decoder to obtain fourth reconstructed feature data; the server processes the fourth reconstructed feature data using the intermediate task execution network to obtain the first task execution result; the server calculates the second loss value based on the first task execution result and the task label corresponding to the second input data; the server reversely transmits the second gradient information based on the second loss value to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server;

[0028] The server sends second gradient information to the edge device, so that the edge device adjusts the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information;

[0029] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-decoder is the first sub-decoder.

[0030] In combination with the first aspect, in a possible implementation, the method of the present application further includes:

[0031] The server trains to obtain the second sub-decoder,

[0032] The server trains to obtain a second sub-decoder, including:

[0033] The server receives a third encoding result from the edge device, where the third encoding result is obtained by the edge device encoding third feature data using the second intermediate sub-encoder; the third feature data is obtained by the edge device extracting features from third input data using the second feature extraction network; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the last training;

[0034] The server processes the received third encoding result according to the second intermediate sub-decoder to obtain fifth reconstructed feature data; the server processes the fifth reconstructed feature data using the second task execution network to obtain the second task execution result; the server calculates a third loss value based on the second task execution result and the task label corresponding to the third input data; the server reversely transmits third gradient information based on the third loss value to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is a sub-decoder whose weight parameters were adjusted during the last training;

[0035] The server sends third gradient information to the edge device, so that the edge device adjusts the weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0036] Repeat the above process until the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-decoder is the second sub-decoder.

[0037] By jointly training the sub-codec, semantic check extractor and semantic verifier in the above manner, the sub-codec, semantic check extractor and semantic verifier can be fully trained, which is beneficial to improving the accuracy of deciding whether to retransmit feature data when using the sub-codec, semantic check extractor and semantic verifier to execute the feature data transmission method of the present application.

[0038] In a second aspect, an embodiment of the present application provides another method for transmitting feature data, which is applied to edge devices.

[0039] The edge device obtains feature data F, and extracts semantic verification information of the feature data F through the first semantic verification extractor; the edge device uses the first sub-encoder to perform t0 encoding operations on the feature data F to obtain t0 groups of encoding results; t0 is the number of initially transmitted data groups; the edge device transmits the semantic verification information and t0 groups of encoding results to the server.

[0040] Among them, the first sub-encoder is implemented based on SNN.

[0041] By sending the semantic verification information to the server, the server introduces channel state information and semantic verification information when determining the similarity between the reconstructed feature data and the feature data F, thereby improving the accuracy of the determined similarity, and thus facilitating the accuracy of the decision result. In the existing scheme, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain different reconstructed feature data of the feature data, a plurality of encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side, which will increase the storage overhead and training overhead of the encoder and decoder. In the scheme of the present application, the sub-encoder and sub-decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is used each time encoding and decoding, which makes the encoding and decoding results obtained each time different, without the need to set a codec with different parameters. The encoding side only needs to store a set of parameters of the sub-encoder, and the decoding side only needs to store a set of parameters of the sub-decoder, which greatly saves the overhead of training the codec and the overhead of storing the codec.

[0042] In conjunction with the second aspect, in a possible implementation, the semantic verification extraction includes a second feature extraction module and a quantification module, and the server extracts semantic verification information of the feature data F through the semantic verification extractor, including:

[0043] The edge device performs feature extraction on the feature data F through a second feature extraction module; the edge device performs quantization processing on the feature vector extracted by the second feature extraction module through a quantization module to obtain semantic verification information of the feature data F.

[0044] In conjunction with the second aspect, in a possible implementation, the method of the present application further includes:

[0045] When the edge device receives the indication information sent by the server to instruct the retransmission of the characteristic data F, it uses the first sub-encoder to encode the characteristic data F to obtain the retransmitted encoding result; the number of retransmitted encoding results is determined based on the transmission resources indicated by the indication information; the edge device sends the retransmitted encoding result to the server.

[0046] By retransmitting the encoding result of the characteristic data F to the server, the server can obtain accurate reconstructed characteristic data, thereby facilitating the execution of subsequent tasks. And the transmission resources used for retransmitting the encoding result are indicated by the indication information, thereby realizing the adjustment of the transmission code rate.

[0047] In conjunction with the second aspect, in a possible implementation, the method of the present application further includes:

[0048] The edge device obtains the transmission resources allocated by the server; the edge device determines the number of initial transmission data groups t0 based on the allocated transmission resources. In this way, the transmission code rate can be adjusted.

[0049] In conjunction with the second aspect, in a possible implementation, the method of the present application further includes:

[0050] The edge device is trained to obtain a first semantic verification extractor;

[0051] The edge device is trained to obtain the first semantic verification extractor, including:

[0052] The edge device obtains first input data, and uses a first feature extraction network to extract features from the first input data to obtain first feature data; the edge device uses a first sub-encoder to encode the first feature data to obtain a first encoding result; the edge device uses an intermediate semantic verification extractor to process the first feature data to obtain semantic verification information of the first feature data; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server;

[0053] The edge device sends the first encoding result and semantic verification information of the first feature data to the server;

[0054] The edge device receives the first gradient information sent by the server; the first gradient information is determined by the server based on the first loss value, and the first loss value is calculated by the server based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the similarity between the third reconstructed feature data and the first feature data is obtained by the server using an intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information; the third reconstructed feature data is obtained by the server using the first sub-decoder to process the received first encoding result; the first gradient information is used by the server to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; when it is not the first training, the intermediate semantic verifier is a semantic verifier whose weight parameters were adjusted during the last training;

[0055] The edge device adjusts a weight parameter of an intermediate semantic check extractor based on the first gradient information;

[0056] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic check extractor and the weight parameters of the intermediate semantic checker are converged; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor are converged, the intermediate semantic check extractor obtained is the first semantic check extractor.

[0057] In conjunction with the second aspect, in a possible implementation, the method of the present application further includes:

[0058] The edge device is trained to obtain a first sub-encoder and a first feature extraction network;

[0059] The edge device is trained to obtain a first sub-encoder and a first feature extraction network, including:

[0060] The edge device obtains second input data, and the edge device uses the intermediate feature extraction network to extract features from the second input data to obtain second feature data; the edge device uses the intermediate sub-encoder to encode the second feature data to obtain a second encoding result; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters are adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters are adjusted during the last training;

[0061] The edge device sends the second encoding result to the server;

[0062] The edge device receives the second gradient information sent by the server; the second gradient information is determined by the server based on the second loss value, and the second loss value is calculated by the server based on the first task execution result and the task label corresponding to the second input data; the first task execution result is obtained by the server using the intermediate task execution network to process the fourth reconstructed feature data; the fourth reconstructed feature data is obtained by the server using the first intermediate sub-decoder to process the received second encoding result; the second gradient information is used by the server to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server;

[0063] The edge device adjusts a weight parameter of the first intermediate sub-encoder and a weight parameter of the intermediate feature extraction network based on the second gradient information;

[0064] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, and the obtained intermediate feature extraction network is the first feature extraction network.

[0065] In conjunction with the second aspect, in a possible implementation, the method of the present application further includes:

[0066] The edge device is trained to obtain a second sub-encoder;

[0067] The edge device is trained to obtain a second sub-encoder, including:

[0068] The edge device obtains third input data, and the edge device uses the second feature extraction network to extract features from the third input data to obtain third feature data; the edge device uses the second intermediate sub-encoder to encode the third feature data to obtain a third encoding result; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the last training;

[0069] The edge device sends the third encoding result to the server;

[0070] The edge device receives the third gradient information sent by the server; the third gradient information is determined by the server based on the third loss value, and the third loss value is calculated by the server based on the second task execution result and the task label corresponding to the third input data; the second task execution result is obtained by the server using the second task execution network to process the fifth reconstructed feature data; the fifth reconstructed feature data is obtained by the server using the second intermediate sub-decoder to process the received third encoding result; the third gradient information is used by the server to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training;

[0071] The edge device adjusts a weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0072] Repeat the above process until the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-encoder is the second sub-encoder.

[0073] In a third aspect, an embodiment of the present application provides a server, comprising a unit or module for executing a method provided in the first aspect or any possible implementation manner of the first aspect.

[0074] In a fourth aspect, an embodiment of the present application provides an edge device, comprising a unit or module for executing a method provided in the second aspect or any possible implementation manner of the second aspect.

[0075] In a fifth aspect, an embodiment of the present application provides a server, including a processor and a memory. The memory is used to store program code. The processor is used to call the program code stored in the memory to execute the method provided in the first aspect or any possible implementation method of the first aspect.

[0076] In a sixth aspect, an embodiment of the present application provides an edge device, including a processor and a memory. The memory is used to store program code. The processor is used to call the program code stored in the memory to execute the method provided in the second aspect or any possible implementation of the second aspect.

[0077] In the seventh aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes a method provided in any possible implementation of the first aspect, or a method provided in any possible implementation of the second aspect.

[0078] In an eighth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute a method provided in any possible implementation of the first aspect, or a method provided in any possible implementation of the second aspect.

[0079] It can be understood that the server described in the third aspect or the fifth aspect provided above is used to execute any method provided in the first aspect, the server described in the fourth aspect or the sixth aspect provided above is used to execute any method provided in the second aspect, and the computer storage medium described in the seventh aspect and the computer program product described in the eighth aspect are both used to implement any method provided in the first aspect, or any method provided in the second aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1a A schematic diagram of a system architecture provided for an embodiment of the present application;

[0081] Figure 1b A schematic diagram of a specific system architecture provided for an embodiment of the present application;

[0082] Figure 2 A flowchart of a characteristic data transmission method provided in an embodiment of the present application;

[0083] Figure 3a A schematic diagram of the structure of a reconstructor provided in an embodiment of the present application;

[0084] Figure 3b A schematic diagram of the structure of a fusion device provided in an embodiment of the present application;

[0085] Figure 4 A schematic diagram of the structure of a semantic checker provided in an embodiment of the present application;

[0086] Figure 5 A flowchart of another characteristic data transmission method provided in an embodiment of the present application;

[0087] Figure 6 A schematic diagram of the structure of a semantic verification extractor provided in an embodiment of the present application;

[0088] Figure 7 A schematic diagram of a result of a sub-encoder provided in an embodiment of the present application;

[0089] Figure 8 A flowchart of another characteristic data transmission method provided by an embodiment of the present application;

[0090] Fig. 9 Another specific system architecture diagram provided in an embodiment of the present application;

[0091] Fig.10 A schematic diagram of a joint training process provided in an embodiment of the present application;

[0092] Fig.11 A schematic diagram of a specific process of joint training provided in an embodiment of the present application;

[0093] Fig.12 A schematic diagram of another specific process of joint training provided in an embodiment of the present application;

[0094] Fig.13 A schematic diagram of another specific process of joint training provided in an embodiment of the present application;

[0095] Fig.14 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0096] Fig.15 A schematic diagram of the structure of an edge device provided in an embodiment of the present application;

[0097] Fig.16 A schematic diagram of the structure of another server provided in an embodiment of the present application;

[0098] Fig.17 A schematic diagram of the structure of another edge device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0099] The terms "first", "second", "third", "fourth" and the like in the specification, claims and drawings of this application are used to distinguish different objects rather than to describe a specific order.

[0100] "Multiple" means two or more. "And / or" describes the association relationship of the associated objects, indicating that there are three relationships. For example, A and / or B means: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0101] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0102] See also Figure 1a , Figure 1a A schematic diagram of a system architecture provided in an embodiment of the present application. Figure 1a As shown, the system includes an edge device 101 and a server 102 .

[0103] Among them, the edge device 101 can be a terminal device or a user device, such as a vehicle-mounted terminal, a desktop computer, a laptop computer, a tablet computer, a smart phone, a smart bracelet, etc.

[0104] The server 102 has a device for processing and storing data. For example, the server 102 is a single server, a server cluster, a cloud server, a cloud computing service center, or other forms of devices with computing capabilities.

[0105] After acquiring the feature data, the edge device 101 encodes the feature data F to obtain the encoding result 1 of the feature data F; and extracts the semantic verification information of the feature data F; the edge device 101 sends the encoding result 1 of the feature data F and the verification data to the server 102 through a binary symmetric channel (BSC). The server 102 decodes the encoding result 1 of the feature data F to obtain the first reconstructed feature data; the server 102 calculates the first similarity based on the semantic verification information, the channel state information and the first reconstructed feature data, and the first similarity is the similarity between the first reconstructed feature data and the feature data F. The server 102 sends indication information to the edge device 101. When the first similarity is greater than the similarity threshold, the indication information is used to indicate that the edge device 101 does not need to retransmit the feature data F; when the first similarity is not greater than the similarity threshold, the indication information is used to indicate that the edge device 101 needs to retransmit the feature data F, the edge device 101 encodes the feature data F to obtain the encoding result 2; sends the encoding result 2 to the server 102; the server 102 decodes based on the encoding result 1 and the encoding result 2 to obtain the second reconstructed feature data; the server 102 calculates the second similarity based on the semantic check information, the channel state information and the second reconstructed feature data, and the second similarity is the similarity between the second reconstructed feature data and the feature data F. The server 102 sends the indication information to the edge device 101 again. When the second similarity is greater than the similarity threshold, the indication information is used to indicate that the edge device 101 does not need to retransmit the feature data F; when the second similarity is not greater than the similarity threshold, the indication information is used to indicate that the edge device 101 needs to retransmit the feature data F. Repeat the above process until the calculated similarity is greater than the similarity threshold, or until the maximum number of transmissions T is reached.

[0106] When the calculated similarity is greater than the similarity threshold or reaches the maximum number of transmissions T, the server performs a preset task based on the reconstructed feature data. In one example, the preset task is a vehicle recognition task, and the feature data F is the feature data of the vehicle; in another example, the preset task is an image classification task, and the feature data F is the feature data of the image.

[0107] In a specific example, Figure 1b As shown, the edge device 101 is deployed with a semantic check extractor and a sub-encoder; the server 102 is deployed with a sub-decoder and a semantic checker.

[0108] The edge device 101 obtains feature data F from other devices, or the edge device extracts features from data (such as image data, video data, etc.) to obtain feature data F. The edge device 101 extracts semantic verification information of the feature data F through a semantic verification extractor, and encodes the feature data F through a sub-encoder to obtain an encoding result 1; the edge device 101 transmits the semantic verification information and the encoding result 1 to the server 102 through the BSC. The server 102 decodes the encoding result 1 through a sub-decoder to obtain first reconstructed feature data; the server 102 calculates a first similarity based on the semantic verification information, the channel state information and the first reconstructed feature data through a semantic verifier, and the first similarity is the similarity between the first reconstructed feature data and the feature data F. The server 102 sends an indication message to the edge device 101. When the first similarity is greater than the similarity threshold, the indication information is ACK, which is used to indicate that the edge device 101 does not need to retransmit the feature data F; when the first similarity is not greater than the similarity threshold, the indication information is NACK, which is used to indicate that the edge device 101 needs to retransmit the feature data F. The edge device 101 encodes the feature data F to obtain the encoding result 2; sends the encoding result 2 to the server 102 through the BSC; the server 102 decodes based on the encoding result 1 and the encoding result 2 through the sub-decoder to obtain the second reconstructed feature data; the server 102 calculates the second similarity based on the semantic verification information, the channel state information and the second reconstructed feature data through the semantic checker, and the second similarity is the similarity between the second reconstructed feature data and the feature data F. The server 102 sends the indication information to the edge device 101 again. When the second similarity is greater than the similarity threshold, the indication information is ACK, which is used to indicate that the edge device 101 does not need to retransmit the feature data F; when the second similarity is not greater than the similarity threshold, the indication information is NACK, which is used to indicate that the edge device 101 needs to retransmit the feature data F. The above process is repeated until the calculated similarity is greater than the similarity threshold, or until the maximum number of transmission times T is reached.

[0109] It should be understood that for the edge device, after encoding the feature data F, the encoding result will be transmitted to the server. Therefore, the maximum number of transmissions T in this application can also be understood as the maximum number of times the edge device encodes the feature data F.

[0110] It can be seen that when deciding whether to retransmit feature data, channel state information is introduced to improve the accuracy of semantic verification, thereby improving the accuracy of the decision whether to retransmit data, and at the same time being able to adapt to different channel scenarios.

[0111] It should be understood that the feature data is only an example, and of course the edge device 101 can also transmit other data to the server 102 based on the above method, such as image data, text data, video data, audio data, etc.

[0112] The following is a detailed description of the process of the circuit performance prediction algorithm of the present application.

[0113] See also Figure 2 , Figure 2 A flow chart of a characteristic data transmission method provided in an embodiment of the present application. The method is applied to Figure 1a or Figure 1b Server 102 in. Figure 2 As shown, the method includes:

[0114] S201. The server receives t0 group encoding results and semantic verification information of feature data F from the edge device.

[0115] Among them, t0 group encoding results are obtained by the edge device using the sub-encoder to encode the feature data F according to the number of initial transmission data groups t0; t0 is the number of initial transmission data groups of the edge device, and t0 is an integer greater than 0. The number of initial transmission data groups t0 is determined by the edge device based on the transmission resources allocated by the server.

[0116] S202: The server uses a sub-decoder to decode the received t0 group encoding result to obtain first reconstructed feature data.

[0117] In a possible implementation, the server processes each group of encoding results in the t0 group of encoding results to obtain membrane potential information and intermediate reconstructed feature data corresponding to each group of encoding results; wherein the dimension of the membrane potential information and the dimension of the intermediate reconstructed feature data are the same as the dimension of the feature data F; the server splices the membrane potential information and the intermediate reconstructed feature data corresponding to the t0 group of encoding results respectively to obtain a spliced ​​result; the server obtains the first reconstructed feature data based on the spliced ​​result.

[0118] The server processes the encoding result of the tth group to obtain the membrane potential information M corresponding to the encoding result of the tth group. t and intermediate reconstructed feature data S t ,include:

[0119] The server is based on the intermediate membrane potential information M' corresponding to the t-1 group of encoding results t-1 The encoding result of the tth group is processed to obtain the intermediate membrane potential information M' corresponding to the encoding result of the tth group t and intermediate feature data S' t ; Based on the membrane potential information M corresponding to the encoding result of the t-1th group t-1 Reconstruct the intermediate feature data S' t Processing is performed to obtain the membrane potential information M corresponding to the encoding result of the tth group t and intermediate reconstructed feature data S t ;

[0120] When t=1, the intermediate membrane potential information M' corresponding to the t-1th group of encoding results t-1 and membrane potential information M t-1 Both are 0.

[0121] In one example, the sub-decoder is implemented based on a spiking neural network.

[0122] Specifically, the sub-decoder includes a reconstructor and a fusion unit. Figure 3a As shown, the reconstructor includes a first convolutional layer, a first batch normalization (BN) layer, a first integrate-fire (IF) layer, a second convolutional layer, a second batch normalization layer, and an integrate and hybrid fire (IHF) layer. The server processes the t-th group of encoding results through the first convolutional layer and the first batch normalization layer to obtain a first intermediate result; the server processes the intermediate membrane potential information M' corresponding to the t-1-th group of encoding results through the first integrate-fire layer t-1 The first intermediate result is processed to obtain the intermediate membrane potential information M' corresponding to the tth group of encoding results t and intermediate feature data S' t ; The server uses the second convolutional layer and the second batch of normalization layers to process the intermediate feature data S' t Processing is performed to obtain a second intermediate result; the membrane potential information M corresponding to the encoding result of the t-1 group is processed by integrating and mixing the issuing layer t-1 The second intermediate result is processed to obtain the membrane potential information M corresponding to the tth group of encoding results t and intermediate reconstructed feature data S t .

[0123] like Figure 3b As shown, the server splices the membrane potential information corresponding to the encoding results of group t0 and the corresponding intermediate reconstructed feature data through the fusion device to obtain a first splicing result; then fills the first splicing result to a fixed length; finally, the fully connected network of the fusion device is used to process the filled first splicing result to obtain the first reconstructed feature data.

[0124] S203: The server uses a semantic verifier to calculate a first similarity based on the semantic verification information, the channel state information and the first reconstructed feature data.

[0125] The first similarity is the similarity between the first reconstructed feature data and the feature data F.

[0126] In one possible implementation, the semantic verifier includes a first feature extraction module and a similarity calculation module. The server extracts features from the first reconstructed feature data through the feature extraction module to obtain a first feature vector; the server fuses the first feature vector, semantic verification information and channel state information to obtain a second feature vector; and the server processes the second feature vector through the similarity calculation module to obtain a first similarity.

[0127] Specifically, Figure 4 As shown, the first feature extraction module includes a third convolution layer, a first activation layer, a fourth convolution layer, a second activation layer, and a first global average pooling layer. The server performs dimensionality reduction processing on the first reconstructed feature data through the third convolution layer, the first activation layer, the fourth convolution layer, the second activation layer, and the first global average pooling layer to obtain a first feature vector.

[0128] In one example, the first activation layer, the second activation layer, the third activation layer, and the fourth activation layer are nonlinear rectified linear units (ReLU); and the fifth activation layer is a sigmoid function (Sigmoid).

[0129] like Figure 4 As shown, the similarity calculation module includes a first fully connected layer, a third activation layer, a second fully connected layer, a fourth activation layer, a third fully connected layer, and a fifth activation layer. After obtaining the first feature vector, the server concatenates the first feature vector, the channel state information, and the semantic verification information to obtain the second feature vector; the server processes the second feature vector in sequence through the first fully connected layer, the third activation layer, the second fully connected layer, the fourth activation layer, the third fully connected layer, and the fifth activation layer to obtain the first similarity.

[0130] Among them, the third activation layer and the fourth activation layer are nonlinear rectification units, and the fifth activation layer is a S-type function.

[0131] S204. The server sends indication information to the edge device based on the first similarity and the similarity threshold, where the indication information is used to indicate whether the edge device retransmits the feature data F.

[0132] When the first similarity is not greater than the similarity threshold, it indicates that the difference between the first reconstructed feature data and the feature data F is large, and the server sends an indication information to the edge device to instruct the edge device to retransmit the feature data F to the server; when the first similarity is greater than the similarity threshold, it indicates that the difference between the first reconstructed feature data and the feature data F is small, and the server sends an indication information to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0133] In one possible manner, the server receives the encoding result retransmitted by the edge device, where the retransmitted encoding result is obtained by the edge device encoding the feature data F. Optionally, the number of retransmitted encoding results is one group or multiple groups. The multiple groups of retransmitted encoding results are obtained by the edge device encoding the feature data F multiple times.

[0134] In a possible implementation, the server sends the instruction information for instructing the edge device to retransmit the characteristic data F to the edge device, and also sends the number of retransmissions to the edge device at the same time. Optionally, the number of retransmissions can be 1, or more. The number of retransmissions is determined by the server according to the transmission resources allocated to the edge device.

[0135] In one possible implementation, the server sends indication information to the edge device for instructing the edge device to retransmit characteristic data F, and the indication information is also used to indicate transmission resource information for retransmission, so that the edge device determines the amount of data to be retransmitted based on the transmission resource information for retransmission.

[0136] After receiving the retransmitted encoding result, the server uses the reconstructor to reconstruct the intermediate membrane potential information M' corresponding to the encoding result of the t0th group. t0 The retransmitted encoding result is processed to obtain the intermediate membrane potential information M' and the intermediate reconstructed feature data S' corresponding to the retransmitted encoding result; based on the membrane potential information M corresponding to the encoding result of the t0th group t The intermediate reconstructed characteristic data S' is processed to obtain the membrane potential information M and the intermediate reconstructed characteristic data S corresponding to the retransmitted encoding result.

[0137] When there are multiple groups of retransmitted coding results, the server uses the reconstructor to process the multiple groups of retransmitted coding results in the same way as the reconstructor processes the t0 group of coding results, so as to obtain the membrane potential information and intermediate reconstruction feature data corresponding to the multiple groups of retransmitted coding results. When processing the first group of retransmitted coding results, the server uses the reconstructor to process the intermediate membrane potential information M' corresponding to the t0 group of coding results. t0 The first group of retransmitted encoding results are processed to obtain the intermediate membrane potential information and intermediate reconstruction feature data corresponding to the first group of retransmitted encoding results; based on the membrane potential information M corresponding to the encoding result of the t0th group t The intermediate reconstructed characteristic data corresponding to the first group of retransmitted encoding results are processed to obtain membrane potential information and intermediate reconstructed characteristic data corresponding to the first group of encoding results.

[0138] The server uses a fusion device to splice the membrane potential information and intermediate reconstructed feature data corresponding to the retransmitted encoding results with the first splicing result to obtain a second splicing result; then fills the second splicing result to a fixed length; finally, uses a fully connected network to process the filled second splicing result to obtain second reconstructed feature data.

[0139] The server uses the semantic checker to calculate the second similarity based on the semantic check information, the channel state information and the second reconstructed feature data; the first similarity is the similarity between the first reconstructed feature data and the feature data F. When the second similarity is not greater than the similarity threshold, it means that the difference between the second reconstructed feature data and the feature data F is large, and the server continues to send the indication information for instructing the edge device to retransmit the feature data F to the server to the edge device; when the second similarity is greater than the similarity threshold, it means that the difference between the second reconstructed feature data and the feature data F is small, and the server sends the indication information for instructing the edge device not to retransmit the feature data F to the server to the edge device.

[0140] The above process is repeated until the number of times the edge device transmits the encoding result of the feature data F reaches the maximum number of transmissions T or the server indicates that the edge device does not need to retransmit the feature data F.

[0141] The maximum number of transmission times T is determined based on the transmission resources allocated by the server.

[0142] It should be pointed out here that the similarity threshold is related to the channel state. For example, the lower the signal-to-noise ratio of the channel, the higher the similarity threshold; for example, the larger the capacity of the channel, the higher the similarity threshold. Through such a setting, the accuracy of the reconstructed feature data obtained by the server can be further improved.

[0143] It can be seen that when calculating the similarity between the reconstructed feature data and the original feature data, the introduction of channel state information is conducive to improving the accuracy of the similarity. When deciding whether the edge device needs to retransmit the feature data based on the similarity, it is conducive to improving the accuracy of the decision result. In the existing scheme, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain the encoding results of different feature data, multiple encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side, which will increase the storage overhead and training overhead of the encoder and decoder. In the scheme of the present application, the encoder and decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is used each time encoding and decoding, which makes the encoding and decoding results obtained each time different, without the need to set a codec with different parameters. The encoding side only needs to store a set of parameters of the encoder, and the decoder only needs to store a set of parameters of the decoder, which greatly saves the overhead of training the codec and the overhead of storing the codec.

[0144] See also Figure 5 , Figure 5 A flow chart of a characteristic data transmission method provided in an embodiment of the present application. The method is applied to Figure 1a or Figure 1b The edge device 101 in FIG. Figure 5 As shown, the method includes:

[0145] S501. The edge device obtains semantic verification information of feature data F and the number of initially transmitted data groups t0.

[0146] In one possible implementation, the edge device requests transmission resources from the server, or the server directly allocates transmission resources to the edge device. It should be understood that the transmission resources here refer to symbols, and the edge device determines the number of initial transmission data groups t0 based on the transmission resources allocated by the server. In this way, the transmission code rate can be adjusted.

[0147] In one possible implementation, the edge device extracts verification information of the feature data F through a semantic verification extractor. The semantic verification extractor includes a second feature extraction module and a quantization module. The edge device performs feature dimension reduction processing on the feature data F through the feature extraction module to obtain a feature vector; and performs quantization processing on the feature vector through the quantization module to obtain verification information of the feature data F.

[0148] like Figure 6 As shown, the second feature extraction module includes a fifth convolution layer, a sixth activation layer, a sixth convolution layer, a seventh activation layer, and a second global average pooling layer; the edge device processes the feature data F in sequence through the fifth convolution layer, the sixth activation layer, the sixth convolution layer, the seventh activation layer, and the second global average pooling layer to obtain a feature vector.

[0149] Optionally, the quantization module is used to implement a uniform quantization function, a non-uniform quantization function, a hash function or other quantization functions. The ninth activation layer and the tenth activation layer are both nonlinear rectification units.

[0150] S502: The edge device uses a sub-encoder to perform t0 encoding operations on the feature data to obtain t0 groups of encoding results.

[0151] In a possible embodiment, performing the i-th encoding operation includes:

[0152] The edge device uses the intermediate membrane potential information feature data F obtained from the i-1th encoding operation for processing to obtain the intermediate membrane potential information and the intermediate encoding result of the i-th encoding operation, and processes the intermediate encoding result based on the membrane potential information corresponding to the i-1th encoding operation to obtain the i-th group of encoding results and the membrane potential information corresponding to the i-th encoding operation, where i is an integer greater than and not greater than t0.

[0153] When i=1, the intermediate membrane potential information obtained by the i-1th encoding operation and the membrane potential information corresponding to the i-1th encoding operation are both 0.

[0154] In one example, the sub-encoder is implemented based on SNN.

[0155] In a specific example, Figure 7 As shown, the sub-encoder includes a seventh convolutional layer, a third batch normalization layer, an eighth convolutional layer, a fourth batch normalization layer, a second integration and release layer, and a third integration and release layer. The edge device processes the feature data F through the seventh convolutional layer and the third batch normalization layer to obtain a third intermediate result; the third intermediate result is processed by the third integration and release layer based on the intermediate membrane potential information obtained by the i-1th encoding operation to obtain the intermediate membrane potential information and the intermediate encoding result of the i-th encoding operation; the edge device processes the third intermediate result through the eighth convolutional layer and the fourth batch normalization layer to obtain a fourth intermediate result; the fourth intermediate result is processed by the fourth integration and release layer based on the membrane potential information obtained by the i-1th encoding operation to obtain the membrane potential information of the i-th encoding operation and the i-th group of encoding results.

[0156] In a possible implementation, the edge device receives the indication information sent by the server, and when the indication information is used to instruct the edge device to retransmit the characteristic data F, the edge device determines the amount of data to be retransmitted according to the remaining transmission resources. For example, the edge device determines to retransmit one or more groups of encoding results of the characteristic data F to the server.

[0157] The edge device encodes the feature data F using a sub-encoder to obtain one or more groups of retransmission encoding results.

[0158] It should be noted here that edge devices Figure 7 The sub-encoder shown encodes the feature data F using the intermediate membrane potential information obtained by the t0th encoding operation and the membrane potential information corresponding to the t0th encoding operation to obtain one or more groups of retransmission codes. The specific encoding process can be found in the relevant description of S502 and will not be described here.

[0159] S503. The edge device transmits the semantic verification information and the t0 group encoding result to the server.

[0160] The edge device sends the semantic verification information and the t0 group encoding result to the server through the BSC.

[0161] In a possible implementation, the method of this embodiment further includes:

[0162] When the edge device receives the indication information sent by the server to instruct the retransmission of the characteristic data F, it uses the first sub-encoder to encode the characteristic data F to obtain the retransmitted encoding result; the number of retransmitted encoding results is determined based on the transmission resources indicated by the indication information; the edge device sends the retransmitted encoding result to the server.

[0163] It should be understood that when receiving the indication information for indicating the retransmission of the characteristic data F, the edge device determines whether the maximum number of transmissions has been reached. If the maximum number of transmissions T has not been reached, the edge device uses the first sub-encoder to encode the characteristic data F. If the maximum number of transmissions T is reached, the edge device stops retransmitting the encoding results of the characteristic data F to the server, and does not perform encoding operations on the characteristic data F. There are one or more groups of encoding results retransmitted by the edge device to the server, which means that the edge device performs one or more encoding operations on the characteristic data F, and the specific amount of data of the retransmitted encoding results is determined by the edge device based on the transmission resources indicated by the indication information. By determining the amount of data to be transmitted based on the transmission resources allocated by the server, dynamic adjustment of the transmission code rate is achieved. The specific process of the edge device encoding the characteristic data F to obtain the retransmitted encoding results can be found in the relevant description of S502, which will not be described here.

[0164] It should be noted here that when the edge device performs one or more encoding operations on the feature data F to obtain the first group of encoding results of the retransmitted encoding results, the edge device uses the intermediate membrane potential information feature data F obtained by the t0th encoding operation to process to obtain the intermediate membrane potential information and intermediate encoding results of the first encoding operation in the above one or more encoding operations, and processes the intermediate encoding results based on the membrane potential information corresponding to the t0th encoding operation to obtain the first group of encoding results in the retransmitted encoding results and the membrane potential information corresponding to the first encoding operation in the above one or more encoding operations. The subsequent encoding process can be referred to the relevant description of S502, which will not be described here.

[0165] It can be seen that in the scheme of this embodiment, by sending the semantic verification information to the server, the server introduces channel state information and semantic verification information when determining the similarity between the reconstructed feature data and the feature data F, thereby improving the accuracy of the determined similarity, and thus facilitating the accuracy of the decision result. In the existing scheme, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain different reconstructed feature data of the feature data, multiple encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side, which will increase the storage overhead and training overhead of the encoder and decoder. In the scheme of the present application, the sub-encoder and sub-decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is used each time encoding and decoding, which makes the encoding and decoding results obtained each time different, and there is no need to set a codec with different parameters. The encoding side only needs to store a set of parameters of the sub-encoder, and the decoding side only needs to store a set of parameters of the sub-decoder, which greatly saves the overhead of training the codec and the overhead of storing the codec. By retransmitting the encoding result of the feature data F to the server, the server can obtain accurate reconstructed feature data, thereby facilitating the execution of subsequent tasks. And the transmission resource used for retransmitting the encoding result is indicated by the indication information, so as to adjust the transmission code rate.

[0166] See also Figure 8 , Figure 8 A flow chart of another characteristic data transmission method provided in an embodiment of the present application. Figure 8 As shown, the method includes:

[0167] S801, initial parameter configuration of edge devices and servers.

[0168] Specifically, the sub-encoder and semantic check extractor are deployed on the edge device side, and the sub-decoder and semantic checker are deployed on the server side. The edge device configures the structural parameters and weights of the sub-encoder and semantic check extractor and the maximum number of transmissions T, and the server configures the structural parameters and weights, similarity thresholds and maximum number of transmissions T of the sub-decoder and semantic checker.

[0169] S802. The edge device obtains feature data F, and inputs the feature data F into SNN-SC-HARQ for JSCC transmission.

[0170] like Fig. 9As shown in the figure, SNN-SC-HARQ mainly includes two modules: a semantic verification module composed of a semantic verification extractor (SimNet_E) deployed in the edge device and a semantic verifier (SimNet_D) deployed in the server, and a multi-rate codec (Multi-RateSNN-SC) module composed of a sub-encoder based on SNN deployed in the edge device and a sub-decoder (Reconstructor + Converter) based on SNN deployed in the server.

[0171] The edge device encodes the feature data F through the sub-encoder based on SNN to obtain multiple sets of encoding results of the feature data F. The specific encoding process can be referred to the relevant description of the Figure 5 embodiment shown in the figure, which will not be described here. The number of encoding results here is determined by the edge device based on the transmission resources allocated by the server.

[0172] In an example, the feature data F is represented by a three-dimensional real number matrix of dimension c×h×w. Each set of encoding results in the multiple sets of encoding results is a three-dimensional pulse matrix of dimension c2×h2×w2, that is, the element values in the matrix are 0 or 1, and c2×h2×w2 < c×h×w.

[0173] In an example, the edge device obtains the feature data F from its local storage or other devices, or the edge device uses the feature extraction module to extract features from the input data to obtain the feature data F. The input data can be image data, audio data, text data, or other types of data, which is not limited here.

[0174] S803. The edge device extracts the semantic verification information of the feature data F.

[0175] Among them, the edge device uses the semantic verification extractor to extract the semantic verification information S of the feature data F e , and the semantic verification information S e is of fixed length, such as 16 bits or 32 bits. The semantic verification information Se of the feature data F is a binary real number matrix of c1×1×1, and c1×1×1 << c×h×w, that is to say, the data volume of the semantic verification information is much less than the data volume of the transmitted feature data.

[0176] S804. The edge device obtains the transmission resources and sends t0 groups of encoding results and the semantic verification information S e to the server.

[0177] Specifically, the edge device sends a request to obtain transmission resources to the server, and the server allocates transmission resources to the edge device based on the request, or the server directly allocates transmission resources to the edge device. The edge device determines to transmit t0 group encoding results to the server based on the transmission resources, and then compares the t0 group encoding results with the semantic verification information S e The t0 group encoding results are identified as Z1, Z2, …, Z t0 .

[0178] S805. The server receives the encoding result and semantic verification information of group t0 from the edge device, and decodes the received encoding result of group t0 to obtain the decoding result of group t0; and obtains the first reconstructed feature data F based on the decoding result of group t0. t0 .

[0179] Specifically, the server uses the reconstructor to encode the result of group t0 Decode to obtain the decoding results S1, M1, S2, M2, ..., S t0 ,M t0 , where membrane potential information S1, S2, …, S t0 They are all three-dimensional matrices of dimension c×h×w, where the value of each element is 0 or 1. For the specific decoding process, see Figure 2 The server uses the fusion device to decode the t0 group of decoding results S1, M1, S2, M2, ..., S t0 ,M t0 splicing to obtain a first splicing result, the length of the first splicing result is B×t0; then padding the first splicing result to a fixed length of B×T, processing the padded splicing result to obtain the first reconstructed feature data F t0 For the specific processing process, please refer to Figure 2 The relevant description of the embodiment shown is not repeated here. t0 is a three-dimensional real matrix.

[0180] S806: The server performs semantic verification on the first reconstructed feature data based on the semantic verification information and the channel state information to obtain a first similarity.

[0181] The server performs semantic verification on the first reconstructed feature data based on the semantic verification information and the channel state information through a semantic verifier to obtain a first similarity. For the specific process, see Figure 2 The relevant description of the illustrated embodiment is not repeated here. The first similarity is used to characterize the similarity between the first reconstructed feature data F and the feature data F, and the similarity can also be called scalar soft information.

[0182] It should be noted that during the transmission process, due to reasons such as channel quality, there are differences between the t0 group coding result and semantic verification information transmitted by the edge device and the t0 group coding result and semantic verification information received by the server. Therefore, the server side identifies the t0 group coding result as Semantic validation information is marked as

[0183] S807: The server sends indication information to the edge device according to the first similarity.

[0184] Among them, when the first similarity is not greater than the similarity threshold, it means that the difference between the first reconstructed feature data and the feature data F is large, and the server sends an indication information (such as NACK) to the edge device to instruct the edge device to retransmit the feature data F to the server; when the first similarity is greater than the similarity threshold, it means that the difference between the first reconstructed feature data and the feature data F is small, and the server sends an indication information (such as ACK) to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0185] In a possible implementation, the server sends the instruction information for instructing the edge device to retransmit the characteristic data F to the edge device, and also sends the number of retransmissions to the edge device at the same time. Optionally, the number of retransmissions can be 1, or more. The number of retransmissions is determined by the server according to the transmission resources allocated to the edge device.

[0186] In one possible implementation, the server sends indication information to the edge device for instructing the edge device to retransmit characteristic data F, and the indication information is also used to indicate transmission resource information for retransmission, so that the edge device determines the amount of data to be retransmitted based on the transmission resource information for retransmission.

[0187] S808. The edge device sends retransmission information to the server.

[0188] Specifically, after receiving the instruction information for instructing the edge device to retransmit the characteristic data F to the server, the edge device determines the number of retransmissions. Optionally, the edge device determines the number of retransmissions based on the remaining transmission resources, or the number of retransmissions is sent by the server. The edge device encodes the characteristic data F based on the number of retransmissions to obtain the encoding result of the retransmission, and the number of the encoding result of the retransmission is the number of retransmissions determined above. For the specific encoding process, please refer to Figure 5 The relevant description of the embodiment shown is not repeated here. The edge device sends the encoding result to the server.

[0189] When receiving the retransmitted coding result, the server decodes the retransmitted coding result to obtain a decoding result; based on the decoding result and the decoding result corresponding to the above t0 group of coding results, the server obtains the second reconstructed feature data; taking the retransmission of a group of coding results as an example, the decoding results of the group of coding results include S t0+1 ,M t0+1 ; The server will S t0+1 ,M t0+1 and S1,M1,S2,M2,…,S t0 ,M t0 spliced ​​together to obtain a second splicing result; the length of the second splicing result is B×(t0+1); then the first splicing result is padded to a fixed length of B×T, and the padded splicing result is processed to obtain the second reconstructed feature data F t0+1 .

[0190] The server uses a semantic checker to check the second reconstructed feature data F based on the semantic check information and the channel state information. t0+1 A semantic check is performed to obtain a second similarity. The second similarity is used to characterize the second reconstructed feature data F t0+1 The similarity between it and the feature data F.

[0191] When the second similarity is not greater than the similarity threshold, the server continues to send indication information to the edge device to instruct the edge device to retransmit the feature data F to the server; when the second similarity is greater than the similarity threshold, the server sends indication information to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0192] The above process is repeated until the number of times the edge device transmits the encoding result of the feature data F reaches the maximum number of transmissions T or the server indicates that the edge device does not need to retransmit the feature data F.

[0193] In a possible implementation, the server obtains the reconstructor, the fusion device and the semantic checker from the training device after the reconstructor, the fusion device and the semantic checker are trained by the training device, or the server and the edge device are trained to obtain the sub-decoder and the semantic checker. At the same time, the edge device and the server can be trained to obtain the sub-encoder and the semantic check extractor.

[0194] The specific training process is as follows: Fig.10 As shown:

[0195] S1. Parameter configuration of edge devices and servers, involving structural parameter configuration of feature extraction network, task execution network, sub-codec, and semantic verification module, as well as initial value configuration of weight parameters of feature extraction network, task execution network, sub-codec, and semantic verification module, semantic verification information length, encoding result length, etc.

[0196] Among them, the semantic verification module includes a semantic verification extractor and a semantic verifier.

[0197] S2. Fix the weight parameters of the second feature extraction network deployed in the edge device and the weight parameters of the second task execution network deployed in the server, and the edge device and the server jointly train the second sub-encoder and the second sub-decoder.

[0198] Among them, the second feature extraction network and the second task execution network are obtained by server training.

[0199] S3, the edge device and the server jointly train the second feature extraction network, the second sub-encoder, the second sub-decoder, and the second task execution network.

[0200] S4. Fix the trained first feature extraction network, the first sub-encoder and the first sub-decoder, and jointly train the semantic verification extractor and the semantic verifier.

[0201] The following is a detailed description of the process of joint training of sub-codecs by edge devices and servers. Fig.11 As shown:

[0202] S21. The server trains a feature extraction network and a task execution network, and deploys the second feature extraction network in the edge device and the second task execution network in the server. The server also deploys an untrained sub-encoder in the edge device and an untrained sub-decoder in the server.

[0203] S22, the edge device obtains input data, and features the third input data through the second feature extraction network to obtain third feature data; using Figure 5 The method described in the illustrated embodiment utilizes a second intermediate sub-encoder to perform feature extraction on feature data to obtain a third encoding result; the edge device transmits the third encoding result to the server.

[0204] In the first training, the second intermediate sub-encoder is an untrained sub-encoder; in the non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted in the last training. An untrained sub-encoder refers to a sub-encoder with randomly initialized parameters, and a random seed can be preset in advance.

[0205] S23, the server receives the third encoding result, and uses Figure 2 The method described in the illustrated embodiment uses the second intermediate sub-decoder to process the received third encoding result to obtain fifth reconstructed feature data; and the fifth reconstructed feature data is input into the second task execution network for processing to obtain the second task execution result.

[0206] S24. The server calculates a third loss value based on the second task execution result and the task label corresponding to the third input data, backpropagates the first gradient information based on the third loss value to adjust the weight parameters in the second intermediate sub-decoder, and feeds back the first gradient information to the edge device.

[0207] It should be understood that, during the first training, the second intermediate sub-decoder here is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder here is a sub-decoder after the weight parameters are adjusted during the last training. An untrained sub-decoder refers to a sub-decoder with randomly initialized parameters, and a random seed can be preset in advance.

[0208] S25. The edge device adjusts a weight parameter of the second intermediate sub-encoder based on the third gradient information.

[0209] S26, repeating the above steps S22-S25 until the third training times reaches a third preset times, or the weight parameters of the second intermediate sub-encoding and the weight parameters of the second intermediate sub-decoding are converged.

[0210] It should be pointed out here that when the third training times reaches the umbrella preset times, or the weight parameters of the second intermediate sub-encoding and the weight parameters of the second intermediate sub-decoding are converged, the obtained second intermediate sub-encoder is the second sub-encoder, and the obtained second intermediate sub-decoder is the second sub-decoder.

[0211] The following specifically describes the process of the edge device and the server jointly training the second feature extraction network, the second sub-encoder, the second sub-decoder, and the second task execution network. Fig.12 As shown:

[0212] S31, the edge device obtains the second input data, and features the second input data through the intermediate feature extraction network to obtain second feature data; using Figure 5 The method described in the illustrated embodiment utilizes the first intermediate sub-encoder to perform feature extraction on the second feature data to obtain a second encoding result; the edge device transmits the second encoding result to the server.

[0213] Among them, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters are adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters are adjusted during the last training.

[0214] S32, the server receives the second encoding result, and uses Figure 2The method described in the illustrated embodiment uses the first intermediate sub-decoder to process the received second encoding result to obtain fourth reconstructed feature data; and the fourth reconstructed feature data is input into the intermediate task execution network for processing to obtain the first task execution result.

[0215] S33. The server calculates a second loss value based on the first task execution result and the task label corresponding to the second input data, back-propagates the second gradient information based on the second loss value to adjust the weight parameters in the first intermediate sub-decoder and the weight parameters of the intermediate task execution network, and feeds back the second gradient information to the edge device.

[0216] Among them, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder with weight parameters adjusted during the last training; the intermediate task execution network is the task execution network with weight parameters adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server.

[0217] S34. The edge device adjusts a weight parameter of the first intermediate sub-encoder based on the second gradient information.

[0218] It should be understood that during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters are adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters are adjusted during the last training. The second feature extraction network and the second task execution network are obtained through server training.

[0219] S35. Repeat the above steps S31-S34 until the second training times reaches the second preset times, or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge.

[0220] It should be pointed out here that when the second training times reaches the second preset times, or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, the obtained first intermediate sub-decoder is the first sub-decoder, the obtained intermediate feature extraction network is the first feature extraction network, and the obtained intermediate task execution network is the first task execution network.

[0221] The following is a detailed description of the process of jointly training the semantic verification extractor and semantic verifier by the edge device and the server. Fig.13 As shown:

[0222] S41. Based on the trained first feature extraction network, the first sub-encoder and the first sub-decoder, an untrained semantic verification extractor is deployed on the edge device side, and an untrained semantic verifier is deployed on the server side.

[0223] S42: The edge device obtains first input data, and features the first input data through a first feature extraction network to obtain first feature data; Figure 5 The method described in the illustrated embodiment uses a first sub-encoder to perform feature extraction on the first feature data to obtain a first encoding result; the edge device uses an intermediate semantic verification extractor to extract semantic verification information of the first feature data, and the edge device transmits the first encoding result and the semantic verification information of the first feature data to the server.

[0224] Among them, during the first training, the intermediate semantic check extractor is an untrained semantic check extractor; during non-first training, the intermediate semantic check extractor is a semantic check extractor with weight parameters adjusted during the last training. An untrained semantic check extractor refers to a semantic check extractor with randomly initialized parameters, and a random seed can be preset in advance.

[0225] S43, the server receives the first encoding result, and uses Figure 2 The method described in the illustrated embodiment uses a first sub-decoder to process the received first encoding result to obtain third reconstructed feature data; and uses an intermediate semantic verifier to verify the first reconstructed feature data based on the semantic verification information of the first feature data and the channel state information to obtain verification soft information.

[0226] The verification soft information is the similarity between the first feature data and the third reconstructed feature data.

[0227] S44. The server calculates a first loss value based on the semantic similarity label of the feature data corresponding to the verification soft information and the first input data, backpropagates the first gradient information based on the third loss value to adjust the weight parameters in the intermediate semantic verifier, and feeds back the first gradient information to the edge device.

[0228] Among them, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier with weight parameters adjusted during the last training. An untrained semantic verifier refers to a semantic verifier with randomly initialized parameters, and the random seed can be preset in advance.

[0229] S45. The edge device adjusts a weight parameter of the intermediate semantic check extractor based on the third gradient information.

[0230] S46. Repeat the above steps S42-S45 until the first training times reaches a first preset times, or the weight parameters of the intermediate semantic verification extractor and the weight parameters of the intermediate semantic verifier converge.

[0231] When the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic check extractor converge, the intermediate semantic check extractor obtained is the first semantic check extractor, and the intermediate semantic verifier obtained is the first semantic verifier.

[0232] By jointly training the sub-codec, semantic check extractor and semantic verifier in the above manner, the sub-codec, semantic check extractor and semantic verifier can be fully trained, which is beneficial to improving the accuracy of deciding whether to retransmit feature data when using the sub-codec, semantic check extractor and semantic verifier to execute the feature data transmission method of the present application.

[0233] Reference Fig.14 FIG. 14 is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1400 is Figure 1a or Figure 1b Server 102 in. Fig.14 As shown, the server 1400 includes a transceiver unit 1401 , a decoding unit 1402 , a calculation unit 1403 , an allocation unit 1404 and a training unit 1405 .

[0234] The transceiver unit 1401 is used to receive semantic verification information and t0 group encoding results of feature data F from an edge device, wherein the t0 group encoding results are obtained by encoding the feature data F using a sub-encoder; t0 is the number of initial transmission data groups of the edge device, and t0 is an integer greater than 0;

[0235] The decoding unit 1402 is used to decode the received t0 group of encoding results using a sub-decoder to obtain first reconstructed feature data;

[0236] A calculation unit 1403 is used to calculate a first similarity using a semantic checker based on the semantic check information, the channel state information and the first reconstructed feature data, where the first similarity is a similarity between the first reconstructed feature data and the feature data F;

[0237] The transceiver unit 1401 is further used to send indication information to the edge device based on the first similarity and the similarity threshold, where the indication information is used to indicate whether the edge device retransmits the feature data F.

[0238] Among them, the sub-encoder and sub-decoder are implemented based on SNN.

[0239] In one possible implementation, when the first similarity is not greater than the similarity threshold, the indication information is used to instruct the edge device to retransmit the feature data F; when the first similarity is greater than the similarity threshold, the indication information is used to instruct the edge device not to retransmit the feature data F.

[0240] Furthermore, the indication information used to instruct the edge device to retransmit the characteristic data F is also used to indicate the transmission resources used to retransmit the characteristic data F.

[0241] In a possible implementation, the semantic verifier includes a first feature extraction module and a similarity calculation module, and the calculation unit 1403 is specifically used to:

[0242] The first reconstructed feature data is subjected to feature extraction by a first feature extraction module to obtain a first feature vector; the second feature vector, semantic verification information and channel state information are fused to obtain a second feature vector; and the second feature vector is processed by a similarity calculation module to obtain a first similarity.

[0243] In a possible implementation, the server 1400 further includes:

[0244] The allocating unit 1404 is used to allocate transmission resources to the edge device, and the number of initial transmission data groups t0 is determined by the edge device based on the allocated transmission resources.

[0245] In a possible implementation, the server 1400 further includes:

[0246] A training unit 1405, used for training to obtain a first semantic checker;

[0247] The training unit 1405 is specifically used for:

[0248] Receive the first encoding result and semantic verification information of the first feature data from the edge device, the first encoding result is obtained by the edge device encoding the first feature data using the trained first sub-encoder, and the first semantic verification information is obtained by the edge device processing the first feature data using the intermediate semantic verification extractor; the first feature data is obtained by the edge device extracting features from the first input data using the first feature extraction network; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server 1400; according to the first sub-encoder The decoder processes the received first encoding result to obtain third reconstructed feature data; uses an intermediate semantic verifier to process the third reconstructed feature data, semantic verification information of the first feature data and channel state information to obtain the similarity between the third reconstructed feature data and the first feature data; calculates a first loss value based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the server reversely transmits the first gradient information based on the first loss value to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier whose weight parameters were adjusted during the last training;

[0249] The transceiver unit 1401 is further configured to send first gradient information to the edge device, so that the edge device adjusts a weight parameter of the intermediate semantic check extractor based on the first gradient information;

[0250] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic check extractor converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic check extractor converge, the intermediate semantic verifier obtained is the first semantic verifier.

[0251] In a possible implementation, the training unit 1405 is further used to train to obtain a first sub-decoder;

[0252] In terms of training to obtain the first sub-decoder, the training unit 1405 is specifically used for:

[0253] Receive a second encoding result from the edge device, the second encoding result is obtained by the edge device using the first intermediate sub-encoder to encode the second feature data; the second feature data is obtained by the edge device using the intermediate feature extraction network to extract features from the second input data; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters are adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters are adjusted during the last training; use the first intermediate sub-decoder to process the received second encoding result to obtain fourth reconstructed feature data; use the intermediate task execution network to process the fourth reconstructed feature data to obtain the first A task execution result; a second loss value is calculated based on the first task execution result and the task label corresponding to the second input data; a second gradient information is transmitted back based on the second loss value to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters are adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters are adjusted during the last training; the second feature extraction network and the second task execution network are trained by the server 1400; the second sub-encoder and the second sub-decoder are jointly trained by the edge device and the server 1400;

[0254] The transceiver unit 1401 is further configured to send second gradient information to the edge device, so that the edge device adjusts a weight parameter of the first intermediate sub-encoder and a weight parameter of the intermediate feature extraction network based on the second gradient information;

[0255] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-decoder is the first sub-decoder.

[0256] In a possible implementation, the training unit 1405 is further configured to obtain a second sub-decoder through training.

[0257] In terms of training to obtain the second sub-decoder, the training unit 1405 is specifically used for:

[0258] Receive a third encoding result from the edge device, the third encoding result is obtained by the edge device encoding the third feature data using the second intermediate sub-encoder; the third feature data is obtained by the edge device extracting features from the third input data using the second feature extraction network; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; when it is not the first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the last training; the received third encoding result is processed according to the second intermediate sub-decoder to obtain fifth reconstructed feature data; the fifth reconstructed feature data is processed using the second task execution network to obtain the second task execution result; a third loss value is calculated based on the task label corresponding to the second task execution result and the third input data; the server reversely transmits third gradient information based on the third loss value to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; when it is not the first training, the second intermediate sub-decoder is a sub-decoder whose weight parameters are adjusted during the last training;

[0259] The transceiver unit 1401 is further configured to send third gradient information to the edge device, so that the edge device adjusts the weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0260] Repeat the above process until the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-decoder is the second sub-decoder.

[0261] By jointly training the sub-codec, semantic check extractor and semantic verifier in the above manner, the sub-codec, semantic check extractor and semantic verifier can be fully trained, which is beneficial to improving the accuracy of deciding whether to retransmit feature data when using the sub-codec, semantic check extractor and semantic verifier to execute the feature data transmission method of the present application.

[0262] It is worth noting that the specific functional implementation of the server 1400 is shown in the above Figure 2The description of the characteristic data transmission method shown in the figure, for example, the decoding unit 1402 is used to implement the relevant content of executing S202, the calculation unit 1403 is used to implement the relevant content of executing S203, and the transceiver unit 1401, the allocation unit 1404, and the training unit 1405 are used to implement the relevant content of executing S201 and S204. Each unit or module in the server 1400 can be separately or completely merged into one or more other units or modules to form, or one (some) of the units or modules can be further divided into multiple smaller units or modules in function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In practical applications, the function of one unit (or module) is implemented by multiple units (or modules), or the function of multiple units (or modules) is implemented by one unit (or module).

[0263] Reference Fig.15 FIG. 1 is a schematic diagram of the structure of an edge device provided in an embodiment of the present application. The edge device 1500 is Figure 1a or Figure 1b The edge device 101 in FIG. Fig.15 As shown, the edge device 1500 includes an acquisition unit 1501, a semantic extraction unit 1502, an encoding unit 1503, a transceiver unit 1504 and a training unit 1505.

[0264] The acquisition unit 1501 is used to acquire feature data F.

[0265] The semantic extraction unit 1502 is used to extract semantic verification information of the feature data F through a first semantic verification extractor;

[0266] The encoding unit 1503 is used to perform t0 encoding operations on the feature data F using the first sub-encoder to obtain t0 sets of encoding results, where t0 is the number of initially transmitted data sets;

[0267] The transceiver unit 1504 is used to transmit the semantic verification information and the t0 group encoding result to the server.

[0268] In conjunction with the second aspect, in a possible implementation, the semantic verification extraction includes a second feature extraction module and a quantification module, and the semantic extraction unit is specifically used for:

[0269] The feature data F is subjected to feature extraction by a second feature extraction module; and the feature vector extracted by the second feature extraction module is subjected to quantization processing by a quantization module to obtain semantic verification information of the feature data F.

[0270] In combination with the second aspect, in a possible implementation, the encoding unit 1503 is further configured to: when the transceiver unit 1504 receives the indication information sent by the server for instructing to retransmit the characteristic data F, encode the characteristic data F using the first sub-encoder to obtain a retransmitted encoding result; the number of retransmitted encoding results is determined based on the transmission resources indicated by the indication information;

[0271] The transceiver unit 1504 is further configured to send the retransmitted encoding result to the server.

[0272] In conjunction with the second aspect, in a possible implementation, the acquiring unit 1501 is further configured to:

[0273] Acquire the transmission resources allocated by the server; determine the number of initial transmission data groups t0 based on the allocated transmission resources.

[0274] In conjunction with the second aspect, in a possible implementation, the edge device 1500 further includes:

[0275] A training unit 1505, configured to obtain a first semantic verification extractor through training;

[0276] The training unit 1505 is specifically used for:

[0277] Obtain first input data, use a first feature extraction network to perform feature extraction on the first input data to obtain first feature data; use a first sub-encoder to encode the first feature data to obtain a first encoding result; use an intermediate semantic verification extractor to process the first feature data to obtain semantic verification information of the first feature data; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device 1500 and the server;

[0278] The transceiver unit is also used to send the first encoding result and the semantic verification information of the first feature data to the server; receive the first gradient information sent by the server; the first gradient information is determined by the server based on the first loss value, and the first loss value is calculated by the server based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the similarity between the third reconstructed feature data and the first feature data is obtained by the server using an intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information; the third reconstructed feature data is obtained by the server using the first sub-decoder to process the received first encoding result; the first gradient information is used by the server to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; when it is not the first training, the intermediate semantic verifier is a semantic verifier whose weight parameters were adjusted during the last training;

[0279] The training unit 1505 is further used to adjust the weight parameter of the intermediate semantic check extractor based on the first gradient information;

[0280] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic check extractor and the weight parameters of the intermediate semantic checker are converged; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor are converged, the intermediate semantic check extractor obtained is the first semantic check extractor.

[0281] In combination with the second aspect, in a possible implementation, the training unit 1505 is further used to train and obtain a first sub-encoder and a first feature extraction network;

[0282] In terms of training to obtain the first sub-encoder and the first feature extraction network, the training unit 1505 is specifically used for:

[0283] Obtain second input data, perform feature extraction on the second input data using the intermediate feature extraction network to obtain second feature data; encode the second feature data using the intermediate sub-encoder to obtain a second encoding result; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the last training;

[0284] The transceiver unit 1504 is also used to send the second encoding result to the server; receive the second gradient information sent by the server; the second gradient information is determined by the server based on the second loss value, and the second loss value is calculated by the server based on the first task execution result and the task label corresponding to the second input data; the first task execution result is obtained by the server using the intermediate task execution network to process the fourth reconstructed feature data; the fourth reconstructed feature data is obtained by the server using the first intermediate sub-decoder to process the received second encoding result; the second gradient information is used by the server to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server;

[0285] The training unit 1505 is further used to adjust the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information;

[0286] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, and the obtained intermediate feature extraction network is the first feature extraction network.

[0287] In a possible implementation, the training unit 1505 is further configured to obtain a second sub-encoder through training;

[0288] In terms of training to obtain the second sub-encoder, the training unit 1505 is specifically used for:

[0289] Obtain third input data, perform feature extraction on the third input data using the second feature extraction network to obtain third feature data; encode the third feature data using the second intermediate sub-encoder to obtain a third encoding result; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the last training;

[0290] The transceiver unit is also used to send the third encoding result to the server; receive the third gradient information sent by the server; the third gradient information is determined by the server based on the third loss value, and the third loss value is calculated by the server based on the second task execution result and the task label corresponding to the third input data; the second task execution result is obtained by the server using the second task execution network to process the fifth reconstructed feature data; the fifth reconstructed feature data is obtained by the server using the second intermediate sub-decoder to process the received third encoding result; the third gradient information is used by the server to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training;

[0291] The training unit 1505 is further configured to adjust a weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0292] Repeat the above process until the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training number reaches the third preset number or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-encoder is the second sub-encoder.

[0293] It is worth noting that the specific functional implementation of the edge device 1500 is shown in the above Figure 5 The description of the feature data transmission method shown in the figure, for example, the acquisition unit 1501 and the semantic extraction unit 1502 are used to implement the relevant content of executing S501, the encoding unit 1503 is used to implement the relevant content of executing S502, and the transceiver unit 1504 and the training unit 1505 are used to implement the relevant content of executing S503. Each unit or module in the edge device 1500 can be separately or completely combined into one or several other units or modules to form, or one (some) of the units or modules can be further divided into multiple smaller functional units or modules to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), or the functions of multiple units (or modules) are implemented by one unit (or module).

[0294] Based on the description of the above method embodiments and related device embodiments, please refer to Fig.16 , an embodiment of the present invention also provides a structural diagram of a server 1600. Fig.16The server 1600 shown includes a memory 1601, a processor 1602, a communication interface 1603 and a bus 1604. The memory 1601, the processor 1602 and the communication interface 1603 are connected to each other through the bus 1604.

[0295] Optionally, the memory 1601 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0296] The memory 1601 can store programs. When the program stored in the memory 1601 is executed by the processor 1602, the processor 1602 and the communication interface 1603 are used to execute Figure 2 The various steps of the characteristic data transmission method of the illustrated embodiment.

[0297] The processor 1602 is a general-purpose CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits for executing related programs to perform Figure 2 The characteristic data transmission method of the illustrated embodiment.

[0298] The processor 1602 can also be an integrated circuit chip with signal processing capabilities. Figure 2 Each step of the characteristic data transmission method shown can be completed by an integrated logic circuit of hardware in the processor 1602 or instructions in software form. Optionally, the processor 1602 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor is a microprocessor or the processor is any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The optional software module is located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 1601, and the processor 1602 reads the information in the memory 1601, and combines its hardware to complete the functions required to be performed by the units included in the server 1400 of the embodiment of the present application, or executes Figure 2 The characteristic data transmission method of the illustrated embodiment.

[0299] The communication interface 1603 uses a transceiver-related device such as, but not limited to, a transceiver.

[0300] The bus 1604 may include a path for transmitting information between various components of the server 1600 (eg, the memory 1601 , the processor 1602 , and the communication interface 1603 ).

[0301] It should be noted that although Fig.16 The server 1600 shown only shows a memory, a processor, and a communication interface, but in the specific implementation process, those skilled in the art should understand that the server 1600 also includes other devices necessary for normal operation. At the same time, based on specific needs, those skilled in the art should understand that the server 1600 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the server 1600 may also only include the devices necessary for implementing the embodiments of the present application, and does not necessarily include Fig.16 All devices shown in .

[0302] Based on the description of the above method embodiments and related device embodiments, please refer to Fig.17 , an embodiment of the present invention also provides a structural diagram of an edge device 1700. Fig.17 The edge device 1700 shown includes a memory 1701 , a processor 1702 , a communication interface 1703 , and a bus 1704 . The memory 1701 , the processor 1702 , and the communication interface 1703 are connected to each other through the bus 1704 .

[0303] Optionally, memory 1701 is ROM, a static storage device, a dynamic storage device or RAM.

[0304] The memory 1701 can store programs. When the program stored in the memory 1701 is executed by the processor 1702, the processor 1702 and the communication interface 1703 are used to execute Figure 5 The various steps of the characteristic data transmission method of the illustrated embodiment.

[0305] The processor 1702 is a general-purpose CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits for executing related programs to perform Figure 5 The characteristic data transmission method of the illustrated embodiment.

[0306] The processor 1702 can also be an integrated circuit chip with signal processing capabilities. Figure 5Each step of the characteristic data transmission method of the illustrated embodiment can be completed by an integrated logic circuit of hardware or software instructions in the processor 1702. Optionally, the processor 1702 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor is a microprocessor or the processor is any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The optional software module is located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 1701, and the processor 1702 reads the information in the memory 1701, and combines its hardware to complete the functions required to be performed by the units included in the edge device 1500 of the embodiment of the present application, or executes Figure 5 The characteristic data transmission method of the illustrated embodiment.

[0307] The communication interface 1703 uses a transceiver or other related device such as, but not limited to, a transceiver.

[0308] The bus 1704 may include a path for transmitting information between various components of the edge device 1700 (eg, the memory 1701 , the processor 1702 , and the communication interface 1703 ).

[0309] It should be noted that although Fig.17 The edge device 1700 shown only shows a memory, a processor, and a communication interface, but in the specific implementation process, those skilled in the art should understand that the edge device 1700 also includes other devices necessary for normal operation. At the same time, based on specific needs, those skilled in the art should understand that the edge device 1700 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the edge device 1700 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Fig.17 All devices shown in .

[0310] An embodiment of the present application further provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to implement the image quality processing method of the embodiment of the present application.

[0311] Optionally, as an implementation method, the chip also includes a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the image quality processing method.

[0312] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0313] The embodiment of the present application further provides a computer program product including instructions. When the computer program product is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0314] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions described by the various illustrative logic blocks, modules, and steps may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or includes any communication media that facilitates the transfer of computer programs from one place to another (e.g., based on a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-temporary tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

[0315] As an example and not limitation, such computer-readable storage media include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwaves are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carriers, signals or other temporary media, but are actually directed to non-temporary tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD) and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0316] Instructions may be executed by one or more processors, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logic blocks, modules, and steps described herein are provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Moreover, the techniques may be fully implemented in one or more circuits or logic elements.

[0317] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the division of the unit is only a logical function division, and there are other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Optionally, the mutual coupling, direct coupling, or communication connection shown or discussed is an indirect coupling or communication connection through some interfaces, devices or units, such as electrical, mechanical or other forms.

[0318] Optionally, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e. located in one place, or distributed over multiple network units. Some or all of the units may be selected based on actual needs to achieve the purpose of the solution of this embodiment.

[0319] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part.

[0320] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A method for transmitting characteristic data, characterized in that: The method is applied to a server, and the method comprises: Receive semantic verification information and t0 group encoding results of feature data F from an edge device, where the t0 group encoding results are obtained by encoding the feature data F using a sub-encoder; t0 is the number of initial data groups transmitted by the edge device, and t0 is an integer greater than 0; Decoding the received t0 group encoding results using a first sub-decoder to obtain first reconstructed feature data; Using a first semantic checker to calculate a first similarity based on the semantic check information, the channel state information and the first reconstructed feature data, the first similarity being a similarity between the first reconstructed feature data and the feature data F; Based on the first similarity and the similarity threshold, indication information is sent to the edge device, where the indication information is used to indicate whether the edge device retransmits the feature data F.

2. The method according to claim 1, characterized in that When the first similarity is not greater than a similarity threshold, the indication information is used to instruct the edge device to retransmit the feature data F; When the first similarity is greater than the similarity threshold, the indication information is used to indicate that the edge device does not need to retransmit the feature data F.

3. The method according to claim 1 or 2, characterized in that: The first semantic checker includes a first feature extraction module and a similarity calculation module, and the first semantic checker is used to calculate the first similarity based on the semantic check information, the channel state information and the first reconstructed feature data, including: Performing feature extraction on the first reconstructed feature data by the first feature extraction module to obtain a first feature vector; fusing the first feature vector, the semantic check information and the channel state information to obtain a second feature vector; The second feature vector is processed by the similarity calculation module to obtain the first similarity.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Transmission resources are allocated to the edge device, and the number of initially transmitted data groups is determined by the edge device based on the allocated transmission resources.

5. The method according to any one of claims 1 to 4, characterized in that: The first sub-encoder and the first sub-decoder are implemented based on a spiking neural network (SNN).

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Training to obtain the first semantic verifier; The training to obtain the first semantic verifier includes: Receive a first encoding result and semantic verification information of first feature data from the edge device, wherein the first encoding result is obtained by the edge device encoding the first feature data using a trained first sub-encoder, and the first semantic verification information is obtained by the edge device processing the first feature data using an intermediate semantic verification extractor; the first feature data is obtained by the edge device performing feature extraction on first input data using a first feature extraction network; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server; The received first encoding result is processed according to the first sub-decoder to obtain third reconstructed feature data; the third reconstructed feature data, semantic verification information of the first feature data and channel state information are processed by an intermediate semantic verifier to obtain similarity between the third reconstructed feature data and the first feature data; a first loss value is calculated based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; first gradient information is reversely transmitted based on the first loss value to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier with weight parameters adjusted during the last training; Sending the first gradient information to the edge device, so that the edge device adjusts the weight parameter of the intermediate semantic check extractor based on the first gradient information; Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge, the intermediate semantic verifier obtained is the first semantic verifier.

7. The method according to claim 6, characterized in that The method further comprises: Train to obtain the first sub-decoder; The training to obtain the first sub-decoder includes: Receive a second encoding result from the edge device, where the second encoding result is obtained by the edge device encoding second feature data using the first intermediate sub-encoder; the second feature data is obtained by the edge device extracting features from second input data using the intermediate feature extraction network; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder for adjusting weight parameters during the last training, and the intermediate feature extraction network is the feature extraction network for adjusting weight parameters during the last training; The received second encoding result is processed by the first intermediate sub-decoder to obtain fourth reconstructed feature data; the fourth reconstructed feature data is processed by the intermediate task execution network to obtain the first task execution result; a second loss value is calculated based on the first task execution result and the task label corresponding to the second input data; the second gradient information is reversely transmitted based on the second loss value to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by the server training; the second sub-encoder and the second sub-decoder are obtained by the joint training of the edge device and the server; Sending the second gradient information to the edge device, so that the edge device adjusts the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information; Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-decoder is the first sub-decoder.

8. The method according to claim 7, characterized in that The method further comprises: The second sub-decoder is obtained by training, The step of training to obtain the second sub-decoder includes: Receive a third encoding result from the edge device, where the third encoding result is obtained by the edge device encoding third feature data using the second intermediate sub-encoder; the third feature data is obtained by the edge device performing feature extraction on third input data using the second feature extraction network; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder for which weight parameters are adjusted during the last training; The received third encoding result is processed according to the second intermediate sub-decoder to obtain fifth reconstructed feature data; the fifth reconstructed feature data is processed using the second task execution network to obtain a second task execution result; a third loss value is calculated based on the second task execution result and the task label corresponding to the third input data; the third gradient information is reversely transmitted based on the third loss value to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is a sub-decoder whose weight parameters were adjusted during the last training; Sending the third gradient information to the edge device, so that the edge device adjusts the weight parameter of the second intermediate sub-encoder based on the third gradient information; Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged, the obtained second intermediate sub-decoder is the second sub-decoder.

9. A method for transmitting characteristic data, characterized in that: Applied to the edge device side, the method further includes: Acquire feature data F, and extract semantic verification information of the feature data F through a first semantic verification extractor; Perform t0 encoding operations on the feature data F using the first sub-encoder to obtain t0 sets of encoding results; t0 is the number of initially transmitted data sets; The semantic verification information and the t0 group encoding result are transmitted to the server.

10. The method according to claim 9, characterized in that The semantic verification extraction includes a second feature extraction module and a quantification module, and the semantic verification information of the feature data F is extracted by the semantic verification extractor, including: Performing feature extraction on the feature data F by using the second feature extraction module; The feature vector extracted by the second feature extraction module is quantized by the quantization module to obtain semantic verification information of the feature data F.

11. The method according to claim 9 or 10, characterized in that: The method further comprises: When receiving the indication information sent by the server for instructing to retransmit the characteristic data F, the characteristic data F is encoded by using the first sub-encoder to obtain a retransmitted encoding result; the number of the retransmitted encoding results is determined based on the transmission resources indicated by the indication information; The retransmitted encoding result is sent to the server.

12. The method according to any one of claims 9 to 11, characterized in that: The method further comprises: Acquire the transmission resources allocated by the server; The number t0 of the initially transmitted data groups is determined based on the allocated transmission resources.

13. The method according to any one of claims 9 to 12, characterized in that: The first sub-encoder is implemented based on a spiking neural network (SNN).

14. The method according to any one of claims 9 to 13, characterized in that: The method further comprises: Training to obtain the first semantic verification extractor; The training to obtain the first semantic verification extractor includes: Obtain first input data, use the first feature extraction network to perform feature extraction on the first input data to obtain first feature data; use the first sub-encoder to encode the first feature data to obtain a first encoding result; use an intermediate semantic verification extractor to process the first feature data to obtain semantic verification information of the first feature data; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server; Sending the first encoding result and semantic verification information of the first feature data to the server; Receive the first gradient information sent by the server; the first gradient information is determined by the server based on a first loss value, and the first loss value is calculated by the server based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the similarity between the third reconstructed feature data and the first feature data is obtained by the server using an intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information; the third reconstructed feature data is obtained by the server using the first sub-decoder to process the received first encoding result; the first gradient information is used by the server to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; when it is not the first training, the intermediate semantic verifier is a semantic verifier whose weight parameters were adjusted during the last training; adjusting a weight parameter of the intermediate semantic check extractor based on the first gradient information; Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic check extractor and the weight parameters of the intermediate semantic checker are both converged; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor are both converged, the intermediate semantic check extractor obtained is the first semantic check extractor.

15. The method according to claim 14, characterized in that The method further comprises: Train to obtain the first sub-encoder and the first feature extraction network; The training to obtain the first sub-encoder and the first feature extraction network includes: Obtain second input data, perform feature extraction on the second input data using an intermediate feature extraction network to obtain second feature data; encode the second feature data using an intermediate sub-encoder to obtain a second encoding result; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the last training; Sending the second encoding result to the server; Receive the second gradient information sent by the server; the second gradient information is determined by the server based on the second loss value, and the second loss value is calculated by the server based on the first task execution result and the task label corresponding to the second input data; the first task execution result is obtained by the server using the intermediate task execution network to process the fourth reconstructed feature data; the fourth reconstructed feature data is obtained by the server using the first intermediate sub-decoder to process the received second encoding result; the second gradient information is used by the server to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by the server training; the second sub-encoder and the second sub-decoder are obtained by the joint training of the edge device and the server; Adjusting a weight parameter of the first intermediate sub-encoder and a weight parameter of the intermediate feature extraction network based on the second gradient information; Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, and the obtained intermediate feature extraction network is the first feature extraction network.

16. The method according to claim 15, characterized in that The method further comprises: Train to obtain the second sub-encoder; The training to obtain the second sub-encoder includes: Acquire third input data, perform feature extraction on the third input data using the second feature extraction network to obtain third feature data; encode the third feature data using the second intermediate sub-encoder to obtain a third encoding result; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the last training; Sending the third encoding result to the server; Receive the third gradient information sent by the server; the third gradient information is determined by the server based on the third loss value, and the third loss value is calculated by the server based on the second task execution result and the task label corresponding to the third input data; the second task execution result is obtained by the server using the second task execution network to process the fifth reconstructed feature data; the fifth reconstructed feature data is obtained by the server using the second intermediate sub-decoder to process the received third encoding result; the third gradient information is used by the server to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; Adjusting a weight parameter of the second intermediate sub-encoder based on the third gradient information; Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged, the obtained second intermediate sub-encoder is the second sub-encoder.

17. A server, characterized in that: The method comprises a unit for executing the method according to any one of claims 1 to 8.

18. An edge device, characterized in that: The method comprises a unit for executing the method according to any one of claims 9 to 16.

19. A server, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program codes, and the processor is used to execute the program codes to implement the method according to any one of claims 1 to 8.

20. An edge device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program codes, and the processor is used to execute the program codes to implement the method according to any one of claims 9 to 16.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 16 is implemented.

Images