Feature data transmission method and related device

Abstract

The present application relates to the field of artificial intelligence, and in particular to a feature data transmission method and a related device. The method comprises: a server receives semantic verification information and t0 groups of encoding results of feature data F from an edge device, wherein the t0 groups of encoding results are obtained by using sub-encoders to encode the feature data F, and t0 is the number of groups of initial data of the edge device; the server uses the sub-decoders to decode the received t0 groups of encoding results, so as to obtain first reconstructed feature data; the server uses a semantic checker to calculate a first similarity on the basis of the semantic verification information, channel state information, and the first reconstructed feature data, the first similarity being a similarity between the first reconstructed feature data and the feature data F; and the server sends, to the edge device on the basis of the first similarity and a similarity threshold, indication information used for indicating whether the edge device retransmits the feature data F. By using embodiments of the present application, the accuracy of the result of determining whether to perform retransmission is improved.

Description

Feature data transmission method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 7, 2023, with application number 202311155494.6 and invention name “Feature Data Transmission Method and Related Equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of AI, and in particular to a feature data transmission method and related equipment. Background Art

[0003] With the continuous advancement of deep learning technology, neural network models are becoming increasingly large and complex to achieve better performance. However, this also brings a new problem: the models require increasingly higher computing resources from the devices running them, posing a significant challenge to device operation.

[0004] To address these issues, the concept of edge-cloud collaboration has been proposed. In this scenario, complex models are split into two parts and deployed on the edge device and cloud server, respectively, to reduce computational overhead on the edge device. The network model on the edge device performs preliminary processing and feature extraction on the collected information, then sends the extracted intermediate features to the cloud. The cloud uses the received intermediate features to run downstream neural network tasks to obtain results. The current approach combines the convolutional neural network (CNN)-based joint source-channel coding (JSCC) model with the hybrid automatic repeat request (HARQ) technology used in traditional communications for data transmission. After each data transmission, the cloud-side device makes a decision based on the reconstructed data, sending an ACK / NACK signal to the edge device to allow the edge device to decide whether to retransmit. However, the accuracy of decisions based solely on reconstructed data is low.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a method for transmitting characteristic data and related equipment. Adoption of the embodiments of the present application is beneficial to improving the accuracy of the result of deciding whether to retransmit.

[0007] In a first aspect, an embodiment of the present application provides a method for transmitting feature data, applied to a server. The method includes:

[0008] The server receives semantic verification information and t0 group encoding results of the feature data F from the edge device, wherein the t0 group encoding results are obtained by encoding the feature data F using a sub-encoder; t0 is the number of initially transmitted data groups of the edge device, and t0 is an integer greater than 0; the server uses a sub-decoder to decode the received t0 group encoding results to obtain first reconstructed feature data; the server uses a semantic verifier to calculate a first similarity based on the semantic verification information, channel state information and the first reconstructed feature data, and the first similarity is the similarity between the first reconstructed feature data and the feature data F; the server sends indication information to the edge device based on the first similarity and the similarity threshold, and the indication information is used to instruct the edge device whether to retransmit the feature data F.

[0009] It should be pointed out here that the channel state information refers to the state information of the channel used for data transmission between the edge device and the server.

[0010] Among them, the sub-encoder and sub-decoder are implemented based on a spiking neural network (SNN).

[0011] When determining the similarity between the reconstructed feature data and the feature data F, channel state information and semantic verification information are introduced to improve the accuracy of the determined similarity, which is beneficial to the accuracy of the decision result. In the existing scheme, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain different encoding results of the feature data, a plurality of encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side. This will increase the storage overhead and training overhead of the encoder and decoder. In the scheme of the present application, the sub-encoder and sub-decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is used each time encoding and decoding, which makes the encoding and decoding results obtained each time different. There is no need to set sub-encoders and decoders with different parameters. The encoding side only needs to store a set of parameters of the sub-encoder, and the decoding side only needs to store a set of parameters of the sub-decoder, which greatly saves the overhead of training the codec and the overhead of storing the codec.

[0012] In combination with the first aspect, in one possible implementation, when the first similarity is not greater than the similarity threshold, the indication information is used to instruct the edge device to retransmit the feature data F; when the first similarity is greater than the similarity threshold, the indication information is used to instruct the edge device not to retransmit the feature data F.

[0013] Furthermore, the indication information used to instruct the edge device to retransmit the characteristic data F is also used to indicate the transmission resources used to retransmit the characteristic data F.

[0014] In conjunction with the first aspect, in one possible implementation, the semantic verifier includes a first feature extraction module and a similarity calculation module, and the semantic verifier calculates the first similarity based on the semantic verification information, the channel state information, and the first reconstructed feature data, including:

[0015] The server extracts features from the first reconstructed feature data through a first feature extraction module to obtain a first feature vector; the server fuses the first feature vector, semantic verification information, and channel state information to obtain a second feature vector; the server processes the second feature vector through a similarity calculation module to obtain a first similarity.

[0016] In conjunction with the first aspect, in one possible implementation, the method of the present application further includes:

[0017] The server allocates transmission resources to the edge device, and the number of initially transmitted data groups is determined by the edge device based on the allocated transmission resources.

[0018] In conjunction with the first aspect, in one possible implementation, the method of the present application further includes:

[0019] The server trains to obtain a first semantic verifier;

[0020] The server trains a first semantic checker, including:

[0021] The server receives the first encoding result and semantic verification information of the first feature data from the edge device, where the first encoding result is obtained by the edge device encoding the first feature data using the trained first sub-encoder, and the first semantic verification information is obtained by the edge device processing the first feature data using the intermediate semantic verification extractor; the first feature data is obtained by the edge device performing feature extraction on the first input data using the first feature extraction network; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server;

[0022] The server processes the received first encoding result according to the first sub-decoder to obtain third reconstructed feature data; the server uses the intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information to obtain the similarity between the third reconstructed feature data and the first feature data; the server calculates a first loss value based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the server reversely transmits the first gradient information based on the first loss value to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier with adjusted weight parameters during the last training;

[0023] The server sends first gradient information to the edge device, so that the edge device adjusts the weight parameter of the intermediate semantic check extractor based on the first gradient information;

[0024] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge, the intermediate semantic verifier obtained is the first semantic verifier.

[0025] In conjunction with the first aspect, in one possible implementation, the method of the present application further includes:

[0026] The server trains to obtain the first sub-decoder;

[0027] The server trains a first sub-decoder, including:

[0028] The server receives a second encoding result from the edge device, where the second encoding result is obtained by the edge device encoding the second feature data using the first intermediate sub-encoder; the second feature data is obtained by the edge device extracting features from the second input data using the intermediate feature extraction network; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the previous training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the previous training;

[0029] The server uses the first intermediate sub-decoder to process the received second encoding result to obtain fourth reconstructed feature data; the server uses the intermediate task execution network to process the fourth reconstructed feature data to obtain the first task execution result; the server calculates the second loss value based on the first task execution result and the task label corresponding to the second input data; the server reversely transmits the second gradient information based on the second loss value to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server;

[0030] The server sends the second gradient information to the edge device, so that the edge device adjusts the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information;

[0031] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-decoder is the first sub-decoder.

[0032] In conjunction with the first aspect, in one possible implementation, the method of the present application further includes:

[0033] The server trains the second sub-decoder,

[0034] The server trains the second sub-decoder, including:

[0035] The server receives a third encoding result from the edge device, where the third encoding result is obtained by the edge device encoding third feature data using the second intermediate sub-encoder; the third feature data is obtained by the edge device performing feature extraction on the third input data using the second feature extraction network; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; and during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters were adjusted during the previous training;

[0036] The server processes the received third encoding result according to the second intermediate sub-decoder to obtain fifth reconstructed feature data; the server processes the fifth reconstructed feature data using the second task execution network to obtain a second task execution result; the server calculates a third loss value based on the second task execution result and the task label corresponding to the third input data; the server reversely transmits third gradient information based on the third loss value to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the previous training;

[0037] The server sends third gradient information to the edge device, so that the edge device adjusts the weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0038] Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-decoder is the second sub-decoder.

[0039] By jointly training the sub-codec, semantic check extractor and semantic verifier in the above manner, the sub-codec, semantic check extractor and semantic verifier can be fully trained, which in turn helps to improve the accuracy of deciding whether to retransmit feature data when using the sub-codec, semantic check extractor and semantic verifier to execute the feature data transmission method of the present application.

[0040] In a second aspect, an embodiment of the present application provides another feature data transmission method, which is applied to edge devices.

[0041] The edge device obtains feature data F and extracts semantic verification information of the feature data F through the first semantic verification extractor; the edge device uses the first sub-encoder to perform t0 encoding operations on the feature data F to obtain t0 groups of encoding results; t0 is the number of initially transmitted data groups; the edge device transmits the semantic verification information and t0 groups of encoding results to the server.

[0042] Among them, the first sub-encoder is implemented based on SNN.

[0043] By sending the semantic verification information to the server, the server introduces channel state information and semantic verification information when determining the similarity between the reconstructed feature data and the feature data F, thereby improving the accuracy of the determined similarity and thus facilitating the accuracy of the decision result. In the existing scheme, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain different reconstructed feature data of the feature data, a plurality of encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side. This will increase the storage overhead and training overhead of the encoder and decoder. In the scheme of the present application, the sub-encoder and sub-decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is utilized each time encoding and decoding, which makes the encoding and decoding results obtained each time different. There is no need to set encoders and decoders with different parameters. The encoding side only needs to store a set of parameters of the sub-encoder, and the decoding side only needs to store a set of parameters of the sub-decoder, which greatly saves the overhead of training the encoder and decoder and the overhead of storing the encoder and decoder.

[0044] In conjunction with the second aspect, in one possible implementation, the semantic verification extraction includes a second feature extraction module and a quantification module. The server extracts semantic verification information of the feature data F through the semantic verification extractor, including:

[0045] The edge device performs feature extraction on the feature data F through the second feature extraction module; the edge device performs quantization processing on the feature vector extracted by the second feature extraction module through the quantization module to obtain semantic verification information of the feature data F.

[0046] In conjunction with the second aspect, in one possible implementation, the method of the present application further includes:

[0047] When the edge device receives the indication information sent by the server to instruct the retransmission of the characteristic data F, it uses the first sub-encoder to encode the characteristic data F to obtain the retransmitted encoding result; the number of retransmitted encoding results is determined based on the transmission resources indicated by the indication information; the edge device sends the retransmitted encoding result to the server.

[0048] By retransmitting the encoding result of the feature data F to the server, the server can obtain accurate reconstructed feature data, thereby facilitating the execution of subsequent tasks. In addition, the transmission resources used for retransmitting the encoding result are indicated through the indication information, thereby achieving the adjustment of the transmission bit rate.

[0049] In conjunction with the second aspect, in one possible implementation, the method of the present application further includes:

[0050] The edge device obtains the transmission resources allocated by the server and determines the number of initial transmission data groups t0 based on the allocated transmission resources. In this way, the transmission bit rate can be adjusted.

[0051] In conjunction with the second aspect, in one possible implementation, the method of the present application further includes:

[0052] The edge device is trained to obtain the first semantic verification extractor;

[0053] The edge device trains the first semantic verification extractor, including:

[0054] The edge device obtains first input data, and uses a first feature extraction network to perform feature extraction on the first input data to obtain first feature data; the edge device uses a first sub-encoder to encode the first feature data to obtain a first encoding result; the edge device uses an intermediate semantic verification extractor to process the first feature data to obtain semantic verification information of the first feature data; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the previous training; the first feature extraction network is obtained by joint training of the edge device and the server;

[0055] The edge device sends the first encoding result and semantic verification information of the first feature data to the server;

[0056] The edge device receives the first gradient information sent by the server; the first gradient information is determined by the server based on the first loss value, and the first loss value is calculated by the server based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the similarity between the third reconstructed feature data and the first feature data is obtained by the server using an intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information; the third reconstructed feature data is obtained by the server using a first sub-decoder to process the received first encoding result; the first gradient information is used by the server to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; when it is not the first training, the intermediate semantic verifier is the semantic verifier whose weight parameters were adjusted during the last training;

[0057] The edge device adjusts a weight parameter of an intermediate semantic check extractor based on the first gradient information;

[0058] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic check extractor and the weight parameters of the intermediate semantic checker converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor converge, the intermediate semantic check extractor obtained is the first semantic check extractor.

[0059] In conjunction with the second aspect, in one possible implementation, the method of the present application further includes:

[0060] The edge device trains a first sub-encoder and a first feature extraction network;

[0061] The edge device trains a first sub-encoder and a first feature extraction network, including:

[0062] The edge device obtains second input data, and the edge device uses the intermediate feature extraction network to extract features from the second input data to obtain second feature data; the edge device uses the intermediate sub-encoder to encode the second feature data to obtain a second encoding result; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters are adjusted during the previous training, and the intermediate feature extraction network is the feature extraction network whose weight parameters are adjusted during the previous training;

[0063] The edge device sends the second encoding result to the server;

[0064] The edge device receives the second gradient information sent by the server; the second gradient information is determined by the server based on the second loss value, and the second loss value is calculated by the server based on the task label corresponding to the first task execution result and the second input data; the first task execution result is obtained by the server using the intermediate task execution network to process the fourth reconstructed feature data; the fourth reconstructed feature data is obtained by the server using the first intermediate sub-decoder to process the received second encoding result; the second gradient information is used by the server to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server;

[0065] The edge device adjusts the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information;

[0066] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, and the obtained intermediate feature extraction network is the first feature extraction network.

[0067] In conjunction with the second aspect, in one possible implementation, the method of the present application further includes:

[0068] The edge device is trained to obtain the second sub-encoder;

[0069] The edge device trains the second sub-encoder, including:

[0070] The edge device obtains third input data, and uses the second feature extraction network to perform feature extraction on the third input data to obtain third feature data; the edge device uses the second intermediate sub-encoder to encode the third feature data to obtain a third encoding result; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the previous training;

[0071] The edge device sends the third encoding result to the server;

[0072] The edge device receives the third gradient information sent by the server; the third gradient information is determined by the server based on the third loss value, and the third loss value is calculated by the server based on the second task execution result and the task label corresponding to the third input data; the second task execution result is obtained by the server using the second task execution network to process the fifth reconstructed feature data; the fifth reconstructed feature data is obtained by the server using the second intermediate sub-decoder to process the received third encoding result; the third gradient information is used by the server to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the previous training;

[0073] The edge device adjusts a weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0074] Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-encoder is the second sub-encoder.

[0075] In a third aspect, an embodiment of the present application provides a server, comprising a unit or module for executing the method provided in the first aspect or any possible implementation manner of the first aspect.

[0076] In a fourth aspect, an embodiment of the present application provides an edge device, comprising a unit or module for executing the method provided in the second aspect or any possible implementation manner of the second aspect.

[0077] In a fifth aspect, an embodiment of the present application provides an apparatus comprising a processor and a memory. The memory is configured to store program code. The processor is configured to invoke the program code stored in the memory to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0078] In a sixth aspect, an embodiment of the present application provides an apparatus comprising a processor and a memory. The memory is configured to store program code. The processor is configured to call the program code stored in the memory to execute the method provided in the second aspect or any possible implementation of the second aspect.

[0079] In the seventh aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are run on an electronic device, the electronic device executes the method provided in any possible implementation of the first aspect, or the method provided in any possible implementation of the second aspect.

[0080] In an eighth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the method provided in any possible implementation of the first aspect, or the method provided in any possible implementation of the second aspect.

[0081] It can be understood that the server described in the third or fifth aspect provided above is used to execute any of the methods provided in the first aspect, the server described in the fourth or sixth aspect provided above is used to execute any of the methods provided in the second aspect, and the computer storage medium described in the seventh aspect and the computer program product described in the eighth aspect are both used to implement any of the methods provided in the first aspect or any of the methods provided in the second aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] FIG1a is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0083] FIG1b is a schematic diagram of a specific system architecture provided in an embodiment of the present application;

[0084] FIG2 is a flow chart of a method for transmitting characteristic data provided in an embodiment of the present application;

[0085] FIG3a is a schematic structural diagram of a reconstructor provided in an embodiment of the present application;

[0086] FIG3 b is a schematic structural diagram of a fusion device provided in an embodiment of the present application;

[0087] FIG4 is a schematic diagram of the structure of a semantic checker provided in an embodiment of the present application;

[0088] FIG5 is a flow chart of another method for transmitting characteristic data provided in an embodiment of the present application;

[0089] FIG6 is a schematic structural diagram of a semantic verification extractor provided in an embodiment of the present application;

[0090] FIG7 is a schematic diagram of a result of a sub-encoder provided in an embodiment of the present application;

[0091] FIG8 is a flow chart of another method for transmitting characteristic data provided by an embodiment of the present application;

[0092] FIG9 is a schematic diagram of another specific system architecture provided in an embodiment of the present application;

[0093] FIG10 is a schematic diagram of a joint training process provided by an embodiment of the present application;

[0094] FIG11 is a schematic diagram of a specific process of joint training provided in an embodiment of the present application;

[0095] FIG12 is a schematic diagram of another specific process of joint training provided in an embodiment of the present application;

[0096] FIG13 is a schematic diagram of another specific process of joint training provided in an embodiment of the present application;

[0097] FIG14 is a schematic structural diagram of a server provided in an embodiment of the present application;

[0098] FIG15 is a schematic structural diagram of an edge device provided in an embodiment of the present application;

[0099] FIG16 is a schematic diagram of the structure of another server provided in an embodiment of the present application;

[0100] FIG17 is a schematic structural diagram of another edge device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0101] The terms "first", "second", "third" and "fourth" in the specification, claims and drawings of this application are used to distinguish different objects rather than to describe a specific order.

[0102] "Multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating the existence of three relationships. For example, "A and / or B" means: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0103] The embodiments of the present application are described below with reference to the accompanying drawings.

[0104] Referring to Figure 1a, which is a schematic diagram of a system architecture provided by an embodiment of the present application, the system includes an edge device 101 and a server 102.

[0105] Among them, the edge device 101 can be a terminal device or a user device, such as a vehicle-mounted terminal, a desktop computer, a laptop computer, a tablet computer, a smart phone, a smart bracelet, etc.

[0106] The server 102 has a device for processing and storing data. For example, the server 102 is a single server, a server cluster, a cloud server, a cloud computing service center, or other forms of devices with computing capabilities.

[0107] After acquiring the feature data, edge device 101 encodes the feature data F to obtain an encoding result 1 of feature data F and extracts semantic verification information of feature data F. Edge device 101 transmits encoding result 1 of feature data F and verification data to server 102 via a binary symmetric channel (BSC). Server 102 decodes encoding result 1 of feature data F to obtain first reconstructed feature data. Based on the semantic verification information, channel state information, and the first reconstructed feature data, server 102 calculates a first similarity between the first reconstructed feature data and feature data F. Server 102 then transmits an indication message to edge device 101. When the first similarity is greater than the similarity threshold, the indication information is used to instruct edge device 101 that it does not need to retransmit feature data F. When the first similarity is not greater than the similarity threshold, the indication information is used to instruct edge device 101 that it needs to retransmit feature data F. Edge device 101 encodes feature data F to obtain encoding result 2 and sends encoding result 2 to server 102. Server 102 decodes encoding result 1 and encoding result 2 to obtain second reconstructed feature data. Server 102 calculates a second similarity based on semantic verification information, channel state information, and the second reconstructed feature data. This second similarity is the similarity between the second reconstructed feature data and feature data F. Server 102 again sends the indication information to edge device 101. When the second similarity is greater than the similarity threshold, the indication information is used to instruct edge device 101 that it does not need to retransmit feature data F. When the second similarity is not greater than the similarity threshold, the indication information is used to instruct edge device 101 that it needs to retransmit feature data F. The above process is repeated until the calculated similarity is greater than the similarity threshold, or until the maximum number of transmissions T is reached.

[0108] When the calculated similarity exceeds the similarity threshold or reaches the maximum number of transmissions T, the server performs a preset task based on the reconstructed feature data. In one example, the preset task is a vehicle recognition task, and the feature data F is the feature data of the vehicle; in another example, the preset task is an image classification task, and the feature data F is the feature data of the image.

[0109] In a specific example, as shown in FIG1 b , the edge device 101 is deployed with a semantic verification extractor and a sub-encoder; the server 102 is deployed with a sub-decoder and a semantic verifier.

[0110] Edge device 101 obtains feature data F from other devices, or performs feature extraction on data (such as image data, video data, etc.) to obtain feature data F. Edge device 101 extracts semantic verification information of feature data F through a semantic verification extractor, and encodes feature data F through a sub-encoder to obtain encoding result 1; edge device 101 transmits the semantic verification information and encoding result 1 to server 102 through BSC. Server 102 decodes encoding result 1 through a sub-decoder to obtain first reconstructed feature data; server 102 calculates a first similarity based on the semantic verification information, channel state information, and the first reconstructed feature data through a semantic verifier. The first similarity is the similarity between the first reconstructed feature data and feature data F. Server 102 sends indication information to edge device 101. When the first similarity is greater than the similarity threshold, the indication information is ACK, indicating that edge device 101 does not need to retransmit feature data F. When the first similarity is not greater than the similarity threshold, the indication information is NACK, indicating that edge device 101 needs to retransmit feature data F. Edge device 101 encodes feature data F to obtain encoding result 2 and sends encoding result 2 to server 102 via the BSC. Server 102 decodes encoding result 1 and encoding result 2 using a subdecoder to obtain second reconstructed feature data. Server 102 calculates a second similarity based on the semantic verification information, channel state information, and the second reconstructed feature data using a semantic checker. This second similarity is the similarity between the second reconstructed feature data and feature data F. Server 102 sends an indication information to edge device 101 again. When the second similarity is greater than the similarity threshold, the indication information is ACK, indicating that edge device 101 does not need to retransmit feature data F. When the second similarity is not greater than the similarity threshold, the indication information is NACK, indicating that edge device 101 needs to retransmit feature data F. The above process is repeated until the calculated similarity is greater than the similarity threshold, or until the maximum number of transmission times T is reached.

[0111] It should be understood that for the edge device, after encoding the feature data F, the encoding result will be transmitted to the server. Therefore, the maximum number of transmissions T in this application can also be understood as the maximum number of times the edge device encodes the feature data F.

[0112] It can be seen that when deciding whether to retransmit feature data, channel state information is introduced, which improves the accuracy of semantic verification, thereby improving the accuracy of the decision on whether to retransmit data, and can adapt to different channel scenarios.

[0113] It should be understood that the feature data is only an example. Of course, the edge device 101 can also transmit other data to the server 102 based on the above method, such as image data, text data, video data, and audio data.

[0114] The following describes in detail the process of the circuit performance prediction algorithm of this application.

[0115] Referring to FIG. 2 , FIG. 2 is a flow chart of a method for transmitting characteristic data according to an embodiment of the present application. The method is applied to the server 102 in FIG. 1 a or FIG. 1 b. As shown in FIG. 2 , the method includes:

[0116] S201. The server receives t0 group encoding results and semantic verification information of feature data F from the edge device.

[0117] The encoding result of group t0 is obtained by the edge device using the sub-encoder to encode the feature data F according to the number of initially transmitted data groups t0. t0 is the number of initially transmitted data groups of the edge device, and t0 is an integer greater than 0. The number of initially transmitted data groups t0 is determined by the edge device based on the transmission resources allocated by the server.

[0118] S202: The server decodes the received encoding result of group t0 using a sub-decoder to obtain first reconstructed feature data.

[0119] In one possible implementation, the server processes each group of encoding results in the t0 group of encoding results to obtain membrane potential information and intermediate reconstructed feature data corresponding to each group of encoding results; wherein the dimension of the membrane potential information, the dimension of the intermediate reconstructed feature data, and the dimension of the feature data F are the same; the server splices the membrane potential information and the intermediate reconstructed feature data corresponding to the t0 group of encoding results respectively to obtain a spliced ​​result; the server obtains the first reconstructed feature data based on the splicing result.

[0120] The server processes the encoding result of the tth group to obtain the membrane potential information M corresponding to the encoding result of the tth group. t and intermediate reconstructed feature data S t ,include:

[0121] The server is based on the intermediate membrane potential information M' corresponding to the t-1 group encoding result t-1 Process the tth group of encoding results to obtain the intermediate membrane potential information M' corresponding to the tth group of encoding results t and intermediate feature data S' t ; Based on the membrane potential information M corresponding to the encoding result of the t-1 group t-1 Reconstruct the intermediate feature data S' t Processing is performed to obtain the membrane potential information M corresponding to the encoding result of group t t and intermediate reconstructed feature data S t ;

[0122] When t=1, the intermediate membrane potential information M' corresponding to the t-1th group of encoding results t-1and membrane potential information M t-1 Both are 0.

[0123] In one example, the sub-decoder is implemented based on a spiking neural network.

[0124] Specifically, the sub-decoder includes a reconstructor and a fusion unit. As shown in Figure 3a, the reconstructor includes a first convolutional layer, a batch normalization (BN) layer, a first integrate-fire (IF) layer, a second convolutional layer, a second batch normalization layer, and an integrate and hybrid fire (IHF) layer. The server processes the encoding results of the tth group through the first convolutional layer and the first batch normalization layer to obtain a first intermediate result; the server processes the intermediate membrane potential information M' corresponding to the encoding results of the t-1th group through the first integrate-fire layer. t-1 Process the first intermediate result to obtain the intermediate membrane potential information M' corresponding to the tth group of encoding results t and intermediate feature data S' t ; The server uses the second convolution layer and the second batch normalization layer to process the intermediate feature data S' t Processing is performed to obtain the second intermediate result; the membrane potential information M corresponding to the t-1 group encoding result is obtained by integrating and mixing the issuing layer t-1 The second intermediate result is processed to obtain the membrane potential information M corresponding to the t-th group encoding result. t and intermediate reconstructed feature data S t .

[0125] As shown in Figure 3b, the server uses the fusion device to splice the membrane potential information corresponding to the encoding results of the t0 group and the corresponding intermediate reconstructed feature data to obtain a first splicing result; then fills the first splicing result to a fixed length; finally, the server uses the fully connected network of the fusion device to process the filled first splicing result to obtain the first reconstructed feature data.

[0126] S203: The server uses a semantic verifier to calculate a first similarity based on the semantic verification information, the channel state information and the first reconstructed feature data.

[0127] The first similarity is the similarity between the first reconstructed feature data and the feature data F.

[0128] In one possible implementation, the semantic verifier includes a first feature extraction module and a similarity calculation module. The server extracts features from the first reconstructed feature data through the feature extraction module to obtain a first feature vector; the server fuses the first feature vector, semantic verification information, and channel state information to obtain a second feature vector; and the server processes the second feature vector through the similarity calculation module to obtain a first similarity.

[0129] Specifically, as shown in Figure 4, the first feature extraction module includes a third convolutional layer, a first activation layer, a fourth convolutional layer, a second activation layer, and a first global average pooling layer. The server performs dimensionality reduction processing on the first reconstructed feature data through the third convolutional layer, the first activation layer, the fourth convolutional layer, the second activation layer, and the first global average pooling layer to obtain a first feature vector.

[0130] In one example, the first activation layer, the second activation layer, the third activation layer, and the fourth activation layer are nonlinear rectified linear units (ReLU); and the fifth activation layer is a sigmoid function (Sigmoid).

[0131] As shown in Figure 4, the similarity calculation module includes a first fully connected layer, a third activation layer, a second fully connected layer, a fourth activation layer, a third fully connected layer, and a fifth activation layer. After obtaining the first feature vector, the server concatenates the first feature vector, the channel state information, and the semantic verification information to obtain a second feature vector. The server then processes the second feature vector sequentially through the first fully connected layer, the third activation layer, the second fully connected layer, the fourth activation layer, the third fully connected layer, and the fifth activation layer to obtain the first similarity.

[0132] Among them, the third activation layer and the fourth activation layer are nonlinear rectification units, and the fifth activation layer is a S-type function.

[0133] S204. The server sends indication information to the edge device based on the first similarity and the similarity threshold. The indication information is used to instruct the edge device whether to retransmit the feature data F.

[0134] When the first similarity is not greater than the similarity threshold, it means that the difference between the first reconstructed feature data and the feature data F is large, and the server sends an indication information to the edge device to instruct the edge device to retransmit the feature data F to the server; when the first similarity is greater than the similarity threshold, it means that the difference between the first reconstructed feature data and the feature data F is small, and the server sends an indication information to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0135] In one possible embodiment, the server receives a coding result retransmitted by the edge device, where the coding result is obtained by the edge device encoding the feature data F. Optionally, the number of retransmitted coding results is one or more groups. The multiple groups of retransmitted coding results are obtained by the edge device encoding the feature data F multiple times.

[0136] In one possible implementation, when the server sends an instruction to the edge device to retransmit the characteristic data F, it also sends the number of retransmissions to the edge device. Optionally, the number of retransmissions can be one or more. The number of retransmissions is determined by the server based on the transmission resources allocated to the edge device.

[0137] In one possible implementation, the server sends indication information to the edge device for instructing the edge device to retransmit characteristic data F, and the indication information is also used to indicate transmission resource information for retransmission, so that the edge device determines the amount of retransmitted data based on the transmission resource information for retransmission.

[0138] After the server receives the retransmitted encoding result, the server uses the reconstructor to calculate the intermediate membrane potential information M' corresponding to the encoding result of the t0th group. t0 The retransmitted encoding result is processed to obtain the intermediate membrane potential information M' and the intermediate reconstructed feature data S' corresponding to the retransmitted encoding result; based on the membrane potential information M corresponding to the encoding result of the t0th group t The intermediate reconstructed feature data S' is processed to obtain the membrane potential information M and the intermediate reconstructed feature data S corresponding to the retransmitted encoding result.

[0139] When there are multiple sets of retransmitted coding results, the server uses the reconstructor to process the multiple sets of retransmitted coding results in the same way as the reconstructor processes the t0 set of coding results, so as to obtain the membrane potential information and intermediate reconstructed feature data corresponding to the multiple sets of retransmitted coding results. Among them, when processing the first set of retransmitted coding results, the server uses the reconstructor to process the intermediate membrane potential information M' corresponding to the t0 set of coding results. t0 The first set of retransmitted coding results is processed to obtain the intermediate membrane potential information and intermediate reconstruction feature data corresponding to the first set of retransmitted coding results; based on the membrane potential information M corresponding to the t0th set of coding results t The intermediate reconstructed feature data corresponding to the first set of retransmitted encoding results are processed to obtain membrane potential information and the intermediate reconstructed feature data corresponding to the first set of encoding results.

[0140] The server uses a fusion device to splice the membrane potential information and intermediate reconstructed feature data corresponding to the retransmitted encoding results with the first splicing result to obtain a second splicing result; then the second splicing result is padded to a fixed length; finally, the padded second splicing result is processed using a fully connected network to obtain second reconstructed feature data.

[0141] The server calculates a second similarity using a semantic checker based on the semantic check information, the channel state information, and the second reconstructed feature data; the first similarity is the similarity between the first reconstructed feature data and the feature data F. When the second similarity is not greater than a similarity threshold, it indicates that the second reconstructed feature data differs significantly from the feature data F, and the server continues to send an instruction message to the edge device to instruct the edge device to retransmit the feature data F to the server. When the second similarity is greater than the similarity threshold, it indicates that the second reconstructed feature data differs slightly from the feature data F, and the server sends an instruction message to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0142] The above process is repeated until the number of times the edge device transmits the encoding result of the feature data F reaches the maximum number of transmissions T or the server indicates that the edge device does not need to retransmit the feature data F.

[0143] The maximum number of transmission times T is determined based on the transmission resources allocated by the server.

[0144] It should be noted that the similarity threshold is related to the channel state. For example, the lower the signal-to-noise ratio of the channel, the higher the similarity threshold; for example, the larger the channel capacity, the higher the similarity threshold. This setting can further improve the accuracy of the reconstructed feature data obtained by the server.

[0145] It can be seen that when calculating the similarity between the reconstructed feature data and the original feature data, the introduction of channel state information is conducive to improving the accuracy of the similarity. When deciding whether the edge device needs to retransmit the feature data based on the similarity, it is conducive to improving the accuracy of the decision result. In the existing scheme, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain the encoding results of different feature data, a plurality of encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side. This will increase the storage overhead and training overhead of the encoder and decoder. In the scheme of the present application, the encoder and decoder are implemented based on a pulse neuron network. Based on the characteristics of the pulse neuron network, the membrane potential information of the previous moment is used each time encoding and decoding. This makes the encoding and decoding results obtained each time different, and there is no need to set encoders and decoders with different parameters. The encoding side only needs to store a set of parameters of the encoder, and the decoder only needs to store a set of parameters of the decoder, which greatly saves the overhead of training the codec and the overhead of storing the codec.

[0146] See Figure 5, which is a flow chart of a method for transmitting feature data according to an embodiment of the present application. The method is applied to the edge device 101 in Figure 1a or Figure 1b. As shown in Figure 5, the method includes:

[0147] S501. The edge device obtains semantic verification information of feature data F and the number of initially transmitted data groups t0.

[0148] In one possible implementation, the edge device requests transmission resources from the server, or the server directly allocates transmission resources to the edge device. It should be understood that the transmission resources here refer to symbols, and the edge device determines the number of initial transmission data groups, t0, based on the transmission resources allocated by the server. In this way, the transmission bit rate can be adjusted.

[0149] In one possible implementation, the edge device extracts verification information from feature data F using a semantic verification extractor. The semantic verification extractor includes a second feature extraction module and a quantization module. The feature extraction module performs feature dimensionality reduction on the feature data F to obtain a feature vector. The quantization module then quantizes the feature vector to obtain verification information for the feature data F.

[0150] As shown in Figure 6, the second feature extraction module includes a fifth convolution layer, a sixth activation layer, a sixth convolution layer, a seventh activation layer, and a second global average pooling layer; the edge device processes the feature data F in sequence through the fifth convolution layer, the sixth activation layer, the sixth convolution layer, the seventh activation layer, and the second global average pooling layer to obtain a feature vector.

[0151] Optionally, the quantization module is used to implement a uniform quantization function, a non-uniform quantization function, a hash function, or other quantization functions. The ninth activation layer and the tenth activation layer are both nonlinear rectification units.

[0152] S502: The edge device uses the sub-encoder to perform t0 encoding operations on the feature data to obtain t0 groups of encoding results.

[0153] In a possible embodiment, performing the i-th encoding operation includes:

[0154] The edge device uses the intermediate membrane potential information characteristic data F obtained from the i-1th encoding operation for processing to obtain the intermediate membrane potential information and intermediate encoding results of the i-th encoding operation, and processes the intermediate encoding results based on the membrane potential information corresponding to the i-1th encoding operation to obtain the i-th group of encoding results and the membrane potential information corresponding to the i-th encoding operation, where i is an integer greater than but not greater than t0.

[0155] When i=1, the intermediate membrane potential information obtained by performing the i-1th encoding operation and the membrane potential information corresponding to the i-1th encoding operation are both 0.

[0156] In one example, the sub-encoder is implemented based on SNN.

[0157] In a specific example, as shown in FIG7 , the sub-encoder includes a seventh convolutional layer, a third batch normalization layer, an eighth convolutional layer, a fourth batch normalization layer, a second integration and release layer, and a third integration and release layer. The edge device processes the feature data F through the seventh convolutional layer and the third batch normalization layer to obtain a third intermediate result; uses the third integration and release layer to process the third intermediate result based on the intermediate membrane potential information obtained from the i-1th encoding operation to obtain the intermediate membrane potential information and the intermediate encoding result of the i-th encoding operation; the edge device processes the third intermediate result through the eighth convolutional layer and the fourth batch normalization layer to obtain a fourth intermediate result; uses the fourth integration and release layer to process the fourth intermediate result based on the membrane potential information obtained from the i-1th encoding operation to obtain the membrane potential information of the i-th encoding operation and the i-th group of encoding results.

[0158] In one possible implementation, the edge device receives instruction information sent by the server. When the instruction information is used to instruct the edge device to retransmit the characteristic data F, the edge device determines the amount of data to be retransmitted based on the remaining transmission resources. For example, the edge device determines to retransmit one or more sets of encoding results of the characteristic data F to the server.

[0159] The edge device encodes the feature data F using a sub-encoder to obtain one or more sets of retransmitted encoding results.

[0160] It should be noted that the edge device encodes the feature data F using the intermediate membrane potential information obtained from the t0th encoding operation and the membrane potential information corresponding to the t0th encoding operation through the sub-encoder shown in Figure 7, thereby obtaining one or more sets of retransmission codes. The specific encoding process can be found in the description of S502 and will not be described here.

[0161] S503. The edge device transmits the semantic verification information and the t0 group encoding result to the server.

[0162] The edge device sends the semantic verification information and the t0 group encoding result to the server through the BSC.

[0163] In one possible implementation, the method of this embodiment further includes:

[0164] When the edge device receives the indication information sent by the server to instruct the retransmission of the characteristic data F, it uses the first sub-encoder to encode the characteristic data F to obtain the retransmitted encoding result; the number of retransmitted encoding results is determined based on the transmission resources indicated by the indication information; the edge device sends the retransmitted encoding result to the server.

[0165] It should be understood that when receiving the indication information for instructing the retransmission of the characteristic data F, the edge device determines whether the maximum number of transmissions has been reached. If the maximum number of transmissions T has not been reached, the edge device uses the first sub-encoder to encode the characteristic data F. If the maximum number of transmissions T has been reached, the edge device stops retransmitting the encoding results of the characteristic data F to the server, and does not perform encoding operations on the characteristic data F. There are one or more groups of encoding results retransmitted by the edge device to the server, which means that the edge device performs one or more encoding operations on the characteristic data F. The specific amount of data of the retransmitted encoding results is determined by the edge device based on the transmission resources indicated by the indication information. By determining the amount of data to be transmitted based on the transmission resources allocated by the server, dynamic adjustment of the transmission code rate is achieved. The specific process of the edge device encoding the characteristic data F to obtain the retransmitted encoding results can be found in the relevant description of S502 and will not be described here.

[0166] It should be noted that when the edge device performs one or more encoding operations on the feature data F to obtain the first set of encoding results for the retransmitted encoding results, the edge device processes the intermediate membrane potential information feature data F obtained from the t0th encoding operation to obtain the intermediate membrane potential information and intermediate encoding results of the first encoding operation in the one or more encoding operations. The edge device processes the intermediate encoding results based on the membrane potential information corresponding to the t0th encoding operation to obtain the first set of encoding results in the retransmitted encoding results and the membrane potential information corresponding to the first encoding operation in the one or more encoding operations. The subsequent encoding process can be found in the relevant description of S502 and will not be described again here.

[0167] It can be seen that in the solution of this embodiment, by sending the semantic verification information to the server, the server introduces channel state information and semantic verification information when determining the similarity between the reconstructed feature data and the feature data F, thereby improving the accuracy of the determined similarity and thus facilitating the accuracy of the decision result. In the existing solution, the encoder and decoder are implemented based on, for example, a convolutional neural network. In order to obtain different reconstructed feature data of the feature data, multiple encoders with different parameters are set on the encoding side, and correspondingly, decoders with different parameters are set on the decoding side. This increases the storage overhead and training overhead of the encoder and decoder. In the solution of the present application, the sub-encoder and sub-decoder are implemented based on a spiking neuron network. Based on the characteristics of the spiking neuron network, the membrane potential information of the previous moment is utilized each time encoding and decoding. This makes the encoding and decoding results obtained each time different, without the need to set up encoders and decoders with different parameters. The encoding side only needs to store a set of parameters for the sub-encoder, and the decoding side only needs to store a set of parameters for the sub-decoder, which greatly saves the overhead of training the encoder and decoder and the overhead of storing the encoder and decoder. By retransmitting the encoding result of the feature data F to the server, the server can obtain accurate reconstructed feature data, thereby facilitating the execution of subsequent tasks. Furthermore, the transmission resource used for retransmitting the coding result is indicated by the indication information, thereby achieving adjustment of the transmission code rate.

[0168] Referring to FIG8 , FIG8 is a flow chart of another method for transmitting characteristic data according to an embodiment of the present application. As shown in FIG8 , the method includes:

[0169] S801: Initial parameter configuration of edge devices and servers.

[0170] Specifically, the sub-encoder and semantic check extractor are deployed on the edge device, while the sub-decoder and semantic checker are deployed on the server. The edge device configures the structural parameters, weights, and maximum number of transmissions T for the sub-encoder and semantic check extractor, while the server configures the structural parameters, weights, similarity threshold, and maximum number of transmissions T for the sub-decoder and semantic checker.

[0171] S802. The edge device obtains feature data F and inputs the feature data F into SNN-SC-HARQ for JSCC transmission.

[0172] As shown in Figure 9, SNN-SC-HARQ mainly includes two modules: a semantic verification module composed of a semantic verification extractor (SimNet_E) deployed in the edge device and a semantic verifier (SimNet_D) deployed in the server, and a multi-rate codec (Multi-Rate SNN-SC) module composed of a sub-encoder based on SNN deployed in the edge device and a sub-decoder (reconstructor (Reconstructor) + converter (Converter)) based on SNN deployed in the server.

[0173] The edge device encodes the feature data F through the sub-encoder based on SNN to obtain multiple groups of encoding results of the feature data F. The specific encoding process can refer to the relevant description of the embodiment shown in Figure 5 and will not be described here. The number of encoding results here is determined by the edge device based on the transmission resources allocated by the server.

[0174] In one example, the feature data F is represented by a three-dimensional real number matrix of dimension c×h×w. Each group of encoding results in the multiple groups of encoding results has a dimension of a three-dimensional pulse matrix of c2×h2×w2, that is, the element values in this matrix are 0 or 1, and c2×h2×w2 < c×h×w.

[0175] In one example, the edge device obtains the feature data F from its local storage or other devices, or the edge device uses the feature extraction module to perform feature extraction on the input data to obtain the feature data F. The input data can be image data, audio data, text data or other types of data, which is not limited here.

[0176] S803. The edge device extracts the semantic verification information of the feature data F.

[0177] Among them, the edge device uses the semantic verification extractor to extract the semantic verification information S of the feature data F e , and the semantic verification information S e is of fixed length, such as 16 bits or 32 bits. The semantic verification information Se of the feature data F is a binary real number matrix of c1×1×1, and c1×1×1 << c×h×w, that is, the data volume of the semantic verification information is much less than the data volume of the transmitted feature data.

[0178] S804. The edge device obtains the transmission resources and sends t0 groups of encoding results and the semantic verification information S e to the server.

[0179] Specifically, the edge device sends a request to the server to obtain transmission resources. The server allocates transmission resources to the edge device based on the request, or the server directly allocates transmission resources to the edge device. The edge device determines to transmit the t0 group encoding result to the server based on the transmission resource, and then compares the t0 group encoding result with the semantic verification information S e The t0 group coding results are identified as Z1, Z2, ..., Z t0 .

[0180] S805, the server receives the t0 group encoding result and semantic verification information from the edge device, decodes the received t0 group encoding result to obtain the t0 group decoding result; obtains the first reconstructed feature data F based on the t0 group decoding result t0 .

[0181] Specifically, the server uses the reconstructor to encode the result of group t0 Decode to obtain the decoding results S1, M1, S2, M2, ..., S t0 ,M t0 , where membrane potential information S1, S2, ..., S t0 The server uses the fusion device to decode the t0 group of decoding results S1, M1, S2, M2, ..., S t0 ,M t0 Splicing is performed to obtain a first splicing result, the length of the first splicing result is B×t0; then the first splicing result is padded to a fixed length B×T, and the padded splicing result is processed to obtain the first reconstructed feature data F t0 The specific processing process can be found in the relevant description of the embodiment shown in FIG2 , which will not be described here. t0 is a three-dimensional real matrix.

[0182] S806: The server performs semantic verification on the first reconstructed feature data based on the semantic verification information and the channel state information to obtain a first similarity.

[0183] The server performs a semantic check on the first reconstructed feature data using the semantic check information and the channel state information to obtain a first similarity. The specific process is described in the embodiment shown in FIG2 and is not described here. The first similarity is used to represent the similarity between the first reconstructed feature data F and the feature data F, and this similarity can also be referred to as scalar soft information.

[0184] It is pointed out here that during the transmission process, due to reasons such as channel quality, there are differences between the t0 group coding result and semantic verification information transmitted by the edge device and the t0 group coding result and semantic verification information received by the server. Therefore, the server side marks the t0 group coding result as Semantic verification information is marked as

[0185] S807: The server sends indication information to the edge device according to the first similarity.

[0186] Among them, when the first similarity is not greater than the similarity threshold, it means that the difference between the first reconstructed feature data and the feature data F is large, and the server sends an indication information (such as NACK) to the edge device to instruct the edge device to retransmit the feature data F to the server; when the first similarity is greater than the similarity threshold, it means that the difference between the first reconstructed feature data and the feature data F is small, and the server sends an indication information (such as ACK) to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0187] In one possible implementation, when the server sends an instruction to the edge device to retransmit the characteristic data F, it also sends the number of retransmissions to the edge device. Optionally, the number of retransmissions can be one or more. The number of retransmissions is determined by the server based on the transmission resources allocated to the edge device.

[0188] In one possible implementation, the server sends indication information to the edge device for instructing the edge device to retransmit characteristic data F, and the indication information is also used to indicate transmission resource information for retransmission, so that the edge device determines the amount of retransmitted data based on the transmission resource information for retransmission.

[0189] S808. The edge device sends retransmission information to the server.

[0190] Specifically, after receiving the instruction information for instructing the edge device to retransmit the characteristic data F to the server, the edge device determines the number of retransmissions. Optionally, the edge device determines the number of retransmissions based on the remaining transmission resources, or the number of retransmissions is sent by the server. The edge device encodes the characteristic data F based on the number of retransmissions to obtain a retransmission encoding result, and the number of the retransmission encoding result is the number of retransmissions determined above. The specific encoding process can be found in the relevant description of the embodiment shown in Figure 5 and will not be described again here. The edge device sends the encoding result to the server.

[0191] When receiving the retransmitted coding result, the server decodes the retransmitted coding result to obtain a decoding result; based on the decoding result and the decoding result corresponding to the above t0 group of coding results, the server obtains the second reconstructed feature data; taking the retransmission of a group of coding results as an example, the decoding results of the group of coding results include S t0+1 ,M t0+1 ; The server will S t0+1 ,M t0+1 and S1,M1,S2,M2,…,S t0 ,M t0 Splice together to obtain a second splicing result; the length of the second splicing result is B×(t0+1); then fill the first splicing result to a fixed length of B×T, and process the padded splicing result to obtain the second reconstructed feature data F t0+1 .

[0192] The server uses a semantic checker to check the second reconstructed feature data F based on the semantic check information and the channel state information. t0+1 Perform semantic verification to obtain a second similarity. The second similarity is used to characterize the second reconstructed feature data F t0+1 The similarity between it and the feature data F.

[0193] When the second similarity is not greater than the similarity threshold, the server continues to send indication information to the edge device to instruct the edge device to retransmit the feature data F to the server; when the second similarity is greater than the similarity threshold, the server sends indication information to the edge device to instruct the edge device not to retransmit the feature data F to the server.

[0194] The above process is repeated until the number of times the edge device transmits the encoding result of the feature data F reaches the maximum number of transmissions T or the server indicates that the edge device does not need to retransmit the feature data F.

[0195] In one possible implementation, the server obtains the reconstructor, fuser, and semantic checker from the training device after training them. Alternatively, the server and the edge device can jointly train the sub-decoder and semantic checker. Simultaneously, the edge device and the server can jointly train the sub-encoder and semantic checker.

[0196] The specific training process is as follows, as shown in Figure 10:

[0197] S1. Parameter configuration of edge devices and servers involves the structural parameter configuration of the feature extraction network, task execution network, sub-codec, and semantic verification module, as well as the initial value configuration of the weight parameters of the feature extraction network, task execution network, sub-codec, and semantic verification module, the length of semantic verification information, the length of the encoding result, etc.

[0198] Among them, the semantic verification module includes a semantic verification extractor and a semantic verifier.

[0199] S2. Fix the weight parameters of the second feature extraction network deployed in the edge device and the weight parameters of the second task execution network deployed in the server, and the edge device and the server jointly train the second sub-encoder and the second sub-decoder.

[0200] Among them, the second feature extraction network and the second task execution network are obtained by server training.

[0201] S3, the edge device and the server jointly train the second feature extraction network, the second sub-encoder, the second sub-decoder, and the second task execution network.

[0202] S4. Fix the trained first feature extraction network, the first sub-encoder, and the first sub-decoder, and jointly train the semantic verification extractor and the semantic verifier.

[0203] The following describes the process of jointly training the sub-codec by the edge device and the server, as shown in Figure 11:

[0204] S21. The server trains the feature extraction network and the task execution network, deploys the second feature extraction network on the edge device, and deploys the second task execution network on the server. The server also deploys the untrained sub-encoder on the edge device and the untrained sub-decoder on the server.

[0205] S22. The edge device obtains input data, features the third input data through the second feature extraction network to obtain third feature data; uses the second intermediate sub-encoder to extract features from the feature data in the manner described in the embodiment shown in Figure 5 to obtain a third encoding result; the edge device transmits the third encoding result to the server.

[0206] During initial training, the second intermediate sub-encoder is an untrained sub-encoder. During non-initial training, the second intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the previous training. An untrained sub-encoder is one with randomly initialized parameters, and the random seed can be preset in advance.

[0207] S23. The server receives the third encoding result, and processes the received third encoding result using the second intermediate sub-decoder in the manner described in the embodiment shown in Figure 2 to obtain fifth reconstructed feature data; and inputs the fifth reconstructed feature data into the second task execution network for processing to obtain the second task execution result.

[0208] S24. The server calculates a third loss value based on the second task execution result and the task label corresponding to the third input data, backpropagates the first gradient information based on the third loss value to adjust the weight parameters in the second intermediate sub-decoder, and feeds back the first gradient information to the edge device.

[0209] It should be understood that during initial training, the second intermediate sub-decoder here is an untrained sub-decoder; during non-initial training, the second intermediate sub-decoder here is a sub-decoder after weight parameters have been adjusted during the previous training. An untrained sub-decoder refers to a sub-decoder with randomly initialized parameters, and the random seed can be preset in advance.

[0210] S25. The edge device adjusts the weight parameters of the second intermediate sub-encoder based on the third gradient information.

[0211] S26 , repeating the above steps S22 - S25 until the third training times reaches a third preset times, or the weight parameters of the second intermediate sub-encoding and the weight parameters of the second intermediate sub-decoding converge.

[0212] It should be pointed out here that when the third training times reaches the umbrella preset times, or the weight parameters of the second intermediate sub-encoding and the weight parameters of the second intermediate sub-decoding converge, the obtained second intermediate sub-encoder is the second sub-encoder, and the obtained second intermediate sub-decoder is the second sub-decoder.

[0213] The following describes in detail the process of the edge device and server jointly training the second feature extraction network, the second sub-encoder, the second sub-decoder, and the second task execution network, as shown in Figure 12:

[0214] S31. The edge device obtains second input data, features the second input data through the intermediate feature extraction network to obtain second feature data; uses the first intermediate sub-encoder to extract features from the second feature data in the manner described in the embodiment shown in Figure 5 to obtain a second encoding result; the edge device transmits the second encoding result to the server.

[0215] Among them, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the last training.

[0216] S32. The server receives the second encoding result, and processes the received second encoding result using the first intermediate sub-decoder in the manner described in the embodiment shown in Figure 2 to obtain fourth reconstructed feature data; and inputs the fourth reconstructed feature data into the intermediate task execution network for processing to obtain the first task execution result.

[0217] S33. The server calculates a second loss value based on the first task execution result and the task label corresponding to the second input data, backpropagates the second gradient information based on the second loss value to adjust the weight parameters in the first intermediate sub-decoder and the weight parameters of the intermediate task execution network, and feeds back the second gradient information to the edge device.

[0218] Among them, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server.

[0219] S34. The edge device adjusts the weight parameters of the first intermediate sub-encoder based on the second gradient information.

[0220] It should be understood that during initial training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network. During non-initial training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the previous training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the previous training. The second feature extraction network and the second task execution network are trained by the server.

[0221] S35. Repeat the above steps S31-S34 until the second training times reaches the second preset times, or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge.

[0222] It should be pointed out here that when the second training times reaches the second preset times, or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, the obtained first intermediate sub-decoder is the first sub-decoder, the obtained intermediate feature extraction network is the first feature extraction network, and the obtained intermediate task execution network is the first task execution network.

[0223] The following describes the process of jointly training the semantic verification extractor and semantic verifier on the edge device and server, as shown in Figure 13:

[0224] S41. Based on the trained first feature extraction network, the first sub-encoder and the first sub-decoder, an untrained semantic verification extractor is deployed on the edge device side, and an untrained semantic verifier is deployed on the server side.

[0225] S42. The edge device obtains first input data, features the first input data through a first feature extraction network to obtain first feature data; features the first feature data are extracted using a first sub-encoder in the manner described in the embodiment shown in Figure 5 to obtain a first encoding result; the edge device uses an intermediate semantic verification extractor to extract semantic verification information of the first feature data, and the edge device transmits the first encoding result and the semantic verification information of the first feature data to the server.

[0226] During initial training, the intermediate semantic check extractor is an untrained semantic check extractor. During non-initial training, the intermediate semantic check extractor is the semantic check extractor with weight parameters adjusted during the previous training. An untrained semantic check extractor is one with randomly initialized parameters, and the random seed can be preset in advance.

[0227] S43. The server receives the first encoding result, and processes the received first encoding result using the first sub-decoder in the manner described in the embodiment shown in Figure 2 to obtain third reconstructed feature data; and uses the intermediate semantic verifier to verify the first reconstructed feature data based on the semantic verification information and channel state information of the first feature data to obtain verification soft information.

[0228] The verification soft information is the similarity between the first feature data and the third reconstructed feature data.

[0229] S44. The server calculates a first loss value based on the semantic similarity label of the feature data corresponding to the verification soft information and the first input data, backpropagates the first gradient information based on the third loss value to adjust the weight parameters in the intermediate semantic verifier, and feeds back the first gradient information to the edge device.

[0230] During initial training, the intermediate semantic verifier is an untrained semantic verifier. During non-initial training, the intermediate semantic verifier is a semantic verifier with weight parameters adjusted during the previous training. An untrained semantic verifier is one with randomly initialized parameters, and the random seed can be preset in advance.

[0231] S45. The edge device adjusts the weight parameters of the intermediate semantic check extractor based on the third gradient information.

[0232] S46. Repeat the above steps S42-S45 until the first training times reach the first preset times, or the weight parameters of the intermediate semantic verification extractor and the weight parameters of the intermediate semantic verifier converge.

[0233] When the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor converge, the intermediate semantic check extractor obtained is the first semantic check extractor, and the intermediate semantic checker obtained is the first semantic checker.

[0234] By jointly training the sub-codec, semantic check extractor and semantic verifier in the above manner, the sub-codec, semantic check extractor and semantic verifier can be fully trained, which in turn helps to improve the accuracy of deciding whether to retransmit feature data when using the sub-codec, semantic check extractor and semantic verifier to execute the feature data transmission method of the present application.

[0235] FIG14 is a schematic diagram illustrating the structure of a server according to an embodiment of the present application. Server 1400 is server 102 in FIG1a or FIG1b. As shown in FIG14 , server 1400 includes a transceiver unit 1401, a decoder unit 1402, a calculation unit 1403, an allocation unit 1404, and a training unit 1405.

[0236] The transceiver unit 1401 is configured to receive semantic verification information and t0 group encoding results of feature data F from an edge device, where the t0 group encoding results are obtained by encoding the feature data F using a sub-encoder; t0 is the number of initially transmitted data groups of the edge device, and t0 is an integer greater than 0;

[0237] The decoding unit 1402 is configured to decode the received t0 group of encoding results using a sub-decoder to obtain first reconstructed feature data;

[0238] A calculation unit 1403 is configured to calculate, using a semantic checker based on the semantic check information, the channel state information, and the first reconstructed feature data, a first similarity, where the first similarity is a similarity between the first reconstructed feature data and the feature data F;

[0239] The transceiver unit 1401 is further configured to send indication information to the edge device based on the first similarity and the similarity threshold, where the indication information is used to instruct the edge device whether to retransmit the feature data F.

[0240] Among them, the sub-encoder and sub-decoder are implemented based on SNN.

[0241] In one possible implementation, when the first similarity is not greater than the similarity threshold, the indication information is used to instruct the edge device to retransmit the feature data F; when the first similarity is greater than the similarity threshold, the indication information is used to instruct the edge device not to retransmit the feature data F.

[0242] Furthermore, the indication information used to instruct the edge device to retransmit the characteristic data F is also used to indicate the transmission resources used to retransmit the characteristic data F.

[0243] In one possible implementation, the semantic verifier includes a first feature extraction module and a similarity calculation module, and the calculation unit 1403 is specifically configured to:

[0244] The first reconstructed feature data is subjected to feature extraction by a first feature extraction module to obtain a first feature vector; the second feature vector, semantic verification information and channel state information are fused to obtain a second feature vector; and the second feature vector is processed by a similarity calculation module to obtain a first similarity.

[0245] In one possible implementation, the server 1400 further includes:

[0246] The allocating unit 1404 is configured to allocate transmission resources to the edge device. The number of initially transmitted data groups t0 is determined by the edge device based on the allocated transmission resources.

[0247] In one possible implementation, the server 1400 further includes:

[0248] A training unit 1405 is configured to train a first semantic checker;

[0249] The training unit 1405 is specifically configured to:

[0250] Receive the first encoding result and semantic verification information of the first feature data from the edge device, the first encoding result is obtained by the edge device encoding the first feature data using the trained first sub-encoder, and the first semantic verification information is obtained by the edge device processing the first feature data using the intermediate semantic verification extractor; the first feature data is obtained by the edge device extracting features from the first input data using the first feature extraction network; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training between the edge device and the server 1400; according to the first sub-encoder The decoder processes the received first encoding result to obtain third reconstructed feature data; uses the intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information to obtain the similarity between the third reconstructed feature data and the first feature data; calculates a first loss value based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the server reversely transmits the first gradient information based on the first loss value to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier with adjusted weight parameters during the previous training;

[0251] The transceiver unit 1401 is further configured to send first gradient information to the edge device, so that the edge device adjusts a weight parameter of the intermediate semantic check extractor based on the first gradient information;

[0252] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge, the intermediate semantic verifier obtained is the first semantic verifier.

[0253] In a possible implementation, the training unit 1405 is further configured to train and obtain a first sub-decoder;

[0254] In terms of training to obtain the first sub-decoder, the training unit 1405 is specifically configured to:

[0255] Receive a second encoding result from the edge device, the second encoding result is obtained by the edge device encoding the second feature data using the first intermediate sub-encoder; the second feature data is obtained by the edge device extracting features from the second input data using the intermediate feature extraction network; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters are adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters are adjusted during the last training; use the first intermediate sub-decoder to process the received second encoding result to obtain fourth reconstructed feature data; use the intermediate task execution network to process the fourth reconstructed feature data to obtain the fourth reconstructed feature data. A task execution result; a second loss value is calculated based on the first task execution result and the task label corresponding to the second input data; second gradient information is reversely transmitted based on the second loss value to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are trained by the server 1400; the second sub-encoder and the second sub-decoder are jointly trained by the edge device and the server 1400;

[0256] The transceiver unit 1401 is further configured to send second gradient information to the edge device, so that the edge device adjusts the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information;

[0257] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-decoder is the first sub-decoder.

[0258] In a possible implementation, the training unit 1405 is further configured to train a second sub-decoder.

[0259] In terms of training to obtain the second sub-decoder, the training unit 1405 is specifically configured to:

[0260] Receive a third encoding result from the edge device, where the third encoding result is obtained by the edge device encoding the third feature data using the second intermediate sub-encoder; the third feature data is obtained by the edge device performing feature extraction on the third input data using the second feature extraction network; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; when it is not the first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters were adjusted during the last training; process the received third encoding result according to the second intermediate sub-decoder to obtain fifth reconstructed feature data; process the fifth reconstructed feature data using the second task execution network to obtain the second task execution result; calculate a third loss value based on the task label corresponding to the second task execution result and the third input data; the server reversely transmits third gradient information based on the third loss value to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; when it is not the first training, the second intermediate sub-decoder is a sub-decoder whose weight parameters were adjusted during the last training;

[0261] The transceiver unit 1401 is further configured to send third gradient information to the edge device, so that the edge device adjusts the weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0262] Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-decoder is the second sub-decoder.

[0263] By jointly training the sub-codec, semantic check extractor and semantic verifier in the above manner, the sub-codec, semantic check extractor and semantic verifier can be fully trained, which in turn helps to improve the accuracy of deciding whether to retransmit feature data when using the sub-codec, semantic check extractor and semantic verifier to execute the feature data transmission method of the present application.

[0264] It is worth noting that, for the specific functional implementation of the server 1400, please refer to the description of the characteristic data transmission method shown in FIG2 above, for example, the decoding unit 1402 is used to implement the relevant content of executing S202, the calculation unit 1403 is used to implement the relevant content of executing S203, and the transceiver unit 1401, the distribution unit 1404, and the training unit 1405 are used to implement the relevant content of executing S201 and S204. Each unit or module in the server 1400 can be individually or completely merged into one or more other units or modules to form a structure, or one (or some) of the units or modules can be further divided into multiple smaller units or modules in terms of function to form a structure, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), or the functions of multiple units (or modules) are implemented by one unit (or module).

[0265] FIG15 is a schematic diagram illustrating the structure of an edge device according to an embodiment of the present application. Edge device 1500 is edge device 101 in FIG1a or FIG1b. As shown in FIG15 , edge device 1500 includes an acquisition unit 1501, a semantic extraction unit 1502, an encoding unit 1503, a transceiver unit 1504, and a training unit 1505.

[0266] The acquisition unit 1501 is used to acquire feature data F.

[0267] The semantic extraction unit 1502 is configured to extract semantic verification information of the feature data F through a first semantic verification extractor;

[0268] The encoding unit 1503 is configured to perform t0 encoding operations on the feature data F using the first sub-encoder to obtain t0 sets of encoding results, where t0 is the number of initially transmitted data sets.

[0269] The transceiver unit 1504 is configured to transmit the semantic verification information and the t0 group encoding result to the server.

[0270] In conjunction with the second aspect, in one possible implementation, the semantic verification extraction includes a second feature extraction module and a quantification module, and the semantic extraction unit is specifically configured to:

[0271] The feature data F is subjected to feature extraction by a second feature extraction module; and the feature vector extracted by the second feature extraction module is subjected to quantization processing by a quantization module to obtain semantic verification information of the feature data F.

[0272] In conjunction with the second aspect, in one possible implementation, the encoding unit 1503 is further configured to: when the transceiver unit 1504 receives instruction information sent by the server for instructing retransmission of the feature data F, encode the feature data F using the first sub-encoder to obtain a retransmitted encoding result; the number of retransmitted encoding results is determined based on the transmission resources indicated by the instruction information;

[0273] The transceiver unit 1504 is further configured to send the retransmitted encoding result to the server.

[0274] In conjunction with the second aspect, in one possible implementation, the acquiring unit 1501 is further configured to:

[0275] Obtain the transmission resources allocated by the server; and determine the number of initial transmission data groups t0 based on the allocated transmission resources.

[0276] In conjunction with the second aspect, in one possible implementation, the edge device 1500 further includes:

[0277] A training unit 1505 is configured to train a first semantic verification extractor;

[0278] The training unit 1505 is specifically used for:

[0279] Obtain first input data, perform feature extraction on the first input data using a first feature extraction network to obtain first feature data; encode the first feature data using a first sub-encoder to obtain a first encoding result; process the first feature data using an intermediate semantic check extractor to obtain semantic check information of the first feature data; wherein, during the first training, the intermediate semantic check extractor is an untrained semantic check extractor; during non-first training, the intermediate semantic check extractor is a semantic check extractor with weight parameters adjusted during the previous training; the first feature extraction network is obtained by joint training of the edge device 1500 and the server;

[0280] The transceiver unit is further used to send the first encoding result and the semantic verification information of the first feature data to the server; receive the first gradient information sent by the server; the first gradient information is determined by the server based on the first loss value, and the first loss value is calculated by the server based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the similarity between the third reconstructed feature data and the first feature data is obtained by the server using an intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information; the third reconstructed feature data is obtained by the server using a first sub-decoder to process the received first encoding result; the first gradient information is used by the server to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; when it is not the first training, the intermediate semantic verifier is the semantic verifier whose weight parameters were adjusted during the last training;

[0281] The training unit 1505 is further configured to adjust a weight parameter of the intermediate semantic check extractor based on the first gradient information;

[0282] Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic check extractor and the weight parameters of the intermediate semantic checker converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor converge, the intermediate semantic check extractor obtained is the first semantic check extractor.

[0283] In conjunction with the second aspect, in one possible implementation, the training unit 1505 is further configured to train a first sub-encoder and a first feature extraction network;

[0284] In terms of training the first sub-encoder and the first feature extraction network, the training unit 1505 is specifically configured to:

[0285] Obtaining second input data, performing feature extraction on the second input data using the intermediate feature extraction network to obtain second feature data; encoding the second feature data using the intermediate sub-encoder to obtain a second encoding result; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the previous training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the previous training;

[0286] The transceiver unit 1504 is also used to send the second encoding result to the server; receive the second gradient information sent by the server; the second gradient information is determined by the server based on the second loss value, and the second loss value is calculated by the server based on the task label corresponding to the first task execution result and the second input data; the first task execution result is obtained by the server using the intermediate task execution network to process the fourth reconstructed feature data; the fourth reconstructed feature data is obtained by the server using the first intermediate sub-decoder to process the received second encoding result; the second gradient information is used by the server to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by server training; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server;

[0287] The training unit 1505 is further configured to adjust the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information;

[0288] Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, and the obtained intermediate feature extraction network is the first feature extraction network.

[0289] In one possible implementation, the training unit 1505 is further configured to train and obtain a second sub-encoder;

[0290] In terms of training to obtain the second sub-encoder, the training unit 1505 is specifically configured to:

[0291] Obtaining third input data, performing feature extraction on the third input data using the second feature extraction network to obtain third feature data; encoding the third feature data using the second intermediate sub-encoder to obtain a third encoding result; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder with weight parameters adjusted during the previous training;

[0292] The transceiver unit is further used to send a third encoding result to the server; receive third gradient information sent by the server; the third gradient information is determined by the server based on a third loss value, and the third loss value is calculated by the server based on the second task execution result and the task label corresponding to the third input data; the second task execution result is obtained by the server using the second task execution network to process the fifth reconstructed feature data; the fifth reconstructed feature data is obtained by the server using the second intermediate sub-decoder to process the received third encoding result; the third gradient information is used by the server to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the previous training;

[0293] The training unit 1505 is further configured to adjust a weight parameter of the second intermediate sub-encoder based on the third gradient information;

[0294] Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder converge, the obtained second intermediate sub-encoder is the second sub-encoder.

[0295] It is worth noting that the specific functional implementation of the edge device 1500 refers to the description of the feature data transmission method shown in Figure 5 above. For example, the acquisition unit 1501 and the semantic extraction unit 1502 are used to implement the relevant content of executing S501, the encoding unit 1503 is used to implement the relevant content of executing S502, and the transceiver unit 1504 and the training unit 1505 are used to implement the relevant content of executing S503. Each unit or module in the edge device 1500 can be individually or completely combined into one or more other units or modules to form a structure, or one (or some) of the units or modules can be further divided into multiple functionally smaller units or modules to form a structure, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), or the functions of multiple units (or modules) are implemented by one unit (or module).

[0296] Based on the description of the above method embodiments and related device embodiments, please refer to Figure 16 , which shows a schematic diagram of the structure of a server 1600 provided in an embodiment of the present invention. The server 1600 shown in Figure 16 includes a memory 1601, a processor 1602, a communication interface 1603, and a bus 1604. The memory 1601, processor 1602, and communication interface 1603 are interconnected via bus 1604.

[0297] Optionally, the memory 1601 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0298] The memory 1601 can store programs. When the program stored in the memory 1601 is executed by the processor 1602, the processor 1602 and the communication interface 1603 are used to perform the various steps of the characteristic data transmission method of the embodiment shown in FIG. 2 .

[0299] The processor 1602 adopts a general CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to perform the characteristic data transmission method of the embodiment shown in FIG. 2 .

[0300] Processor 1602 can also be an integrated circuit chip with signal processing capabilities. During implementation, each step of the characteristic data transmission method shown in Figure 2 can be completed by hardware integrated logic circuits in processor 1602 or software instructions. Optionally, processor 1602 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor is a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The optional software module is located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 1601, and the processor 1602 reads the information in the memory 1601 and combines its hardware to complete the functions required to be performed by the units included in the server 1400 of the embodiment of the present application, or executes the characteristic data transmission method of the embodiment shown in Figure 2.

[0301] The communication interface 1603 uses, for example but not limited to, a transceiver and other transceiver-related devices.

[0302] The bus 1604 may include a path for transmitting information between various components of the server 1600 (eg, the memory 1601 , the processor 1602 , and the communication interface 1603 ).

[0303] It should be noted that although the server 1600 shown in FIG16 only shows a memory, a processor, and a communication interface, in a specific implementation, those skilled in the art will understand that the server 1600 also includes other components necessary for normal operation. Furthermore, based on specific needs, those skilled in the art will understand that the server 1600 may also include hardware components that implement other additional functions. Furthermore, those skilled in the art will understand that the server 1600 may only include the components necessary to implement the embodiments of the present application, and does not necessarily need to include all of the components shown in FIG16.

[0304] Based on the description of the above method embodiments and related device embodiments, please refer to Figure 17, which shows a schematic diagram of the structure of an edge device 1700 provided in an embodiment of the present invention. The edge device 1700 shown in Figure 17 includes a memory 1701, a processor 1702, a communication interface 1703, and a bus 1704. The memory 1701, processor 1702, and communication interface 1703 are connected to each other via bus 1704.

[0305] Optionally, the memory 1701 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0306] The memory 1701 can store programs. When the program stored in the memory 1701 is executed by the processor 1702, the processor 1702 and the communication interface 1703 are used to perform the various steps of the characteristic data transmission method of the embodiment shown in FIG. 5 .

[0307] The processor 1702 adopts a general CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to perform the characteristic data transmission method of the embodiment shown in Figure 5.

[0308] Processor 1702 can also be an integrated circuit chip with signal processing capabilities. During implementation, each step of the characteristic data transmission method of the embodiment shown in Figure 5 can be completed by hardware integrated logic circuits in processor 1702 or software instructions. Optionally, processor 1702 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor is a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The optional software module is located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 1701, and the processor 1702 reads the information in the memory 1701 and combines its hardware to complete the functions required to be performed by the units included in the edge device 1500 of the embodiment of the present application, or executes the characteristic data transmission method of the embodiment shown in Figure 5.

[0309] The communication interface 1703 uses, for example but not limited to, a transceiver and other transceiver-related devices.

[0310] The bus 1704 may include a path for transmitting information between various components of the edge device 1700 (eg, the memory 1701 , the processor 1702 , and the communication interface 1703 ).

[0311] It should be noted that although the edge device 1700 shown in FIG17 only shows a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the edge device 1700 also includes other devices necessary for normal operation. At the same time, based on specific needs, those skilled in the art should understand that the edge device 1700 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the edge device 1700 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include all the devices shown in FIG17.

[0312] An embodiment of the present application further provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to implement the image quality processing method of the embodiment of the present application.

[0313] Optionally, as an implementation method, the chip also includes a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the image quality processing method.

[0314] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or processor executes one or more steps in any of the above methods.

[0315] The present application also provides a computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute one or more steps in any of the above methods.

[0316] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions described by the various illustrative logic blocks, modules, and steps may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another (e.g., based on a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

[0317] By way of example, and not limitation, such computer-readable storage media include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are actually directed to non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), and Blu-ray disc. Disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above are also intended to be included within the scope of computer-readable media.

[0318] Instructions may be executed by one or more processors, such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described by the various illustrative logical blocks, modules, and steps described herein is provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0319] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the division of the unit is only a logical function division, and there are other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Optionally, the mutual coupling, direct coupling, or communication connection shown or discussed is an indirect coupling or communication connection through some interfaces, devices or units, such as electrical, mechanical or other forms.

[0320] Optionally, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., located in one place, or distributed across multiple network units. Some or all of the units may be selected based on actual needs to achieve the purpose of the solution of this embodiment.

[0321] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.

[0322] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for transmitting characteristic data, characterized in that: The method is applied to a server, and the method comprises: Receive semantic verification information and t0 group encoding results of feature data F from an edge device, where the t0 group encoding results are obtained by encoding the feature data F using a sub-encoder; t0 is the number of initial data groups transmitted by the edge device, and t0 is an integer greater than 0; Decoding the received t0 group encoding results using a first sub-decoder to obtain first reconstructed feature data; Using a first semantic checker to calculate a first similarity based on the semantic check information, the channel state information and the first reconstructed feature data, the first similarity being a similarity between the first reconstructed feature data and the feature data F; Based on the first similarity and the similarity threshold, indication information is sent to the edge device, where the indication information is used to indicate whether the edge device retransmits the feature data F.

2. The method according to claim 1, characterized in that When the first similarity is not greater than a similarity threshold, the indication information is used to instruct the edge device to retransmit the feature data F; When the first similarity is greater than the similarity threshold, the indication information is used to indicate that the edge device does not need to retransmit the feature data F.

3. The method according to claim 1 or 2, characterized in that: The first semantic checker includes a first feature extraction module and a similarity calculation module, and the first semantic checker is used to calculate the first similarity based on the semantic check information, the channel state information and the first reconstructed feature data, including: Performing feature extraction on the first reconstructed feature data by the first feature extraction module to obtain a first feature vector; fusing the first feature vector, the semantic check information and the channel state information to obtain a second feature vector; The second feature vector is processed by the similarity calculation module to obtain the first similarity.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Transmission resources are allocated to the edge device, and the number of initially transmitted data groups is determined by the edge device based on the allocated transmission resources.

5. The method according to any one of claims 1 to 4, characterized in that: The first sub-encoder and the first sub-decoder are implemented based on a spiking neural network (SNN).

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Training to obtain the first semantic verifier; The training to obtain the first semantic verifier includes: Receive a first encoding result and semantic verification information of first feature data from the edge device, wherein the first encoding result is obtained by the edge device encoding the first feature data using a trained first sub-encoder, and the first semantic verification information is obtained by the edge device processing the first feature data using an intermediate semantic verification extractor; the first feature data is obtained by the edge device performing feature extraction on first input data using a first feature extraction network; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server; The received first encoding result is processed according to the first sub-decoder to obtain third reconstructed feature data; the third reconstructed feature data, semantic verification information of the first feature data and channel state information are processed by an intermediate semantic verifier to obtain similarity between the third reconstructed feature data and the first feature data; a first loss value is calculated based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; first gradient information is reversely transmitted based on the first loss value to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; during non-first training, the intermediate semantic verifier is a semantic verifier with weight parameters adjusted during the last training; Sending the first gradient information to the edge device, so that the edge device adjusts the weight parameter of the intermediate semantic check extractor based on the first gradient information; Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic verifier and the weight parameters of the intermediate semantic verification extractor converge, the intermediate semantic verifier obtained is the first semantic verifier.

7. The method according to claim 6, characterized in that The method further comprises: Train to obtain the first sub-decoder; The training to obtain the first sub-decoder includes: Receive a second encoding result from the edge device, where the second encoding result is obtained by the edge device encoding second feature data using the first intermediate sub-encoder; the second feature data is obtained by the edge device extracting features from second input data using the intermediate feature extraction network; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder for adjusting weight parameters during the last training, and the intermediate feature extraction network is the feature extraction network for adjusting weight parameters during the last training; The received second encoding result is processed by the first intermediate sub-decoder to obtain fourth reconstructed feature data; the fourth reconstructed feature data is processed by the intermediate task execution network to obtain the first task execution result; a second loss value is calculated based on the first task execution result and the task label corresponding to the second input data; the second gradient information is reversely transmitted based on the second loss value to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; the intermediate task execution network is the task execution network whose weight parameters were adjusted during the last training; the second feature extraction network and the second task execution network are obtained by the server training; the second sub-encoder and the second sub-decoder are obtained by the joint training of the edge device and the server; Sending the second gradient information to the edge device, so that the edge device adjusts the weight parameters of the first intermediate sub-encoder and the weight parameters of the intermediate feature extraction network based on the second gradient information; Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the first intermediate sub-decoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-decoder is the first sub-decoder.

8. The method according to claim 7, characterized in that The method further comprises: The second sub-decoder is obtained by training, The step of training to obtain the second sub-decoder includes: Receive a third encoding result from the edge device, where the third encoding result is obtained by the edge device encoding third feature data using the second intermediate sub-encoder; the third feature data is obtained by the edge device performing feature extraction on third input data using the second feature extraction network; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder for which weight parameters are adjusted during the last training; The received third encoding result is processed according to the second intermediate sub-decoder to obtain fifth reconstructed feature data; the fifth reconstructed feature data is processed using the second task execution network to obtain a second task execution result; a third loss value is calculated based on the second task execution result and the task label corresponding to the third input data; the third gradient information is reversely transmitted based on the third loss value to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is a sub-decoder whose weight parameters were adjusted during the last training; Sending the third gradient information to the edge device, so that the edge device adjusts the weight parameter of the second intermediate sub-encoder based on the third gradient information; Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged, the obtained second intermediate sub-decoder is the second sub-decoder.

9. A method for transmitting characteristic data, characterized in that: Applied to the edge device side, the method further includes: Acquire feature data F, and extract semantic verification information of the feature data F through a first semantic verification extractor; Perform t0 encoding operations on the feature data F using the first sub-encoder to obtain t0 sets of encoding results; t0 is the number of initially transmitted data sets; The semantic verification information and the t0 group encoding result are transmitted to the server.

10. The method according to claim 9, characterized in that The semantic verification extraction includes a second feature extraction module and a quantification module, and the semantic verification information of the feature data F is extracted by the semantic verification extractor, including: Performing feature extraction on the feature data F by using the second feature extraction module; The feature vector extracted by the second feature extraction module is quantized by the quantization module to obtain semantic verification information of the feature data F.

11. The method according to claim 9 or 10, characterized in that: The method further comprises: When receiving the indication information sent by the server for instructing to retransmit the characteristic data F, the characteristic data F is encoded by using the first sub-encoder to obtain a retransmitted encoding result; the number of the retransmitted encoding results is determined based on the transmission resources indicated by the indication information; The retransmitted encoding result is sent to the server.

12. The method according to any one of claims 9 to 11, characterized in that: The method further comprises: Acquire the transmission resources allocated by the server; The number t0 of the initially transmitted data groups is determined based on the allocated transmission resources.

13. The method according to any one of claims 9 to 12, characterized in that: The first sub-encoder is implemented based on a spiking neural network (SNN).

14. The method according to any one of claims 9 to 13, characterized in that: The method further comprises: Training to obtain the first semantic verification extractor; The training to obtain the first semantic verification extractor includes: Obtain first input data, use the first feature extraction network to perform feature extraction on the first input data to obtain first feature data; use the first sub-encoder to encode the first feature data to obtain a first encoding result; use an intermediate semantic verification extractor to process the first feature data to obtain semantic verification information of the first feature data; wherein, during the first training, the intermediate semantic verification extractor is an untrained semantic verification extractor; during non-first training, the intermediate semantic verification extractor is a semantic verification extractor with weight parameters adjusted during the last training; the first feature extraction network is obtained by joint training of the edge device and the server; Sending the first encoding result and semantic verification information of the first feature data to the server; Receive the first gradient information sent by the server; the first gradient information is determined by the server based on a first loss value, and the first loss value is calculated by the server based on the similarity between the third reconstructed feature data and the first feature data and the semantic similarity scale of the feature data corresponding to the first input data; the similarity between the third reconstructed feature data and the first feature data is obtained by the server using an intermediate semantic verifier to process the third reconstructed feature data, the semantic verification information of the first feature data and the channel state information; the third reconstructed feature data is obtained by the server using the first sub-decoder to process the received first encoding result; the first gradient information is used by the server to adjust the weight parameters of the intermediate semantic verifier; wherein, during the first training, the intermediate semantic verifier is an untrained semantic verifier; when it is not the first training, the intermediate semantic verifier is a semantic verifier whose weight parameters were adjusted during the last training; adjusting a weight parameter of the intermediate semantic check extractor based on the first gradient information; Repeat the above process until the first training times reaches the first preset times or the weight parameters of the intermediate semantic check extractor and the weight parameters of the intermediate semantic checker are both converged; when the first training times reaches the first preset times or the weight parameters of the intermediate semantic checker and the weight parameters of the intermediate semantic check extractor are both converged, the intermediate semantic check extractor obtained is the first semantic check extractor.

15. The method according to claim 14, characterized in that The method further comprises: Train to obtain the first sub-encoder and the first feature extraction network; The training to obtain the first sub-encoder and the first feature extraction network includes: Obtain second input data, perform feature extraction on the second input data using an intermediate feature extraction network to obtain second feature data; encode the second feature data using an intermediate sub-encoder to obtain a second encoding result; wherein, during the first training, the first intermediate sub-encoder is the second sub-encoder, and the intermediate feature extraction network is the second feature extraction network; during non-first training, the first intermediate sub-encoder is the sub-encoder whose weight parameters were adjusted during the last training, and the intermediate feature extraction network is the feature extraction network whose weight parameters were adjusted during the last training; Sending the second encoding result to the server; Receive the second gradient information sent by the server; the second gradient information is determined by the server based on the second loss value, and the second loss value is calculated by the server based on the first task execution result and the task label corresponding to the second input data; the first task execution result is obtained by the server using the intermediate task execution network to process the fourth reconstructed feature data; the fourth reconstructed feature data is obtained by the server using the first intermediate sub-decoder to process the received second encoding result; the second gradient information is used by the server to adjust the weight parameters of the first intermediate sub-decoder and the weight parameters of the intermediate task execution network; wherein, during the first training, the first intermediate sub-decoder is the second sub-decoder, and the intermediate task execution network is the second task execution network; during non-first training, the first intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the previous training; the intermediate task execution network is the previous The second feature extraction network and the second task execution network are obtained by training the server; the second sub-encoder and the second sub-decoder are obtained by joint training of the edge device and the server; Adjusting a weight parameter of the first intermediate sub-encoder and a weight parameter of the intermediate feature extraction network based on the second gradient information; Repeat the above process until the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge; when the second training number reaches the second preset number or the weight parameters of the first intermediate sub-encoder, the weight parameters of the second intermediate sub-encoder, the weight parameters of the intermediate feature extraction network and the weight parameters of the intermediate task execution network all converge, the obtained first intermediate sub-encoder is the first sub-encoder, and the obtained intermediate feature extraction network is the first feature extraction network.

16. The method according to claim 15, characterized in that The method further comprises: Train to obtain the second sub-encoder; The training to obtain the second sub-encoder includes: Acquire third input data, perform feature extraction on the third input data using the second feature extraction network to obtain third feature data; encode the third feature data using the second intermediate sub-encoder to obtain a third encoding result; wherein, during the first training, the second intermediate sub-encoder is an untrained sub-encoder; during non-first training, the second intermediate sub-encoder is a sub-encoder whose weight parameters are adjusted during the last training; Sending the third encoding result to the server; Receive the third gradient information sent by the server; the third gradient information is determined by the server based on the third loss value, and the third loss value is calculated by the server based on the second task execution result and the task label corresponding to the third input data; the second task execution result is obtained by the server using the second task execution network to process the fifth reconstructed feature data; the fifth reconstructed feature data is obtained by the server using the second intermediate sub-decoder to process the received third encoding result; the third gradient information is used by the server to adjust the weight parameters of the second intermediate sub-decoder; wherein, during the first training, the second intermediate sub-decoder is an untrained sub-decoder; during non-first training, the second intermediate sub-decoder is the sub-decoder whose weight parameters were adjusted during the last training; Adjusting a weight parameter of the second intermediate sub-encoder based on the third gradient information; Repeat the above process until the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged; when the third training times reaches the third preset times or the weight parameters of the second intermediate sub-encoder and the weight parameters of the second intermediate sub-decoder are converged, the obtained second intermediate sub-encoder is the second sub-encoder.

17. A server, characterized in that: The method comprises a unit for executing the method according to any one of claims 1 to 8.

18. An edge device, characterized in that: The method comprises a unit for executing the method according to any one of claims 9 to 16.

19. A device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program codes, and the processor is used to execute the program codes to implement the method according to any one of claims 1 to 8.

20. A device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program codes, and the processor is used to execute the program codes to implement the method according to any one of claims 9 to 16.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 16 is implemented.

22. A computer program product, characterized in that When the computer program product runs on a computer, the method according to any one of claims 1 to 16 is implemented.

Images