The invention discloses a source-channel joint encoding and decoding method based on a cross-
modal communication
system, which relates to the technical field of cross-
modal image signal reconstruction, and comprises the following steps of: firstly, designing a cross-
modal source
encoder, inputting a tactile
signal with a category
label and an
image signal into the cross-modal source
encoder, and outputting the tactile
signal with the category
label and the
image signal with the category
label; and mining deep-
level fusion feature information between the modals. And secondly, designing a channel
encoder and a
channel decoder, and directly mapping the extracted fusion features into a bit
stream for channel transmission, thereby improving the
adaptive capacity of the
system to channel change. And finally, designing a corresponding information source decoder at a receiving end, and completing high-quality image
signal reconstruction from deep feature information. According to the source-channel joint coding and decoding method based on the cross-modal communication
system, the
power consumption in the network is reduced, the load balance between the cloud network and the
server (host) is measured by using the
fitness function, and the load balance problem is converted into the
optimization problem, so that the method can be suitable for
processing resource optimization. The result of the particle swarm is used as the initial
population of the
genetic algorithm, and the method achieves better effects in the aspects of execution cost, load balance and
completion time.