Semantic communication and perception integrated communication method and device, equipment, storage medium and product

Abstract

The invention discloses a semantic communication perception integrated communication method and device, equipment, a storage medium and a product, and relates to the technical field of data communication, and the method comprises the steps: constructing a semantic communication end-to-end distortion model which is used for simulating end-to-end semantic distortion, and the semantic distortion comprises source distortion and channel distortion; based on the semantic communication end-to-end distortion model, transmitting parameters corresponding to all codec models are determined, and all the codec models correspond to different information source rates; and semantic communication perception integrated communication is carried out based on the emission parameters. According to the method, the transmitting parameters corresponding to the codec models are determined based on the semantic communication end-to-end distortion model, and the codec models correspond to different information source rates; and semantic communication perception integrated communication is carried out based on the emission parameters. Compared with an existing communication mode in which communication or perception performance needs to be sacrificed to guarantee another performance, the mode of the invention can improve the semantic communication and perception integrated communication efficiency.

Description

Technical Field

[0001] This application relates to the field of data communication technology, and in particular to semantic communication and perception integrated communication methods, devices, equipment, storage media and products. Background Technology

[0002] Existing research combining semantic communication and sensing largely focuses on reducing transmission overhead by compressing sensing data, with less attention paid to system performance when semantic communication and sensing signals are transmitted within the same framework. Unlike traditional communication, which requires precise transmission of bit information, semantic communication extracts only task-related semantic features, thereby reducing transmission redundancy, improving resource utilization, and lowering feedback overhead. Furthermore, semantic communication exhibits stronger robustness to noise under low signal-to-noise ratio conditions, significantly outperforming traditional communication. Moreover, traditional integrated sensing and communication (ISAC) systems inherently involve performance trade-offs: under resource constraints, communication or sensing performance is typically sacrificed to ensure the performance of the other. Therefore, improving the communication efficiency of integrated sensing and sensing systems has become a pressing technical challenge. Summary of the Invention

[0003] The main purpose of this application is to provide a semantic communication and sensing integrated communication method, device, equipment, storage medium and product, which aims to solve the technical problem of low efficiency in existing semantic communication and sensing integrated communication.

[0004] To achieve the above objectives, this application proposes a semantic communication-aware integrated communication method, which includes: Construct a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, the semantic distortion including source distortion and channel distortion; Based on the semantic communication end-to-end distortion model, the transmission parameters corresponding to each codec model are determined, and each codec model corresponds to a different source rate. Based on the transmission parameters, semantic communication and perception are integrated.

[0005] Optionally, the semantic communication-aware integrated communication based on the transmission parameters includes: Based on the transmission parameters and the semantic communication end-to-end distortion model, determine the target transmission parameters corresponding to the minimum semantic distortion; Semantic communication and awareness are integrated based on the target transmission parameters and the corresponding codec model.

[0006] Optionally, determining the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model includes: The end-to-end distortion optimization problem is determined based on the semantic communication end-to-end distortion model. The end-to-end distortion optimization problem is decomposed into three sub-problems; The three sub-problems are solved iteratively using a convex optimization modeling toolkit to obtain the transmission parameters corresponding to each codec model.

[0007] Optionally, the end-to-end distortion optimization problem is decomposed into three sub-problems, including: The end-to-end distortion optimization problem is decomposed into a convex optimization problem with respect to channel rate and a problem of maximizing the communication signal-to-dryness ratio. The problem of maximizing the communication signal-to-dryness ratio is transformed into a standard semidefinite programming problem and a convex problem.

[0008] Optionally, the end-to-end distortion optimization problem is:

[0009] Among them, R s Used to characterize the source rate Rc represents the set of source rates corresponding to G semantic communication end-to-end distortion models, W0 represents the sensing beamforming vector, and W represents the channel rate. c Used to characterize semantic beamforming vectors and Used to characterize the fitting parameters, Used to characterize bit error rate Used to characterize the moisture-to-dryness ratio Current information drying ratio Channel capacity under certain conditions; HCRB is used to characterize the hybrid Cramer-Rao bound. Used to characterize the perceptual HCRB threshold Used to characterize the maximum number of times a channel can be used. The function Tr() is used to characterize the base station transmit power, and Tr() is used to characterize the trace operation.

[0010] Optionally, the transmission parameters include at least one of the following: channel rate, source rate, semantic beamforming vector, and sensing beamforming vector.

[0011] Furthermore, to achieve the above objectives, this application also proposes a semantic communication sensing integrated communication device, which includes: The construction module is used to construct a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, the semantic distortion including source distortion and channel distortion; The determination module is used to determine the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, wherein each codec model corresponds to a different source rate; The communication module is used for semantic communication and perception integration based on the transmission parameters.

[0012] In addition, to achieve the above objectives, this application also proposes a semantic communication sensing integrated communication device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the semantic communication sensing integrated communication method as described above.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the semantic communication and perception integrated communication method described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the semantic communication-aware integrated communication method described above.

[0015] This application constructs a semantic communication end-to-end distortion model, which simulates end-to-end semantic distortion, including source distortion and channel distortion. Based on the semantic communication end-to-end distortion model, transmission parameters corresponding to each semantic communication end-to-end distortion model are determined. Semantic communication and sensing integrated communication are then performed based on these transmission parameters. Since this application determines the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, each codec model corresponds to a different source rate; and semantic communication and sensing integrated communication are performed based on these transmission parameters, compared to existing communication methods that sacrifice communication or sensing performance to guarantee another performance characteristic, the method described in this application can improve the efficiency of semantic communication and sensing integrated communication. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the semantic communication and perception integrated communication method of this application. Figure 2 This is a schematic diagram of the system model provided in Embodiment 1 of the semantic communication and perception integrated communication method of this application; Figure 3 This is a flowchart illustrating Embodiment 2 of the semantic communication and perception integrated communication method of this application. Figure 4 This is a schematic diagram of the module structure of the semantic communication and perception integrated communication device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the semantic communication and perception integrated communication method in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application is as follows: Constructing a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, including source distortion and channel distortion; determining the transmission parameters corresponding to each semantic communication end-to-end distortion model based on the semantic communication end-to-end distortion model; and performing integrated semantic communication and sensing communication based on the transmission parameters. Since this application determines the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, each codec model corresponds to a different source rate; and performs integrated semantic communication and sensing communication based on the transmission parameters, compared to existing communication methods that sacrifice communication or sensing performance to guarantee another performance characteristic, the above method of this application can improve the efficiency of integrated semantic communication and sensing communication.

[0023] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or base station capable of performing the above functions. The following description uses a base station as an example to illustrate this embodiment and the subsequent embodiments.

[0024] Based on this, embodiments of this application provide a semantic communication-aware integrated communication method, referring to... Figure 1, Figure 1 This is a flowchart illustrating an embodiment of the semantic communication and perception integrated communication method of this application.

[0025] In this embodiment, the semantic communication-aware integrated communication method includes the following steps: Step S10: Construct a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, and the semantic distortion includes source distortion and channel distortion; It should be noted that the initial intention of this embodiment is to fill the technological gap in existing semantic communication systems, which lack an integrated transmission framework capable of simultaneously handling semantic communication and sensing tasks. Therefore, the semantic communication and sensing integrated communication method proposed in this embodiment is an adaptive source-channel coding method for Integrated Sensing and Semantic Communications (ISSC) systems. It combines semantic communication with ISSC, leveraging the high resource utilization advantage of semantic communication under low signal-to-noise ratio conditions to dynamically allocate more resources to sensing tasks, thereby improving system performance. Specifically, this embodiment considers an ISSC communication scenario, which includes a base station, a sensing target, a semantic receiver, and a sensing receiver. Figure 2 As shown, Figure 2 This is a schematic diagram of the system model provided in Embodiment 1 of the semantic communication and sensing integrated communication method of this application. Semantic source information is encoded by semantic source coding and channel coding to form a bitstream containing semantic information. Sensing and detection data is generated by a bitstream generator to form a bitstream containing sensing information. Both are modulated by the same module to form semantic and sensing symbols, and then beamforming and superposition coding are used to form an ISSC signal. The base station, acting as the transmitter of the ISSC signal, sends the ISSC signal to the sensing target and two receivers. The semantic receiver performs semantic decoding on the received signal and then uses the decoded data to complete semantic tasks (e.g., image reconstruction). The sensing receiver receives the echo signal reflected from the target and then samples the signal to achieve target localization. In this embodiment, by designing parameters such as source-channel coding rate and beamforming at the ISSC base station, end-to-end semantic distortion is minimized under the constraint of ensuring sensing accuracy thresholds. Figure 2 The meanings of the letters involved are as follows: b: semantic source encoding result; b c : Semantic information channel coding result; b s : Sensing and probing data bitstream; S c (t): Semantic information modulation result; S0(t): Sensing detection data modulation result; W c : Semantic beamforming vector; W0: Perceptual beamforming vector; y c : Semantic channel received signal; ys : Sensing channel receives signals.

[0026] In this embodiment, the base station models end-to-end semantic distortion by designing the source-channel coding rate and beamforming, obtaining the semantic communication end-to-end distortion model. It also derives the hybrid Cramér-Rao bound (HCRB) for the ISSC scenario considering clock errors. HCRB is the theoretical limit of any unbiased estimate; in this embodiment, it is an index used to characterize sensing errors, minimizing end-to-end semantic distortion under threshold constraints that guarantee sensing accuracy. End-to-end semantic distortion can be modeled as the sum of source distortion and channel distortion. Source distortion is determined by the feature extraction and recovery model (i.e., the semantic communication end-to-end distortion model) under different source coding rates, while channel distortion is related to the Lipshitz constant of the deep neural network and the channel coding method, making analytical solutions difficult. Therefore, this embodiment adopts a data regression method: first, multiple sets of data recovery models (i.e., semantic communication end-to-end distortion models) are pre-trained, and the source distortion is measured under ideal error-free transmission conditions; then, through simulated transmission under different bit error rates, an approximate expression for end-to-end distortion is obtained by fitting, thereby approximating the overall distortion. Among them, the approximate expression for end-to-end distortion can be written as the following formula (1): Formula (1) Among them, D o R is used to characterize end-to-end semantic distortion. s Used to characterize the source rate and Used to characterize the fitting parameters, Used to characterize bit error rate.

[0027] Step S20: Determine the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, wherein each codec model corresponds to a different source rate; It should be noted that determining the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model can be achieved by determining an end-to-end distortion optimization problem based on the semantic communication end-to-end distortion model, and then solving the end-to-end distortion optimization problem based on multiple codec models to obtain the solved transmission parameters corresponding to each codec model. The codec model is a pre-trained parameterizable source-channel joint coding-decoding pair model, with each codec model corresponding to a different source rate. The transmission parameters include the channel rate, source rate, semantic beamforming vector, and sensing beamforming vector. The end-to-end distortion optimization problem is constructed based on the end-to-end distortion approximation expression in the above formula (1) and the conditions of guaranteed power, maximum channel usage, and sensing accuracy threshold.

[0028] Step S30: Perform semantic communication and perception integrated communication based on the transmission parameters.

[0029] It should be noted that the semantic communication and awareness integrated communication based on the transmission parameters can be performed by determining the transmission parameters that minimize semantic distortion and the corresponding target codec model based on the transmission parameters corresponding to each codec model, and then performing semantic communication and awareness integrated communication based on the transmission parameters and the target codec model.

[0030] In this specific implementation, random channel coding is considered. It can be expressed as the following formula (2): Formula (2) in, , Indicates the current information drying ratio Under these conditions, the channel capacity and signal-to-dryness ratio are expressed as: h c Represents the semantic channel. R represents the semantic noise variance; c L represents the channel rate, and L represents the length of the symbol vector formed by modulation at the transmitter.

[0031] The base station transmits an ISSC signal, which is reflected by the target and reaches the sensing receiver. The sensing receiver samples the continuous signal and maps it to the frequency domain through DFT transformation. Therefore, the received signal corresponding to the k-th sampling point in the frequency domain can be expressed as the following formula (3): Formula (3) in, This represents the frequency domain component obtained by DFT transformation of the transmitted signal at k sampling points in the time domain. W represents the frequency domain angular frequency corresponding to the k-th sampling point, and W0 is used to characterize the sensing beamforming vector. c Used to characterize semantic beamforming vectors Indicates the sampling interval. and The frequency domain representations of the semantic and perceptual symbols after DFT transformation are represented as follows: and , N represents the number of sampling points. and H0 represents the nth semantic and sensing symbol formed by modulation at the transmitting end; H0 represents the channel parameters of the base station-target-sensing receiver, which are modeled as follows: , and These represent the receive and transmit steering vectors, respectively. Indicates the angle of arrival of the signal. This indicates the departure angle of the signal from the base station. This represents the reflection coefficient, which includes the effects of radar cross section and path loss; n s This represents complex symmetric circular Gaussian white noise. Representing transmission delay, it is modeled as , Indicates transmission delay. This indicates random clock error.

[0032] definition ,in This represents the position coordinates of the perceived target. Based on the above formula, the Fisher information matrix (FIM) of position and clock error is obtained as follows (4): Formula (4) in, , , ; After derivation, it can be expressed as follows: , , W represents the variance of the perceived noise. c W0 and W0 represent the semantic beamforming matrix (i.e., the semantic beamforming vector) and the perceptual beamforming matrix (perceptual beamforming vector), respectively. It can be represented as: , , , , , , , , and Representing the sensing channel about and The first derivative, and They represent about and The first derivative, in addition. , This represents the variance of the clock error.

[0033] The purpose of this embodiment is to minimize end-to-end distortion in the ISSC scenario while ensuring power, maximum channel usage, and sensing accuracy threshold. The end-to-end distortion optimization problem can be referred to the following formula (5): Formula (5) Among them, R s Used to characterize the source rate Let Rc represent the set of source rates corresponding to G codec models, W0 represent the sensing beamforming vector, and W represent the channel rate. c Used to characterize semantic beamforming vectors and Used to characterize the fitting parameters, Used to characterize bit error rate Used to characterize the moisture-to-dryness ratio Current information drying ratio Channel capacity under certain conditions; HCRB is used to characterize the hybrid Cramer-Rao bound. Used to characterize the perceptual HCRB threshold Used to characterize the maximum number of times a channel can be used. The function Tr() is used to characterize the base station transmit power, and Tr() is used to characterize the trace operation.

[0034] This embodiment constructs a semantic communication end-to-end distortion model, which simulates end-to-end semantic distortion, including source distortion and channel distortion. Based on the semantic communication end-to-end distortion model, transmission parameters corresponding to each semantic communication end-to-end distortion model are determined. Semantic communication and sensing integrated communication are then performed based on these transmission parameters. Since this embodiment determines the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, each codec model corresponds to a different source rate; semantic communication and sensing integrated communication are then performed based on these transmission parameters. Compared to existing communication methods that sacrifice communication or sensing performance to guarantee another performance characteristic, the above method in this embodiment can improve the efficiency of semantic communication and sensing integrated communication.

[0035] This embodiment presents an integrated sensing and semantic communications (ISSC) communication method that combines semantic communication with sensing. It is an adaptive channel source coding method that integrates semantic communication with sensing. An end-to-end distortion model of semantic communication is established and the hybrid Cramér-Rao bound (HCRB) under clock error is derived. By optimizing the source channel coding rate and beamforming parameters of the integrated sensing and semantic communications (ISSC) system, the system performance under limited resource conditions is studied.

[0036] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3This is a flowchart illustrating a second embodiment of the semantic communication and perception integrated communication method of this application. Step S20 further includes the following steps: Step S201: Determine the end-to-end distortion optimization problem based on the semantic communication end-to-end distortion model; It should be noted that the determination of the end-to-end distortion optimization problem based on the semantic communication end-to-end distortion model can be based on the end-to-end distortion approximation expression in the above formula (1) and the conditions of guaranteed power, maximum number of channel uses, and perception accuracy threshold. For details, please refer to the above formula (5).

[0037] Step S202: Decompose the end-to-end distortion optimization problem into three sub-problems; It should be noted that decomposing the end-to-end distortion optimization problem into three sub-problems can be decomposing the end-to-end distortion optimization problem into a convex optimization problem with respect to the channel rate and a problem of maximizing the communication signal-to-dryness ratio. The problem of maximizing the communication signal-to-dryness ratio is transformed into a standard semidefinite programming problem and a convex problem.

[0038] Step S203: Solve the three sub-problems alternately in an iterative manner using a convex optimization modeling toolkit to obtain the transmission parameters corresponding to each codec model.

[0039] Specifically, first, we obtain information about one base station, one target, and two receivers in the ISSC system; then, we obtain the direct channel coefficient h from the base station to the semantic receiver. c The channel coefficient H0 of the base station-target-semantic receiver, and the departure angle of the base station. Angle of arrival of the sensing receiver Reflectance coefficient, including the effects of radar cross section and path loss Gaussian noise power Signal sampling number N, signal bandwidth B, number of base station transmit antennas N T The number of receiving antennas N of the sensing receiver R Base station transmit power P T HCRB threshold perception ; Based on the above formulas (3) and (4), the sensing HCRB in the ISSC system is derived. The model of the received signal is required to derive the HCRB, which is formula (3). Formula (4) is a necessary step to derive the HCRB. Formula 4 is the Fisher information matrix, and its inverse is the HCRB. Therefore, the derived sensing HCRB in the ISSC system includes each parameter in formula (4).

[0040] Pre-train G groups of deep learning-based codecs (i.e., codec models) on the dataset (communication sample data) to obtain the parameters of the neural network and the compression ratio (i.e., channel rate) corresponding to different encoders (i.e. codec models) in G groups. ; With Rs fixed, the non-convex problem (5) is decomposed into a convex optimization problem (6) with respect to the channel rate and a problem of maximizing the communication signal-to-dryness ratio (7) using an alternating optimization method: Formula (6) Formula (7) in, , , Let represent the initial values ​​for the i-th iteration; and As an introduced intermediate variable, the first two constraints in formula (6) are new constraints introduced by the intermediate variable, and the third constraint is obtained by fusing the second and third constraints of problem (5). It can be seen that subproblem (6) is a convex problem with respect to channel rate. The optimization problem can be solved directly using the Convex Optimization Modeling Toolkit (CVX) to obtain the channel rate R. c Another subproblem can be expressed as the following formula (8): Formula (8) The objective function of subproblem (8) is to maximize the communication signal-to-dryness ratio because when the source-channel rate is fixed, the end-to-end distortion D o As the signal-to-dryness ratio increases, it monotonically decreases; therefore, the initial end-to-end distortion minimization problem is transformed into a problem of maximizing the signal-to-dryness ratio. The second constraint of subproblem (8) is non-convex. Utilizing the Schur complement property of CRB, fractional programming, and the alternating optimization method, subproblem (8) can be transformed into the following two subproblems (9) and (10): Formula (9) Formula (10) Subproblem (9) is a standard positive semidefinite programming problem, and subproblem (10) is a convex problem. Therefore, the beamforming vector W can be obtained by solving it using CVX. c And the value of W0. Where, , and For the introduction of intermediate variables, Furthermore, the non-convex form of the signal-to-dryness ratio can be transformed into a convex form as shown in the following formula (11) using fractional programming; Formula (11) in, , k is the penalty parameter, whose value changes as the iteration progresses, and can be expressed as: , This represents the step size used to control the rate of descent of the objective function. To obtain using the continuous convex approximation method The approximate convex form, where It can be expressed as the following formula (12):

[0041] Formula (12) Where i represents the iteration number. By iteratively optimizing and solving problems (6), (9), and (10), the corresponding ISSC emission parameters of the current semantic coding model (i.e., the codec model) can be obtained. Then, by repeating the above process, the ISSC emission parameters of all codec models can be obtained. The corresponding ISSC transmission parameters are calculated, and the semantic distortion of different semantic coding models (coder-decoder models) under the current channel conditions is calculated. The coder-decoder model with the minimum semantic distortion and its corresponding ISSC parameters (transmission parameters: channel rate, source rate, semantic beamforming vector, and sensing beamforming vector) are selected.

[0042] The following scenario illustrates in more detail how an ISSC system transmits ISSC signals to a semantic receiver and a perception receiver respectively through a direct semantic channel and a perception channel reflected from the target to complete image reconstruction and perception localization tasks. Step S01: Obtain one base station, one target, and two receivers from the ISSC system; Step S02: Obtain the direct channel coefficient h from the base station to the semantic receiver. c The channel coefficient H0 of the base station-target-semantic receiver, and the departure angle of the base station. Angle of arrival of the sensing receiver Reflectance coefficient, including the effects of radar cross section and path loss Gaussian noise power Signal sampling number N, signal bandwidth B, number of base station transmit antennas N T The number of receiving antennas N of the sensing receiver R Base station transmit power P T HCRB threshold perception ; Step S03: Derive the sensing HCRB in the ISSC system according to formulas (3)-(4); Step S04: Pre-train the deep learning-based codecs of group G on the dataset to obtain the parameters of the neural network and the compression ratio (channel rate) corresponding to different encoders in group G. ; Step S05: Pre-train multiple data recovery models (coder-decoder models) and measure source distortion under ideal error-free transmission conditions; then, through simulated transmission under different bit error rates, use data regression to fit the fitting parameters in formula (1).

[0043] Step S06: Based on formulas (1)-(4), the end-to-end distortion minimization problem (end-to-end distortion optimization problem) under the constraints of guaranteed power, maximum channel usage, and sensing accuracy threshold is obtained. This problem is a non-convex optimization problem, which is difficult to solve. The specific solution to problem (5) is as follows: Step S07: Analyze the optimization objective and HCRB constraints in the original problem (5). By introducing intermediate variables, continuous convex approximation, alternating optimization, fractional programming and other methods, the original problem (5) is transformed into problem (6), problem (9) and problem (10). Step S08: Use the CVX tool to iteratively solve problems (6), (9), and (10) to obtain the ISSC signal transmission parameters R. s R c W0 and W c .

[0044] Step S09: Repeat step S08 to calculate different codec models, i.e., different R... s The corresponding ISSC transmission parameters and semantic distortion are used to select the codec model with the minimum semantic distortion and its corresponding ISSC transmission parameters.

[0045] This embodiment determines the end-to-end distortion optimization problem based on the semantic communication end-to-end distortion model; it decomposes the end-to-end distortion optimization problem into three sub-problems; and iteratively solves the three sub-problems using a convex optimization modeling toolkit to obtain the transmission parameters corresponding to each codec model. This embodiment derives the Cramer-Rao bound of the ISSC system under clock error. By designing parameters such as source-channel coding rate and beamforming vector, the transmitter studies the semantic communication and sensing performance under the ISSC system, and presents an adaptive source-channel coding scheme based on deep learning, improving the efficiency of integrated semantic communication and sensing communication.

[0046] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the semantic communication and perception integrated communication method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0047] This application also provides a semantic communication and sensing integrated communication device, please refer to... Figure 4 The semantic communication perception integrated communication device includes: Module 10 is used to construct a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, and the semantic distortion includes source distortion and channel distortion; The determining module 20 is used to determine the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, wherein each codec model corresponds to a different source rate; The communication module 30 is used for semantic communication and perception integration communication based on the transmission parameters.

[0048] This embodiment constructs a semantic communication end-to-end distortion model, which simulates end-to-end semantic distortion, including source distortion and channel distortion. Based on the semantic communication end-to-end distortion model, transmission parameters corresponding to each semantic communication end-to-end distortion model are determined. Semantic communication and sensing integrated communication are then performed based on these transmission parameters. Since this embodiment determines the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, each codec model corresponds to a different source rate; semantic communication and sensing integrated communication are then performed based on these transmission parameters. Compared to existing communication methods that sacrifice communication or sensing performance to guarantee another performance characteristic, the above method in this embodiment can improve the efficiency of semantic communication and sensing integrated communication.

[0049] The semantic communication sensing integrated communication device provided in this application, employing the semantic communication sensing integrated communication method in the above embodiments, can solve the technical problem of low efficiency in existing semantic communication sensing integrated communication methods. Compared with the prior art, the beneficial effects of the semantic communication sensing integrated communication device provided in this application are the same as those of the semantic communication sensing integrated communication method provided in the above embodiments, and other technical features in the semantic communication sensing integrated communication device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0050] This application provides a semantic communication sensing integrated communication device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the semantic communication sensing integrated communication method in the above embodiment 1.

[0051] The following is for reference. Figure 5This document illustrates a structural schematic diagram of an integrated semantic communication sensing communication device suitable for implementing embodiments of this application. The integrated semantic communication sensing communication device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The semantic communication awareness integrated communication device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0052] like Figure 5 As shown, the semantic communication sensing integrated communication device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the semantic communication sensing integrated communication device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the semantic communication-aware integrated communication device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows semantic communication-aware integrated communication devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0053] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0054] The semantic communication sensing integrated communication device provided in this application, employing the semantic communication sensing integrated communication method in the above embodiments, can solve the technical problem of low efficiency in existing semantic communication sensing integrated communication methods. Compared with the prior art, the beneficial effects of the semantic communication sensing integrated communication device provided in this application are the same as those of the semantic communication sensing integrated communication method provided in the above embodiments, and other technical features in this semantic communication sensing integrated communication device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0055] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0056] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0057] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the semantic communication-aware integrated communication method in the above embodiments.

[0058] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0059] The aforementioned computer-readable storage medium may be included in the semantic communication sensing integrated communication device; or it may exist independently and not assembled into the semantic communication sensing integrated communication device.

[0060] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Python, Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0061] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0062] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0063] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described semantic communication-aware integrated communication method, thereby solving the technical problem of low efficiency in existing semantic communication-aware integrated communication methods. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the semantic communication-aware integrated communication method provided in the above embodiments, and will not be repeated here.

[0064] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the semantic communication-aware integrated communication method described above.

[0065] The computer program product provided in this application can solve the technical problem of low efficiency in existing semantic communication and sensing integrated communication methods. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the semantic communication and sensing integrated communication method provided in the above embodiments, and will not be repeated here.

[0066] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A semantic communication and perception integrated communication method, characterized in that, The semantic communication-aware integrated communication method includes the following steps: Construct a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, the semantic distortion including source distortion and channel distortion; Based on the semantic communication end-to-end distortion model, the transmission parameters corresponding to each codec model are determined, and each codec model corresponds to a different source rate. Based on the transmission parameters, semantic communication and perception are integrated.

2. The semantic communication and perception integrated communication method as described in claim 1, characterized in that, The semantic communication-aware integrated communication based on the transmission parameters includes: Based on the transmission parameters and the semantic communication end-to-end distortion model, determine the target transmission parameters corresponding to the minimum semantic distortion; Semantic communication and awareness are integrated based on the target transmission parameters and the corresponding codec model.

3. The semantic communication and perception integrated communication method as described in claim 1, characterized in that, The determination of the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model includes: The end-to-end distortion optimization problem is determined based on the semantic communication end-to-end distortion model. The end-to-end distortion optimization problem is decomposed into three sub-problems; The three sub-problems are solved iteratively using a convex optimization modeling toolkit to obtain the transmission parameters corresponding to each codec model.

4. The semantic communication and perception integrated communication method as described in claim 3, characterized in that, The end-to-end distortion optimization problem is decomposed into three sub-problems, including: The end-to-end distortion optimization problem is decomposed into a convex optimization problem with respect to channel rate and a problem of maximizing the communication signal-to-dryness ratio. The problem of maximizing the communication signal-to-dryness ratio is transformed into a standard semidefinite programming problem and a convex problem.

5. The semantic communication and perception integrated communication method as described in claim 3, characterized in that, The end-to-end distortion optimization problem is as follows: Among them, R s Used to characterize the source rate Rc represents the set of source rates corresponding to G semantic communication end-to-end distortion models, W0 represents the sensing beamforming vector, and W represents the channel rate. c Used to characterize semantic beamforming vectors and Used to characterize the fitting parameters, Used to characterize bit error rate Used to characterize the moisture-to-dryness ratio Current information drying ratio Channel capacity under certain conditions; HCRB is used to characterize the hybrid Cramer-Rao bound. Used to characterize the perceptual HCRB threshold Used to characterize the maximum number of times a channel can be used. The function Tr() is used to characterize the base station transmit power, and Tr() is used to characterize the trace operation.

6. The semantic communication and perception integrated communication method as described in any one of claims 1-4, characterized in that, The transmission parameters include at least one of the following: channel rate, source rate, semantic beamforming vector, and sensing beamforming vector.

7. A semantic communication and sensing integrated communication device, characterized in that, The semantic communication perception integrated communication device includes: The construction module is used to construct a semantic communication end-to-end distortion model, wherein the semantic communication end-to-end distortion model is used to simulate end-to-end semantic distortion, the semantic distortion including source distortion and channel distortion; The determination module is used to determine the transmission parameters corresponding to each codec model based on the semantic communication end-to-end distortion model, wherein each codec model corresponds to a different source rate; The communication module is used for semantic communication and perception integration based on the transmission parameters.

8. A semantic communication sensing integrated communication device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the semantic communication-aware integrated communication method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the semantic communication and perception integrated communication method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the semantic communication-aware integrated communication method as described in any one of claims 1 to 6.

Images