Three-port semantic communication architecture optimization method introducing semantic mapping end

Abstract

The invention provides a three-port semantic communication architecture optimization method introducing a semantic mapping end, and belongs to the technical field of information processing. The technical problem that semantic communication service content is not beneficial to explanation, supervision and integration is solved. According to the technical scheme, the method comprises the following steps: S1, constructing and training a multi-modal semantic coding model; s2, constructing and training a multi-modal semantic decoding model; s3, constructing a requirement test on the precision and dimension of the high-compression floating point format vector in a fixed scene; s4, fixing a service scene; s5, according to the collected business original multi-modal data; s6, collecting coded data corresponding to the original multi-modal data of the service according to the step S4; s7, constructing and training a mapping model from the coded data to the text or label data; s8, realizing a normal semantic communication system transmission process; and S9, executing subsequent supervision or processing operation. And more research and optimization spaces are provided for a future semantic communication system.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method for optimizing a three-port semantic communication architecture by introducing a semantic mapping end. Background Art

[0002] With the continuous development of communication technology, semantic communication has gradually become an important research direction in the field of information transmission. Traditional communication networks usually focus on the accurate transmission of signals, that is, the transmission of information at the bit level. However, with the explosive growth of information volume and the widespread popularity of intelligent applications, simply transmitting bit information can no longer meet the needs of complex application scenarios, especially in situations where the content of information needs to be understood and interpreted. Semantic communication significantly improves communication efficiency and content understanding capabilities by converting raw information into higher-level semantic representations, and has become an important technical development trend for the next generation of communication networks.

[0003] However, most existing semantic communication systems adopt a two-end structure, i.e., a sender and a receiver. The sender encodes the information into a semantic vector, and the receiver decodes and reconstructs the original content accordingly. This design structure ignores the monitoring and interpretation of semantic information during the communication process to some extent, resulting in undesirable results such as unexplainable or inconsistent semantics in the system in practical applications, especially in the fields involving artificial intelligence and automated decision-making.

[0004] In semantic communication systems, semantic loss or semantic distortion is an important issue that affects communication quality and system reliability. Semantic loss refers to the failure to successfully transmit or the fact that part of the semantic information is ignored during information transmission, while semantic distortion refers to the fact that the transmitted semantic information is misunderstood or deviates from the original intention at the receiving end. These problems may cause the receiver of the information to be unable to accurately understand the sender's intention, especially in applications involving automated decision-making, intelligent control or human-computer interaction, where semantic distortion may have serious consequences.

[0005] Therefore, current semantic communication technology requires a new implementation method to ensure efficient transmission while managing and processing semantic information. Summary of the invention

[0006] The purpose of the present invention is to provide a three-port semantic communication architecture optimization method introducing a semantic mapping end, aiming at the technical problems existing in the prior art, including that the semantic channel state of semantic communication is opaque and cannot cope with the semantic control and processing requirements of complex scenarios; in addition, the bandwidth occupancy of semantic vectors transmitted in semantic channels in the prior art is difficult to achieve the optimal value. In the scenario where only transmission is considered, there is a certain amount of optimization space. Therefore, the purpose of the present invention is to solve the above two problems and realize the architecture optimization of semantic communication.

[0007] The inventive idea of ​​the present invention is: starting from interpretability, by introducing a semantic mapping end, the problem of semantic loss and semantic distortion is effectively addressed; first, the semantic mapping end monitors and maps the semantic vector in real time while transmitting it, converting the floating-point array into interpretable semantic information. This mapping not only provides an intuitive understanding of the transmission content, but also can detect the difference between the mapping result and the expected semantics, and timely discover and correct possible semantic distortion. In addition to the traditional transceiver, this three-port semantic communication network with enhanced interpretability adds a semantic mapping end for interpretable semantic mapping of the semantic vector while transmitting it; by introducing the semantic mapping end, the system can monitor and manage the transmitted semantic information in real time, provide higher interpretability and transparency, and thus better adapt to the needs of complex application scenarios. This innovation not only improves the reliability and controllability of semantic communication, but also lays a solid foundation for the development of future intelligent communication systems; by monitoring and managing semantic information, the semantic mapping end can identify semantic loss in the transmission process, and notify the sending end through a feedback mechanism to re-encode or supplement transmission; this mechanism greatly improves the robustness of the communication system, ensuring that the core semantics of the information can be retained and conveyed to the maximum extent even under undesirable network conditions.

[0008] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is specifically: a three-port semantic communication architecture optimization method introducing a semantic mapping end, comprising the following steps:

[0009] S1: Construct and train a multimodal semantic coding model to translate the multimodal information transmitted by the channel into a highly compressed floating-point format vector that is convenient for transmission; compression here means that the multimodal information is flattened, normalized, and standardized to convert the data into a one-dimensional vector form with a limited value range that is easy for computers to process. Then, the vector is forward propagated through the neural network model. Depending on the specific scenario and data volume, after passing through different numbers and types of hidden layers, the output is a highly compressed floating-point format vector with a relatively small amount of data, which contains the information that the business is concerned about and other information required to restore the multimodality;

[0010] S2: Build and train a multimodal semantic decoding model to restore the floating-point format vector in step S1 to the original multimodal data form as losslessly as possible;

[0011] Specifically, the highly compressed floating-point format vector obtained in step S1 will continue to be flattened as the input of the subsequent neural network, and will be forward propagated through the neural network model to obtain a vector of multimodal information with increased data volume that directly corresponds to the original input; then, according to the composition of the vector, the data will be reorganized, classified, and restored to a multimodal data form such as voice, video, and text;

[0012] S3: Construct a test on the accuracy and dimension requirements of highly compressed floating-point format vectors in a fixed scenario, and find the data accuracy and dimension that occupies the most bandwidth resources when the data restoration effect of the multimodal codec can meet the business needs.

[0013] For example, in the case of limiting the compression rate, if the required data compression rate is 50%, then the vector dimension of the output part in step S1 can be set to 50% of the input layer data volume. This indicator can be achieved comprehensively through data quantization or reduction in the number of neurons. On this basis, the restoration effect of the output of step S2 is optimized, that is, the transmission quality is optimized as much as possible under the condition of limiting the bandwidth.

[0014] If the scene requires restoration effect, for example, the restoration rate is 80%, the restored multimodal data output by step S2 is compared with the original data and the cross entropy loss is quantified. After a large amount of training, the restoration rate can be finally stabilized at more than 80%. Then, the amount of intermediate data is limited by quantizing the data or reducing the number of neurons, so as to find the most efficient data accuracy and dimension. The quantization of the restoration data effect can follow the following formula:

[0015]

[0016] Where: X is the original input data, Y is the data restored by the model, and R is the restoration rate based on relative error;

[0017] S4: Fix the business scenario, conduct a certain amount of business tests, and collect business data. The data here refers to the multimodal data used as input in step S1. If it is a monitoring-type business scenario, it is necessary to collect a large amount of monitoring videos as business data.

[0018] S5: Based on the collected original multimodal business data, industry experts annotate the multimodal original business data with text or labels; for example, when an event or phenomenon of business concern appears in monitoring, the experts need to mark and annotate the corresponding data part;

[0019] S6: Collect the coded data corresponding to the original multimodal data of the business according to step S4, and construct a two-tuple data set with the text or label annotation in step S5, so as to complete the mapping of multimodal data to business information, which is convenient for subsequent sorting and training;

[0020] S7: For the binary data set collected in a fixed business scenario, a mapping model from encoded data to text or label data is constructed and trained, and training is continued as the business data increases. Here, experts in the field of artificial intelligence need to design a neural network of corresponding complexity and a customized training strategy based on information such as data format, data volume, and number of labels to ensure that the training proceeds in an orderly manner.

[0021] S8: Through the model trained in step S7, in the normal semantic communication system transmission process, an additional semantic port is exported and semantic text or label data is output; the semantic port exported here can be considered as an output port directly derived from a hidden layer of the overall neural network, and it and the multimodal restoration information output in step S2 are both used as training optimization targets, but the semantic port and the multimodal restoration information are two independent ports and are not directly related;

[0022] The optimization of multimodal restoration information can follow the loss function based on the following rules:

[0023]

[0024] L compress =D KL (q(Z|X)||p(Z))

[0025] L=L recon +β·L compress

[0026] Among them, L recon is the reconstruction error loss function, X is the original input data, Y is the data restored by the model, and P is the norm type. This loss function measures the difference between the restored data and the original input data, and the goal is to minimize the reconstruction error.

[0027] Where L compress is the semantic bandwidth loss function, p(Z) is the prior distribution of the standard normal distribution, X is the original input data, q(Z|X) is the posterior probability distribution of the potential representation based on the input X, and D is the posterior probability distribution of the output based on the input X. KL is the Kullback-Leibler divergence of the prior and posterior distributions, i.e., the distance between distributions. This term is used to constrain the compactness of the latent semantic representation, i.e., to limit the channel bandwidth.

[0028] Where L is the overall loss function and β is a balance coefficient used to weigh the channel bandwidth (compression degree) and restoration quality. By optimizing this loss function with the goal of minimizing L, it is expected to achieve a good balance between channel width and restoration quality.

[0029] S9: The control system collects the semantic text or label derived in step S8 and performs subsequent supervision or processing operations. When an alarm label appears on a semantic communication system or semantic port in a production environment, the staff can flexibly handle it according to the label content. For example, unimportant alarm types can be ignored, and urgent alarm types can directly apply to view the restored multimodal information, thereby flexibly performing subsequent operations and avoiding frequent processing of a large amount of multimodal information, which may cause staff fatigue or hardware resource occupation.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The actual content in the transmission channel of semantic communication in the present invention is related to the semantics of the multimodal content, but they are not equivalent. The actual content in the channel should be smaller than the multimodal content in data volume, thereby reflecting the role of semantic communication in saving bandwidth. In a network environment with the same bandwidth, if the compression rate of semantic communication is 10%, the amount of multimodal information that can be transmitted is increased to 10 times the original amount, that is, under the same conditions, more multimodal information can be transmitted, or under the same business volume, the cost of bandwidth is reduced accordingly.

[0032] 2. The actual content in the transmission channel of the present invention should not only pay attention to the transmission bandwidth, but also consider the integrity of key semantic features. Since the multimodal information restored by semantic communication needs to match the original input multimodal information as much as possible, the actual content in the transmission channel must contain necessary semantic information. For example, changes in the state of objects, occurrences of events, switching of scenes, etc. Once the bandwidth of the actual content in the transmission channel is excessively restricted for the sake of compression rate, some important semantic information may be lost, resulting in unacceptable distortion of the restored multimodal information.

[0033] 3. The redundant semantic features of the present invention should be optimized by the codec and not be transmitted. For example, the properties of objects that remain unchanged for a long time may not be transmitted, and the secondary features that are irrelevant to the business focus may not be transmitted, which is more conducive to maintaining a smaller transmission bandwidth.

[0034] 4. The semantic text and label exported by the third port of the present invention should be a standard universal format that is easy for a third party to directly understand and process based on the actual content of the transmission and the integrated redundant semantic features. For example, for the movement of objects in video services, a relatively universal label format that is easy for operators to understand can be used, such as up, down, left, right, etc. If not necessary, a mixture of multiple types of descriptions should not be used, such as relatively non-fixed descriptions such as lifting, pulling up, lowering, and increasing the horizontal height. By using standard expression labels, the reaction speed of operators can be improved, or the archiving, integration, and statistics of semantic labels can be facilitated.

[0035] 5. The semantic text and labels of the third port of the present invention can be used to integrate the automated control process of the business scenario and perform subsequent supervision or processing operations. The semantic text and labels can be exported in a relatively standardized json format, including objects of semantic information, features and attributes of the objects, etc., thereby avoiding the need for re-segmentation, classification or more complex natural language processing in subsequent processing.

[0036] 6. The important advantages of the present invention are its flexibility and scalability. By introducing the semantic mapping end, the system can not only adapt to different application scenarios, but also be customized according to specific needs. For example, in a security-sensitive communication environment, the semantic mapping end can be configured to detect potential malicious content or abnormal semantics in real time, thereby providing additional security. In cross-language communication scenarios, the semantic mapping end can act as an intermediate layer to achieve real-time translation at the semantic level, further expanding the application scope of semantic communication.

[0037] 7. The three-port architecture of the present invention provides more research and optimization space for future semantic communication systems. For example, a machine learning algorithm can be integrated at the semantic mapping end to enable it to adaptively optimize the semantic mapping process and improve the overall performance and adaptability of the system. This design not only improves the interpretability of the system, but also facilitates subsequent performance optimization and function expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0039] Figure 1 The three-port semantic communication architecture of the present invention is a network structure from business data input to the semantic port and output port respectively. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0041] Example 1

[0042] An Example of Semantic Communication in a Production Line Video Surveillance System

[0043] System Overview

[0044] In modern intelligent manufacturing environments, real-time monitoring of production lines is crucial to ensuring product quality and production efficiency. The three-port semantic communication network proposed in this invention can be effectively applied to assembly line video monitoring systems to achieve high-efficiency, low-bandwidth real-time monitoring while providing intelligent early warning functions.

[0045] This embodiment optimizes the video surveillance system on the manufacturing assembly line. Traditional methods usually directly transmit high-definition video streams, which not only takes up a lot of bandwidth, but also makes it difficult to achieve real-time intelligent analysis. This system uses the principle of semantic communication to convert video information into abstract semantic vectors directly related to the product status, greatly reducing the amount of data transmission and improving the system's intelligent analysis capabilities.

[0046] Transmitter design

[0047] 1. Data acquisition module

[0048] Deploy high-resolution cameras to cover key locations on the assembly line.

[0049] The acquisition frame rate can be adjusted (such as 30fps) to meet different monitoring needs.

[0050] 2. Preprocessing unit

[0051] Perform preprocessing such as denoising and contrast enhancement on the original video stream.

[0052] Computer vision algorithms are used to extract key information in each frame, such as product outline, location coordinates, motion vectors, etc.

[0053] 3. Semantic Encoder

[0054] A special deep neural network is designed to encode the pre-processed video frame information into a fixed-dimensional semantic vector. The semantic vector is related to key status information such as product posture, speed, and position. It is not the original pixel data, nor the key information itself, but a high-dimensional compressed representation of the key information.

[0055] For example, a floating-point data is mapped to the sum of the mean square error losses of multiple product trajectories on the assembly line and the preset trajectory. It does not represent the running trajectory of a specific object, but contains constraints on the motion characteristics of each object.

[0056] The floating-point data obtained from actual neural network training may be more abstract high-dimensional features, which are positively correlated with a large number of physical features, but do not correspond to specific physical meanings.

[0057] To compress semantic vectors, quantization techniques or efficient encoding algorithms can be used.

[0058] The compression ratio can be adjusted dynamically to balance transmission efficiency and information fidelity.

[0059] Receiver Design

[0060] 1. Data decompression module

[0061] Receive the compressed semantic vector and decompress it.

[0062] Realize automatic recognition and decoding of various compression algorithms.

[0063] 2. Semantic Decoder

[0064] A deep neural network structure is used to match the encoder at the sending end.

[0065] The decompressed semantic vector is reconstructed into a video frame sequence to achieve the mapping between semantic coding and pipeline monitoring video.

[0066] 3. Video reconstruction module

[0067] Based on the reconstructed frame sequence, a continuous video stream is generated.

[0068] Interpolation algorithms are applied to improve the smoothness and realism of the reconstructed video.

[0069] 4. Display output unit

[0070] Supports multiple display devices, such as monitoring screens, mobile terminals, etc.

[0071] It provides a user-friendly interface and supports functions such as multi-view switching and image zooming.

[0072] Semantic Mapping End Design

[0073] 1. Vector capture module

[0074] Capture semantic vectors in real time on the fly.

[0075] Achieve compatibility with different network protocols and ensure data integrity.

[0076] 2. Semantic Mapping Model

[0077] Industry experts annotate multi-dimensional information that needs to be focused on in business videos, such as product posture, position deviation, speed anomaly, etc., and combine the annotated content with the video frame into a binary group as training material.

[0078] Build a specialized neural network model to map abstract semantic vectors into specific product status descriptions.

[0079] The model output includes semantic text or semantic labels of multi-dimensional information such as product posture, position deviation, speed anomaly, etc.

[0080] 3. Status Analysis Unit

[0081] Based on the mapping results, real-time status assessment is performed. The assessment can be obtained by directly inputting semantic text or tags into a large language model that is familiar with industry processes to obtain an assessment of the current status, such as whether there are abnormalities, whether they need to be processed, and how to process them. Multi-level thresholds can be set to distinguish between normal status, minor abnormalities, and serious failures.

[0082] 4. Alarm generation module

[0083] When an abnormal state is detected, an alarm message of the corresponding level is automatically generated.

[0084] Supports multiple alarm methods, such as system pop-up windows, SMS notifications, sound and light alarms, etc.

[0085] The alarm system can cascade corresponding emergency response facilities, such as triggering emergency stop buttons, power off buttons, fire switches, etc. as needed according to the actual emergency situation.

[0086] System workflow

[0087] 1. Normal monitoring process

[0088] (1) The camera at the sending end continuously collects pipeline video.

[0089] (2) The preprocessing unit extracts key information and the semantic encoder generates a semantic vector.

[0090] (3) The compressed vector is transmitted to the receiving end and the semantic mapping end through the network.

[0091] (4) The receiving end reconstructs the video image for real-time monitoring by operators.

[0092] (5) The semantic mapping end converts the vector into a state description, such as "the product posture is normal and the conveyor belt runs smoothly."

[0093] 2. Anomaly Detection Process

[0094] (1) The semantic mapping end detects anomalies, such as "Product A deviates 2.5 cm from the center line."

[0095] (2) The status analysis unit evaluates the degree of abnormality and determines the condition requiring intervention.

[0096] (3) The alarm generation module immediately sends an alarm to relevant personnel.

[0097] (4) At the same time, the system can automatically adjust the display of the receiving end and highlight the abnormal area.

[0098] System Advantages

[0099] 1. High-efficiency transmission

[0100] Compared with traditional video streaming, semantic vector transmission can reduce bandwidth usage by more than 90%. It supports more cameras online at the same time and improves monitoring coverage.

[0101] 2. Intelligent analysis capabilities

[0102] The semantic mapping end realizes intelligent interpretation from abstract data to specific status.

[0103] Supports anomaly detection in complex scenarios, such as when multiple products deviate from their intended trajectory at the same time.

[0104] 3. Real-time response

[0105] Low-latency data transmission and processing ensures that abnormal conditions can be discovered and handled in a timely manner. Supports millisecond-level alarm triggering to minimize production losses.

[0106] 4. System Scalability

[0107] Semantic encoding and mapping models can be continuously optimized through continuous learning.

[0108] It is easy to integrate other sensor data, such as temperature, vibration, etc., to achieve multi-modal monitoring. Comparison with traditional methods is shown in Table 1

[0109] Table 1

[0110]

[0111]

[0112] This embodiment demonstrates the innovative application of a three-port semantic communication network in the field of intelligent manufacturing. By compressing high-dimensional video information into low-dimensional semantic vectors, not only the transmission efficiency is greatly improved, but also an ideal data foundation is provided for intelligent analysis and early warning. The introduction of the semantic mapping terminal enables the system to automatically interpret complex product status, realizing the leap from passive monitoring to active early warning. This method provides new ideas for the intelligent upgrading of the manufacturing industry, and is expected to play an important role in improving production efficiency and reducing human intervention. In practical application scenarios such as smart factories and cross-language communications, the present invention has demonstrated significant advantages, paving the way for smarter and safer communication systems in the future.

[0113] Example 2

[0114] Farm acoustic monitoring system embodiment

[0115] System Overview

[0116] This embodiment optimizes the design of the acoustic monitoring system for modern farms. Traditional methods usually use continuous recording or sampling recording, which not only generates massive data that needs to be stored, but also makes it difficult to achieve real-time analysis and early warning of poultry sounds. This system uses the principle of semantic communication to convert audio information into descriptive semantic tags, which not only significantly reduces the amount of data transmission, but also realizes intelligent recognition and analysis of poultry calls and equipment sounds.

[0117] Transmitter design

[0118] 1. Data acquisition module

[0119] Deploy omnidirectional microphone arrays at key locations on the farm;

[0120] The sampling rate is adjustable (such as 44.1kHz) to ensure the collection of sound details;

[0121] Supports sound source localization function, which can determine the location of the sound source.

[0122] 2. Preprocessing unit

[0123] Perform environmental noise cancellation and sound enhancement processing

[0124] Extracting audio features using spectrogram analysis

[0125] Perform sound framing and feature extraction, such as Mel-frequency cepstral coefficients (MFCC)

[0126] 3. Semantic Encoder

[0127] A specially trained deep neural network is used to encode audio features into semantic vectors

[0128] The semantic vector contains the characteristic information of the chicken call type (such as eating, resting, fighting, courting, etc.) and the equipment operation sound (such as fans, feeders, water supply systems, etc.)

[0129] Use efficient compression algorithms to compress semantic vectors

[0130] Support real-time encoding to ensure the timeliness of the monitoring system.

[0131] Receiver Design

[0132] 1. Data decompression module

[0133] Receive and decompress the semantic vector

[0134] Ensure data integrity and real-time

[0135] 2. Semantic Decoder

[0136] Adopting a neural network structure matching the encoder

[0137] Reconstructing semantic vectors into audio signals

[0138] Supports selective reconstruction, which can only reconstruct the audio of a specific time period

[0139] 3. Audio reconstruction module

[0140] Generate a continuous audio stream

[0141] Apply audio enhancement algorithms to improve the quality of reconstructed sound

[0142] Support multi-channel output

[0143] 4.Playback control unit

[0144] Provides an audio playback interface that supports functions such as progress bar dragging and volume adjustment

[0145] Supports quick location of abnormal audio segments by timestamp

[0146] Provides visual display of audio data (such as spectrogram)

[0147] Semantic Mapping End Design

[0148] 1. Vector analysis module

[0149] Capturing semantic vectors for audio in real time

[0150] Perform vector feature extraction and classification

[0151] 2. Semantic Tag Generator

[0152] Mapping semantic vectors to specific sound type descriptions

[0153] Support multi-level label system:

[0154] Chicken call classification: normal call, fighting sound, pain sound, courtship sound, etc. Equipment sound classification: normal operation, abnormal vibration, bearing wear, etc. In addition, it supports the generation of associated records of timestamps and tags. 3. Status evaluation unit

[0155] Statistical analysis based on semantic tags to evaluate the health status of the group (such as fighting frequency, abnormal call ratio) and monitor the operating status of equipment (such as the normal operation rate of fans)

[0156] Generate trend reports and health indices

[0157] 4. Early warning system

[0158] Set multi-level warning thresholds

[0159] When abnormal sound patterns are detected, timely alarms are issued. Supports multiple warning methods such as SMS and application push

[0160] Record warning events and generate analysis reports

[0161] System workflow

[0162] 1. Daily monitoring process

[0163] (1) Microphone array continuously collects farm audio

[0164] (2) Generate semantic vectors after preprocessing and transmit

[0165] (3) The receiving end reconstructs the audio on demand for manual review

[0166] (4) The semantic mapping end generates tags and performs analysis

[0167] (5) Generate operational reports regularly

[0168] 2. Exception handling process

[0169] (1) The system detects abnormal sounds (such as intensive fighting sounds)

[0170] (2) Automatically generate abnormal event records, including time and location information (3) Send warning information to management personnel

[0171] (4) Supports quick access to the original audio of the corresponding time period

[0172] (5) Record processing results for system optimization

[0173] System Features

[0174] 1. Intelligent analysis function

[0175] Analysis of flock behavior patterns

[0176] Equipment health status diagnosis

[0177] Environmental anomaly detection

[0178] 2. Statistical analysis function

[0179] Generate daily / weekly / monthly operation reports

[0180] Equipment operation status statistics

[0181] Flock health trend analysis

[0182] 3. Search function

[0183] Retrieve sound events by time and type

[0184] Support similar event association query

[0185] Abnormal event tracing analysis

[0186] System Advantages

[0187] 1. Efficient operation

[0188] Significantly reduce data storage requirements

[0189] Support large-scale farm monitoring

[0190] Reduce the frequency of manual inspections

[0191] 2. Intelligent early warning

[0192] Discover abnormal situations promptly

[0193] Reduce the risk of disease transmission

[0194] Preventing equipment failure

[0195] 3. Data value

[0196] Support long-term data analysis

[0197] Assisted breeding decision optimization

[0198] Provide scientific breeding basis

[0199] The comparison with the traditional method is shown in Table 2:

[0200] Table 2

[0201]

[0202]

[0203] This example demonstrates the innovative application of a three-port semantic communication network in the field of smart farming. By converting complex acoustic information into structured semantic tags, it not only achieves efficient data transmission and storage, but also provides a powerful tool for the intelligent management of farms. The semantic mapping function of the system enables farm managers to quickly grasp the situation on the farm and realize intelligent monitoring of poultry health and equipment operation, providing a new technical solution for the modern farming industry.

[0204] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for optimizing a three-port semantic communication architecture by introducing a semantic mapping terminal, characterized in that: The following steps are involved: S1: Build and train a multimodal semantic encoding model to translate the multimodal information transmitted through the channel into a highly compressed floating-point format vector that is conducive to transmission; S2: Build and train a multimodal semantic decoding model to losslessly restore the floating-point format vector in step S1 to the original multimodal data form; S3: Construct a test on the accuracy and dimension of highly compressed floating-point format vectors in a fixed scenario, and find the data accuracy and dimension that occupies the most bandwidth resources when the data restoration effect of the multimodal codec meets the business needs. The quantification of the data restoration effect can follow the following formula: Where: X is the original input data, Y is the data restored by the model, and R is the restoration rate based on relative error; S4: Fixed business scenarios, conducted a certain amount of business testing, and collected business data; S5: Based on the collected original multimodal business data, industry experts annotate the multimodal original business data with text or labels; S6: Collect the coded data corresponding to the original multimodal data of the business according to step S4, and construct a two-tuple data set with the text or label annotation in step S5, so as to complete the mapping of multimodal data to business information; S7: For a fixed set of binary data collected in a business scenario, build and train a mapping model from encoded data to text or label data, and continue training as business data increases; S8: Using the model trained in step S7, additionally exporting semantic ports and outputting semantic text or label data during normal semantic communication system transmission; S9: The control system collects the semantic text or label derived in step S8 and performs subsequent supervision or processing operations.

2. The method for optimizing a three-port semantic communication architecture by introducing a semantic mapping terminal according to claim 1, characterized in that: The compression in step S1 refers to the process of flattening, normalizing, and standardizing the multimodal information to convert the data into a one-dimensional vector form with a limited value range that is easy for computers to process. Then, the vector is forward propagated through a neural network model, and after passing through different numbers and types of hidden layers according to specific scenarios and data volumes, the output is a highly compressed floating-point format vector with a relatively small amount of data, which contains the information of concern to the business and other information required to restore the multimodality.

3. The three-port semantic communication architecture optimization method of introducing a semantic mapping terminal according to claim 1 is characterized in that: In the step S2, the highly compressed floating-point format vector obtained in step S1 is further flattened as the input of the subsequent neural network, and forward propagated through the neural network model to obtain a vector of multimodal information with increased data volume that directly corresponds to the original input. Then, according to the composition of the vector, the data is reintegrated and classified to be restored to the multimodal data form of voice, video, and text.

4. The three-port semantic communication architecture optimization method of introducing a semantic mapping terminal according to claim 1 is characterized in that: In step S3, the vector dimension of the output part in step S1 is set to 50% of the amount of input layer data. This indicator is achieved comprehensively by quantizing the data or reducing the number of neurons. On this basis, the restoration effect of the output of step S2 is optimized.

5. The three-port semantic communication architecture optimization method of introducing a semantic mapping terminal according to claim 1 is characterized in that: The data in step S4 refers to the multimodal data used as input in step S1. If it is a monitoring type business scenario, it is necessary to collect a large amount of monitoring videos as business data.

6. The method for optimizing a three-port semantic communication architecture by introducing a semantic mapping terminal according to claim 1, characterized in that: The semantic port derived in step S8 is an output port directly derived from a hidden layer of the overall neural network, and the multimodal restoration information output in step S2 should be used as the training optimization target, wherein the semantic port and the multimodal restoration information are two independent ports and are not directly related; The optimization for multimodal restoration information follows a loss function based on the following rules: L compress =D KL (q(Z|X)||p(Z)) L=L recon +β·L compress Among them, L recon is the reconstruction error loss function, X is the original input data, Y is the data restored by the model, P is the norm type, this loss function measures the difference between the restored data and the original input data, and the goal is to reduce the reconstruction error; Among them, L compress is the semantic bandwidth loss function, p(Z) is the prior distribution of the standard normal distribution, X is the original input data, q(Z|X) is the posterior probability distribution of the potential representation based on the input X, and D is the posterior probability distribution of the output based on the input X. KL is the Kullback-Leibler divergence of the prior and posterior distributions, i.e., the distance between distributions. This term is used to constrain the compactness of the latent semantic representation, i.e., to limit the channel bandwidth; Where L is the overall loss function, β is the balance coefficient, which is used to weigh the channel bandwidth and restoration quality. The goal is to minimize L. By optimizing this loss function, it is expected to achieve a good balance between channel width and restoration quality.

7. The method for optimizing a three-port semantic communication architecture by introducing a semantic mapping terminal according to claim 1, characterized in that: In step S9, when an alarm-type label appears on the semantic port of the semantic communication system in the production environment, the staff will handle it flexibly according to the label content, ignore unimportant alarm types, and directly apply to view the restored multimodal information for urgent alarm types, so as to flexibly execute subsequent operations.

Images