Data communication method and device, electronic equipment and storage medium

Abstract

The invention provides a data communication method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of semantic communication. The method is applied to a receiving end, and the specific implementation scheme is as follows: receiving original information sent by a sending end, the original information being original data displayed in any one of a sketch form, an image form and a text form; according to the original information, a pre-constructed knowledge base is adopted to carry out image restoration, an image restoration result is obtained, and the knowledge base comprises a set of multiple semantic features; if the recovered image exists in the image recovery result, evaluating the quality of the recovered image to obtain a quality evaluation result; and adjusting a data transmission form of transmitting the original information to a receiving end according to a quality evaluation result so as to carry out data communication. According to the invention, the flexibility and robustness of communication are improved, the data transmission efficiency is optimized, and the best image communication quality can be achieved in different communication environments.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to the field of semantic communication technology, and especially to a data communication method, device, electronic device and storage medium. Background Art

[0002] In the field of traditional communications, research has mainly focused on grammatical communication, that is, ensuring the accurate transmission of information symbols from the sender to the receiver, with the focus on signal integrity and accuracy. However, as the capacity of communication systems approaches the Shannon limit, this traditional approach has encountered an efficiency bottleneck. To overcome this limitation, a new paradigm of semantic communication has been proposed, which focuses on the semantic content of information, that is, the actual meaning of information, and uses artificial intelligence technology for semantic extraction, compression and transmission, thereby improving the efficiency and depth of communication.

[0003] Given the obvious advantages of semantic communication, how to effectively utilize it to meet the complex needs of future communications has become an urgent problem that needs to be solved. Summary of the invention

[0004] The present disclosure provides a data communication method, device, electronic device and storage medium.

[0005] According to one aspect of the present disclosure, a data communication method is provided, which is applied to a receiving end, and the method includes:

[0006] Receiving original information sent from a sending end, wherein the original information is original data displayed in any form of sketch, image, and text;

[0007] Performing image restoration using a pre-built knowledge base according to the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features;

[0008] If a restored image exists in the image restoration result, evaluating the quality of the restored image to obtain a quality evaluation result;

[0009] The data transmission form of the original information transmitted to the receiving end is adjusted according to the quality evaluation result to perform data communication.

[0010] According to another aspect of the present disclosure, a data communication method is provided, which is applied to a transmitting end, and the method includes:

[0011] Sending original information to a receiving end, wherein the original information is original data displayed in any form of sketch, image, or text;

[0012] The receiving end uses a pre-built knowledge base to perform image restoration based on the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features; if a restored image exists in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; and the data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to perform data communication.

[0013] According to a third aspect of the present disclosure, a data communication method is provided, the method comprising:

[0014] The sending end sends the image data corresponding to the multiple scenes to the receiving end;

[0015] The receiving end extracts semantic features from the image data and integrates all semantic features to construct a knowledge base;

[0016] The transmitting end sends original information to the receiving end according to the current channel state, wherein the original information is original data displayed in any form of sketch, image or text;

[0017] The receiving end uses the knowledge base to perform image restoration based on the original information to obtain an image restoration result; if a restored image exists in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; and the data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to perform data communication.

[0018] According to a fourth aspect of the present disclosure, a data communication device is provided, applied to a receiving end, comprising:

[0019] A receiving module, used for receiving original information sent from a sending end, wherein the original information is original data displayed in any form of sketch, image and text;

[0020] A restoration module, used to perform image restoration based on the original information using a pre-built knowledge base to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features;

[0021] An evaluation module, configured to evaluate the quality of the restored image if there is a restored image in the image restoration result, and obtain a quality evaluation result;

[0022] The adjustment module is used to adjust the data transmission form of the original information to the receiving end according to the quality evaluation result to perform data communication.

[0023] According to a fifth aspect of the present disclosure, a data communication device is provided, which is applied to a transmitting end and includes:

[0024] A sending module, used for sending original information to a receiving end, wherein the original information is original data displayed in any form of sketch, image or text;

[0025] The receiving end uses a pre-built knowledge base to perform image restoration based on the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features; if a restored image exists in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; and the data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to perform data communication.

[0026] According to a sixth aspect of the present disclosure, there is provided an electronic device, including:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method in any of the above technical solutions.

[0030] According to a seventh aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the above technical solutions.

[0031] According to an eighth aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements any one of the methods described in the above technical solutions when executed by a processor.

[0032] The present disclosure provides a data communication method, apparatus, device and storage medium. The present disclosure receives original information presented in the form of sketches, images or texts, and then uses a knowledge base containing a rich set of semantic features to perform image restoration to achieve efficient and accurate image reconstruction. After the image is reconstructed, the quality of the restored image is evaluated, and the data transmission form is dynamically adjusted according to the quality evaluation result. In this way, by dynamically adjusting the data transmission form under different communication environments, it is possible to ensure that the best image communication quality can be achieved in different communication environments. In this way, not only the flexibility and robustness of communication are improved, but also the data transmission efficiency is optimized, and a more accurate and efficient data communication method is achieved.

[0033] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0035] Figure 1 is a schematic diagram of steps of a data communication method in an embodiment of the present disclosure;

[0036] Figure 2 It is a flowchart corresponding to the data communication method in one embodiment of the present disclosure.

[0037] Figure 3 A principle block diagram of a data communication device in an embodiment of the present disclosure;

[0038] Figure 4 The block diagram is a block diagram of an electronic device for implementing the data communication method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] The present disclosure provides a data communication method, see Figure 1 As shown, Figure 1 : is a schematic diagram of the steps of a data communication method in an embodiment of the present disclosure, the method is applied to a receiving end, and the method includes:

[0041] Step S101, receiving original information sent from a sending end, where the original information is original data displayed in any form of sketch, image or text.

[0042] Specifically, the original information can be considered as the original data received from the sender, and its display form is, for example, a sketch form, a text form, or an image form. The original data is displayed in different forms in order to convey a certain content or concept, which is conducive to adapting to various channel states and data transmission rates.

[0043] Among them, "sketch form" usually refers to a picture that simplifies the representation of objects or scenes with lines and shadows, which emphasizes the basic structure and shape of visual elements; "image form" may refer to a more complete visual representation, such as a photograph or a detailed drawing, which provides rich visual details; and "text form" describes information through language and words, focusing on semantic expression.

[0044] For the "image form", since images are usually composed of a large number of pixels, images need to transmit more data when transmitted compared to sketches and text forms. Therefore, images are considered to have the highest rate. Compared with images, sketches contain less information. Sketches are usually simplified and abstract lines and outlines, requiring a relatively small amount of data and a relatively low transmission rate. Compared with images and sketches, text contains the smallest amount of information. Therefore, text is considered to have the lowest rate. Since these forms of data require different amounts of data and have different transmission rates, they can be applied to different channel states.

[0045] In this way, the receiving end receives these different forms of original information from the sending end, and then performs subsequent processing and analysis according to the specific type of information, so as to effectively transmit and understand the data during the communication process.

[0046] Step S102, performing image restoration using a pre-built knowledge base according to the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features.

[0047] Specifically, the "knowledge base" is a database containing a variety of semantic feature sets. These features are extracted from a large amount of image data through deep learning technology and can represent the key visual and semantic information of the image. The database is pre-built.

[0048] After receiving the original information sent by the sender, it is necessary to perform image restoration based on the original information. "Image restoration" refers to the process of reconstructing a high-quality image based on the original information using advanced algorithms and a pre-built knowledge base. The implementation process is as follows: First, the receiver obtains the original information from the sender; then, based on the original information, it searches and matches the corresponding semantic features in the knowledge base; then, it applies an image restoration algorithm, such as a generative adversarial network (GAN) or a deep convolutional network, to reconstruct the image in combination with these features; finally, it obtains the image restoration result, that is, the reconstructed high-quality image. In this way, the image restoration result is obtained by using a pre-built knowledge base for image restoration, providing accurate visual information for subsequent communication or display.

[0049] Step S103: If there is a restored image in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result.

[0050] Specifically, if a restored image is successfully generated during the image restoration process, the quality of the generated restored image needs to be evaluated next. "Quality assessment" is a key step that uses specific metrics, such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), to quantify the visual quality and semantic accuracy of the image. The evaluation process includes: first, selecting appropriate quality assessment indicators; then, comparing the restored image with the original image or reference image; then, calculating the evaluation indicator to obtain a value to reflect the quality of the image; finally, based on the quality assessment results obtained, it can be determined whether the restored image meets specific quality requirements, thereby deciding whether it needs further processing or is directly used in subsequent applications. This evaluation mechanism ensures the effectiveness and reliability of image restoration.

[0051] Step S104, adjusting the data transmission form of the original information to the receiving end according to the quality evaluation result to perform data communication.

[0052] Specifically, the "quality assessment result" refers to the score or rating obtained by quantitatively analyzing the restored image through specific image quality evaluation indicators, such as PSNR or SSIM, which can reflect key quality attributes such as image clarity, contrast, and structural similarity. Based on these evaluation results, it is further decided how to adjust the transmission strategy of the original information to adapt to the current communication status. This dynamic adjustment mechanism makes the communication process more flexible and efficient, and can adapt to different network conditions and communication needs.

[0053] The present disclosure provides a data communication method, apparatus, device and storage medium. The present disclosure receives original information presented in the form of sketches, images or texts, and then uses a knowledge base containing a rich set of semantic features to perform image restoration to achieve efficient and accurate image reconstruction. After the image is reconstructed, the quality of the restored image is evaluated, and the data transmission form is dynamically adjusted according to the quality evaluation result. In this way, by dynamically adjusting the data transmission form under different communication environments, it is possible to ensure that the best image communication quality can be achieved in different communication environments. In this way, not only the flexibility and robustness of communication are improved, but also the data transmission efficiency is optimized, and a more accurate and efficient data communication method is achieved.

[0054] In some optional embodiments, before receiving the original information sent from the sending end, the method further includes:

[0055] Receiving image data corresponding to multiple scenes sent from a sending end;

[0056] Extract semantic features from image data and integrate all semantic features to construct a knowledge base.

[0057] Specifically, before receiving the original information sent from the sender, a database can be pre-built. When building the database, image data corresponding to multiple scenes sent from the sender can be first received, wherein "image data corresponding to multiple scenes" refers to a set of images captured under a variety of different scenes and environmental conditions, and these image data can represent a wide range of visual and semantic diversity. After receiving the image data corresponding to multiple scenes sent from the sender, semantic features are extracted from these data, wherein "semantic features" refer to key information extracted from the image data that can represent the image content and context information, such as the shape, color, texture, etc. of the object. When extracting semantic features, deep learning algorithms, such as convolutional neural networks, can be used to analyze these images and extract semantic features therein. After the semantic features are extracted, all the extracted semantic features are then integrated to form a comprehensive knowledge base, which can be used to support tasks such as image restoration and enhancement in the future. The knowledge base constructed in this way provides rich semantic information and context support for subsequent image restoration and data communication, thereby improving the accuracy and efficiency of communication.

[0058] In this way, by pre-receiving and processing multi-scene image data from the sender before receiving the original information and extracting key semantic features from it, a comprehensive knowledge base can be built. This knowledge base not only enriches the system's understanding of different scenes, but also enhances its capabilities in image restoration and data communication. In this way, by pre-building the knowledge base, image content can be more accurately identified and reconstructed, improving the efficiency and accuracy of communication.

[0059] In some optional embodiments, image restoration is performed based on original information using a pre-built knowledge base to obtain an image restoration result, including:

[0060] Extract semantic features from original information;

[0061] Matching the semantic features in the original information with the semantic features in the knowledge base to obtain matching semantic features;

[0062] Image restoration is performed based on the matched semantic features to obtain a restored image.

[0063] Specifically, when the image restoration result is obtained based on the original information, these semantic features are first extracted from the original information, whether it is from a sketch, image or text description; then, these extracted features are compared and matched with the semantic features stored in the knowledge base to find the most similar feature set; finally, using these matched features as the basis, the image is reconstructed through the image restoration algorithm to generate a restored image. This process ensures that even if the image information is incomplete or damaged during the transmission process, the semantic content and visual information of the original image can be accurately restored.

[0064] In this way, by using the pre-built knowledge base to restore the original information, it is possible to accurately extract and match semantic features from various forms of raw data, and use these features to reconstruct high-quality images. This can significantly improve the accuracy and efficiency of image restoration, so that even when the information is incomplete or the transmission conditions are not ideal, images that are highly similar to the original scene can be generated, thereby enhancing the reliability of communication and the overall quality of image applications.

[0065] In some optional embodiments, if there is a restored image in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result, including:

[0066] Calculate the structural similarity index between the restored image and the corresponding category image in the knowledge base;

[0067] If the structural similarity index exceeds the set threshold, the quality assessment result is that the restored image quality passes;

[0068] If the structural similarity index does not exceed the set threshold, the quality assessment result is that the restored image quality fails.

[0069] Specifically, the "restored image" refers to the image reconstructed through the image restoration process, and the "structural similarity index" (SSIM) is an indicator that measures the visual similarity of two images, which takes into account the brightness, contrast and structural information of the image. The process of implementing image quality assessment includes: first, the SSIM value between the restored image and the corresponding category image in the knowledge base is calculated; then, the calculated SSIM value is compared with the pre-set threshold. If the SSIM value is higher than this threshold, it means that the restored image is visually close enough to the original image, and the quality assessment result is "passed"; conversely, if the SSIM value is lower than the threshold, it is considered that the quality of the restored image fails to meet acceptable standards, and the quality assessment result is recorded as "failed". This evaluation mechanism ensures that only high-quality images are used in subsequent communication or processing processes, thereby ensuring the quality and reliability of image communication. For example, the threshold is set to 0.7. If the calculated SSIM is greater than or equal to 0.7, it means that the quality of the generated image is passed and communication can continue. If the quality of the image generation is not passed, a lower-speed text communication method is used for communication until the lowest-speed text communication is reached. If the SSIM of the generated image and its corresponding category image is still less than 0.7 at this time, communication is not possible.

[0070] In this way, by calculating the structural similarity index (SSIM) between the restored image and the corresponding category image in the knowledge base, and comparing the index with the preset threshold, it is determined whether the image passes the quality assessment. This evaluation method provides an objective and automated method to ensure the quality of image restoration, thereby avoiding the use of low-quality images in communication or further processing, and ensuring the reliability and validity of image data. In addition, it can also guide the sender to adjust the transmission strategy according to the evaluation results, optimize resource allocation, and improve overall communication efficiency.

[0071] In some optional embodiments, adjusting the data transmission form of the original information transmitted to the receiving end according to the quality evaluation result to perform data communication includes:

[0072] If the quality assessment result is that the restored image quality fails, the data transmission rate corresponding to the current original information is obtained;

[0073] If the data transmission rate corresponding to the current original information is not the lowest transmission rate, the data transmission form of the original information transmitted to the receiving end is switched to a lower data transmission rate to perform data communication.

[0074] Specifically, the "quality evaluation result" is obtained by evaluating the quality of the restored image, and the quality evaluation result includes whether the restored image quality passes or fails. The "minimum transmission rate" refers to the minimum data transmission speed that can be supported under the current communication conditions.

[0075] In this embodiment, if the quality assessment result shows that the quality of the restored image fails, the data transmission rate used by the current original information is checked; secondly, if the current transmission rate is not the lowest transmission rate, the system will automatically reduce the data transmission rate by one level to reduce the amount of data transmitted; finally, data communication is performed again at the adjusted transmission rate until a suitable rate is found to ensure that the image restoration quality meets the requirements. This dynamic adjustment mechanism enables the system to flexibly adapt to different communication environments, optimize transmission efficiency, and ensure the quality of image restoration.

[0076] In this way, when the quality of the restored image fails to pass the evaluation, the system will automatically check and reduce the current data transmission rate until the minimum transmission rate is reached. The beneficial effect of this adaptive adjustment mechanism is that it can ensure that the success rate of image recovery and the reliability of communication are improved by reducing the amount of data and optimizing the transmission rate under different network conditions and communication environments. At the same time, this method also helps to reduce the bandwidth waste caused by transmitting high-resolution or complex images, thereby improving communication efficiency and reducing transmission costs.

[0077] In some optional embodiments, adjusting the data transmission form of the original information transmitted to the receiving end according to the quality evaluation result to perform data communication further includes:

[0078] If the quality assessment result shows that the image quality is restored, communication is performed at the current data transmission rate.

[0079] Specifically, if the quality assessment result is that the restored image quality passes, then the quality assessment result of the restored image is determined to pass, that is, the image restoration has reached the expected quality requirements, and the current data transmission rate will be maintained for communication. This means that there is no need to reduce the transmission rate or change the transmission strategy, and data can continue to be transmitted at the existing rate, ensuring the continuity and efficiency of communication. This approach allows the system to optimize the use of communication resources while ensuring image quality, avoiding unnecessary transmission rate adjustments, and thus improving the overall data communication performance.

[0080] In this way, when the quality assessment result of the restored image shows that the image quality meets the standard, the system will maintain the current data transmission rate for communication, thus ensuring that high-quality image data is transmitted in the fastest and most economical way, thereby improving transmission efficiency and reducing unnecessary bandwidth waste while ensuring communication quality. In addition, this strategy also helps to make full use of available bandwidth when network conditions are good, ensuring the stability and real-time performance of data transmission, and providing users with a smoother and more reliable communication experience.

[0081] In some optional embodiments, the method further comprises:

[0082] If the restored image does not exist in the image restoration result, the knowledge base is updated again.

[0083] Specifically, "re-updating the knowledge base" is a countermeasure that will be triggered when a restored image is not successfully generated during the image restoration process. After obtaining the image restoration result, the image restoration result will be confirmed to confirm whether a restored image has been generated; if there is no restored image, this may mean that the existing knowledge base lacks sufficient information to support the restoration process; at this time, the system will start the update process to enrich and improve the knowledge base by adding new image data and semantic features; the updated knowledge base will contain more comprehensive features, improve the system's ability to understand the original information, and thus increase the success rate in future image restoration attempts. This mechanism ensures the continuous optimization of the knowledge base and the continuous improvement of system performance.

[0084] In this way, by triggering the update mechanism of the knowledge base when the restored image is not detected in the image restoration result, the continuous optimization and expansion of the knowledge base can be ensured. This adaptive update strategy not only improves the system's ability to handle future image restoration tasks, but also enhances the system's adaptability to new scenarios and new features. In addition, regular updates of the knowledge base help improve the accuracy and reliability of image restoration, ensuring that the system can maintain high-performance image reconstruction quality even when faced with new or uncommon image types, thereby maintaining the effectiveness and efficiency of communication in a dynamically changing data environment.

[0085] In some optional embodiments, before receiving the original information sent from the sending end, the method further includes:

[0086] Get the current channel status;

[0087] The current channel state is sent to the sending end, so that the sending end determines the data form of the original information sent to the receiving end according to the current channel state.

[0088] Specifically, the "channel state" describes the quality of the current communication link, including factors such as signal strength, interference level, and bandwidth. Before receiving the original information, the receiving end first "obtains the current channel state", which is a process of real-time monitoring of the quality of the communication link. Subsequently, the receiving end "sends the current channel state to the sending end", a step that ensures that the sending end can understand the channel conditions in real time. Finally, the sending end determines the data form of the original information sent to the receiving end based on the received channel state information, which means that the sending end will select the most appropriate data transmission format and rate according to the actual conditions of the channel to ensure the efficiency and reliability of data transmission. This process enables the communicating parties to dynamically adjust based on real-time channel state information, optimizes the data transmission strategy, and improves the quality of communication.

[0089] In this way, by actively acquiring and sharing the current channel status with the sender before receiving the original information, dynamic optimization of the data transmission strategy is achieved. This real-time channel status monitoring and communication mechanism enables the sender to adjust the data transmission format and rate according to the latest channel conditions, thereby minimizing transmission errors and improving transmission efficiency.

[0090] The present disclosure provides a data communication method, which is applied to a sending end and includes:

[0091] Sending original information to the receiving end, the original information is original data displayed in any form of sketch, image or text;

[0092] The receiving end uses a pre-built knowledge base to perform image restoration based on the original information to obtain an image restoration result, where the knowledge base includes a collection of multiple semantic features. If a restored image exists in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result. The data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to carry out data communication.

[0093] In this way, the method achieves high efficiency and accuracy in data transmission by sending original information in various forms and using a pre-built knowledge base to perform image restoration at the receiving end. Specifically, the sending end can send original data in the form of sketches, images or texts, and the receiving end uses a rich set of semantic features in the knowledge base to restore the original information and generate restored images. Subsequently, the system performs quality assessment on these restored images and dynamically adjusts the data transmission strategy based on the assessment results to communicate in the most appropriate form. This method, which comprehensively considers the form of original information, image restoration supported by the knowledge base, quality assessment, and transmission form adjustment, not only improves the accuracy of image restoration, but also optimizes the data transmission process, ensuring efficient and reliable data communication in various communication environments.

[0094] In some optional embodiments, before sending the original information to the receiving end, the method further includes:

[0095] Receive the channel status sent from the receiving end;

[0096] The data format of the original information sent to the receiving end is determined according to the channel status.

[0097] Specifically, the "channel status" reflects the quality of the current communication link, including factors such as signal strength, interference level and bandwidth. Before sending the original information, the sender first "receives the channel status sent from the receiver", a step that allows the sender to understand the current communication environment. Next, the sender determines the data form of the original information sent to the receiver based on the channel status, which means that the sender will choose the most appropriate data format and transmission rate based on the actual conditions of the channel. For example, if the channel status is good, the sender may choose to send a high-resolution image; if the channel status is poor, it may choose to send a low-resolution image or text description. This process ensures the efficiency and reliability of data transmission while reducing transmission errors or data loss due to poor channel conditions.

[0098] In this way, by receiving the channel status from the receiving end before sending the original information and determining the form of the data to be sent, more intelligent and flexible data transmission can be achieved. This a priori channel status assessment mechanism can select the most appropriate data format and transmission rate according to the real-time channel conditions, significantly improving the efficiency and reliability of data transmission, reducing transmission errors and retransmissions caused by poor channel quality, thereby saving time and resources. In addition, this method also enhances the adaptability of the system, enabling it to maintain stable communication performance in a changing network environment, ensuring the continuity of communication and the integrity of data.

[0099] In some optional embodiments, determining the data format of sending the original information to the receiving end according to the channel state includes:

[0100] If the channel state is the first state, the original data in the form of an image is sent to the receiving end;

[0101] If the channel state is the second state, the original data in the form of sketch is sent to the receiving end;

[0102] If the channel state is the third state, the original data in text form is sent to the receiving end;

[0103] The data transmission rate in the first state is greater than the data transmission rate in the second state, and the data transmission rate in the second state is greater than the data transmission rate in the third state.

[0104] Specifically, before the sender sends the original information to the receiver, the sender evaluates the current channel state and then selects the most appropriate data transmission form according to the channel state. If the channel state is the "first state", it means that the channel conditions are good and the data transmission rate is fast. The sender will send the original data in the form of images to the receiver, because the image data volume is large and requires a higher transmission rate. If the channel state is the "second state", the transmission rate is slow, and the sender chooses to send the original data in the form of sketches. The data volume of sketches is smaller than that of images, which is suitable for medium transmission rates. Finally, if the channel state is the "third state", the transmission rate is the slowest, and the sender will send the original data in the form of text, because the text data volume is the smallest and can be effectively transmitted even under low transmission rate conditions. In this way, the sender can dynamically adjust the transmission strategy according to the real-time channel state to ensure the efficiency and reliability of data transmission, while reducing transmission errors and unnecessary retransmissions.

[0105] In this way, the transmission strategy can be optimized by dynamically adjusting the data form of the original information sent to the receiving end according to the channel status. When the channel status is good, image data is sent first, sketch data is sent when the status is moderate, and text data is sent when the status is poor. Such hierarchical processing not only ensures effective communication under different network conditions, but also maximizes the utilization of transmission rate and reduces delays and packet loss in data transmission. In addition, this method improves the flexibility and adaptability of communication, ensuring accurate transmission of information even in unfavorable network environments, thereby improving the reliability and efficiency of overall communication.

[0106] In some optional embodiments, before sending the original information to the receiving end, the method further includes:

[0107] Get the original data;

[0108] The edge extraction model is used to extract the edge structure information in the original data, and the original data is displayed in the form of sketches;

[0109] A multimodal model is used to extract semantic features from the original data and obtain the original data presented in text form.

[0110] Specifically, before the sender sends data to the receiver, it will perform some preprocessing steps on the data. First, the image information to be transmitted is collected. Then, the sender analyzes the data by applying the "edge extraction model", which focuses on identifying and extracting edge and structural information in the image and converting this information into a sketch form. The sketch form of data usually contains fewer details, but can highlight the basic outline and shape of the image. In addition, the sender also uses a "multimodal model" to extract the semantic features of the image. This model can understand the content of the image and convert it into a text description. The text form of data is easy to understand and process, and occupies less transmission bandwidth. In this way, the sender can generate two different forms of representation based on the original data: sketch form and text form, providing a variety of data form options for subsequent communication.

[0111] In this way, the original data is first obtained, and then the edge extraction model and multimodal model are used to extract the edge structure and semantic features of the image respectively. This process converts the data into sketch and text forms, providing a variety of data form options for subsequent communication. That is, by preprocessing the data before sending the original information, the information can be transmitted in a more efficient way.

[0112] In order to facilitate the overall understanding of the technical solution of the present application, the embodiment of the present disclosure explains the communication process of the solution as a whole. First, the sending end sends image data corresponding to multiple scenes to the receiving end; the receiving end extracts semantic features in the image data, and integrates all semantic features to construct a knowledge base; the sending end sends original information to the receiving end according to the current channel state, and the original information is original data displayed in any form of sketch, image, or text; the receiving end uses the knowledge base to restore the image according to the original information to obtain an image restoration result; if there is a restored image in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; according to the quality evaluation result, the data transmission form between the sending end and the receiving end is adjusted to perform data communication.

[0113] Specifically, see Figure 2 , Figure 2 It is a flowchart corresponding to the data communication method in one embodiment of the present disclosure. The flowchart describes the workflow of an image semantic communication method, involving the transmission of images, the construction and update of knowledge bases, the selection of communication methods, image restoration, and quality assessment and decision-making. The following is a detailed working principle: First, the entire communication process is started, and the transmitter (also referred to as the transmitting end) transmits the picture to the receiver (also referred to as the receiving end), wherein the transmitter and the receiver can be either mobile phones and base stations with relatively stable channel states, or drones and ground control stations with real-time changes in channel states. After receiving the picture, the receiver extracts semantic features to form a knowledge base, that is, the receiver extracts semantic features from the received picture, such as shape, texture, etc., and uses these features to update the knowledge base. For the knowledge base, the knowledge base is a database that stores these semantic features for subsequent image restoration processes.

[0114] Next, the transmitter selects a semantic communication method according to the channel state and communicates. Specifically, the transmitter evaluates the current channel state, such as signal strength, interference level, etc. According to the channel state, the transmitter selects the most suitable semantic communication method (such as text, sketch, image) for communication. After receiving the information, the receiver starts to restore the image. That is, after the receiver receives the information (text, sketch or image) sent by the transmitter, the receiver checks whether the knowledge base has enough information to restore the image. If the knowledge base is incomplete, the knowledge base needs to be updated. If the knowledge base is complete and the image can be restored, it is determined whether the quality of the generated image is passed. If it is passed, the communication continues until the communication ends. If the quality of the generated image is not passed, it is determined whether the current transmission rate is the lowest. If it is not the lowest transmission rate, the rate is reduced and transmitted again. If it is already the lowest rate, then the communication cannot continue. This entire flowchart shows a dynamically adjusted communication mechanism that optimizes the image transmission and recovery process according to the channel conditions and the state of the knowledge base, ensuring the reliability and effectiveness of the communication.

[0115] The following describes an apparatus embodiment of the present application, which can be used to execute the data communication method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the data communication method of the present application.

[0116] The present disclosure also provides a data communication device 300, such as Figure 3 As shown, it is applied to the receiving end and includes:

[0117] The receiving module 301 is used to receive the original information sent from the sending end, where the original information is original data displayed in any form of sketch, image or text;

[0118] Restoration module 302, used to perform image restoration based on original information using a pre-built knowledge base to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features;

[0119] An evaluation module 303 is used to evaluate the quality of the restored image if there is a restored image in the image restoration result, and obtain a quality evaluation result;

[0120] The adjustment module 304 is used to adjust the data transmission form of the original information transmitted to the receiving end according to the quality evaluation result to perform data communication.

[0121] In some optional embodiments, before receiving the original information sent from the sending end, the receiving module 301 is further used to:

[0122] Receiving image data corresponding to multiple scenes sent from a sending end;

[0123] Extract semantic features from image data and integrate all semantic features to construct a knowledge base.

[0124] In some optional embodiments, the restoration module 302 performs image restoration based on the original information using a pre-built knowledge base to obtain an image restoration result, including:

[0125] Extract semantic features from original information;

[0126] Matching the semantic features in the original information with the semantic features in the knowledge base to obtain matching semantic features;

[0127] Image restoration is performed based on the matched semantic features to obtain a restored image.

[0128] In some optional embodiments, if the image restoration result contains a restored image, the evaluation module 303 evaluates the quality of the restored image to obtain a quality evaluation result, including:

[0129] Calculate the structural similarity index between the restored image and the corresponding category image in the knowledge base;

[0130] If the structural similarity index exceeds the set threshold, the quality assessment result is that the restored image quality passes;

[0131] If the structural similarity index does not exceed the set threshold, the quality assessment result is that the restored image quality fails.

[0132] In some optional embodiments, the adjustment module 304 adjusts the data transmission form of the original information transmitted to the receiving end according to the quality evaluation result to perform data communication, including:

[0133] If the quality assessment result is that the restored image quality fails, the data transmission rate corresponding to the current original information is obtained;

[0134] If the data transmission rate corresponding to the current original information is not the lowest transmission rate, the data transmission form of the original information transmitted to the receiving end is switched to a lower data transmission rate to perform data communication.

[0135] In some optional embodiments, the adjustment module 304 adjusts the data transmission form of the original information transmitted to the receiving end according to the quality evaluation result to perform data communication, and further includes:

[0136] If the quality assessment result shows that the image quality is restored, communication is performed at the current data transmission rate.

[0137] In some optional embodiments, the restoration module 302 is further configured to re-update the knowledge base if the restored image does not exist in the image restoration result.

[0138] In some optional embodiments, before receiving the original information sent from the sending end, the receiving module 301 is further used to:

[0139] Get the current channel status;

[0140] The current channel state is sent to the sending end, so that the sending end determines the data form of the original information sent to the receiving end according to the current channel state.

[0141] The present disclosure also provides a data communication device, applied to a sending end, comprising:

[0142] The sending module is used to send the original information to the receiving end, where the original information is the original data displayed in any form of sketch, image or text;

[0143] The receiving end uses a pre-built knowledge base to perform image restoration based on the original information to obtain an image restoration result, where the knowledge base includes a collection of multiple semantic features. If a restored image exists in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result. The data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to carry out data communication.

[0144] In some optional embodiments, before sending the original information to the receiving end, the sending module is further used to:

[0145] Receive the channel status sent from the receiving end;

[0146] The data format of the original information sent to the receiving end is determined according to the channel status.

[0147] In some optional embodiments, the sending module determines the data format of sending the original information to the receiving end according to the channel state, including:

[0148] If the channel state is the first state, the original data in the form of an image is sent to the receiving end;

[0149] If the channel state is the second state, the original data in the form of sketch is sent to the receiving end;

[0150] If the channel state is the third state, the original data in text form is sent to the receiving end;

[0151] The data transmission rate in the first state is greater than the data transmission rate in the second state, and the data transmission rate in the second state is greater than the data transmission rate in the third state.

[0152] In some optional embodiments, before sending the original information to the receiving end, the sending module is further used to obtain the original data;

[0153] The edge extraction model is used to extract the edge structure information in the original data, and the original data is displayed in the form of sketches;

[0154] A multimodal model is used to extract semantic features from the original data and obtain the original data presented in text form.

[0155] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0156] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0157] Figure 4A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0158] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0159] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0160] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as data communication methods. For example, in some embodiments, the data communication method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the applet distribution described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the data communication method in any other appropriate manner (e.g., by means of firmware).

[0161] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0163] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0165] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0166] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0167] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0168] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data communication method, applied to a receiving end, the method comprising: Receiving original information sent from a sending end, wherein the original information is original data displayed in any form of sketch, image, and text; Performing image restoration using a pre-built knowledge base according to the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features; If a restored image exists in the image restoration result, evaluating the quality of the restored image to obtain a quality evaluation result; The data transmission form of the original information transmitted to the receiving end is adjusted according to the quality evaluation result to perform data communication.

2. The method according to claim 1, wherein: Before receiving the original information sent from the sending end, the method further includes: Receiving image data corresponding to multiple scenes sent from the sending end; Semantic features in the image data are extracted, and all semantic features are integrated to construct a knowledge base.

3. The method according to claim 1, wherein: The method of restoring the image using a pre-built knowledge base according to the original information to obtain an image restoration result includes: Extracting semantic features from the original information; Matching the semantic features in the original information with the semantic features in the knowledge base to obtain matching semantic features; Image restoration is performed according to the matched semantic features to obtain a restored image.

4. The method according to claim 1, wherein: If the image restoration result includes a restored image, evaluating the quality of the restored image to obtain a quality evaluation result includes: Calculating a structural similarity index between the restored image and an image of a corresponding category in the knowledge base; If the structural similarity index exceeds a set threshold, the quality assessment result is that the restored image quality passes; If the structural similarity index does not exceed the set threshold, the quality assessment result is that the restored image quality fails.

5. The method according to claim 1, wherein: The step of adjusting the data transmission form of the original information to the receiving end according to the quality evaluation result to perform data communication includes: If the quality assessment result is that the restored image quality fails, obtaining the data transmission rate corresponding to the current original information; If the data transmission rate corresponding to the current original information is not the lowest transmission rate, the data transmission form of transmitting the original information to the receiving end is switched to a lower data transmission rate to perform data communication.

6. The method according to claim 5, wherein: The step of adjusting the data transmission form of the original information to the receiving end according to the quality evaluation result to perform data communication further includes: If the quality assessment result shows that the restored image quality passes, communication is performed at the current data transmission rate.

7. The method according to claim 1, wherein: The method further comprises: If the restored image does not exist in the image restoration result, the knowledge base is updated again.

8. The method according to any one of claims 1 to 7, wherein: Before receiving the original information sent from the sending end, the method further includes: Get the current channel status; The current channel state is sent to the transmitting end, so that the transmitting end determines the data form of the original information sent to the receiving end according to the current channel state.

9. A data communication method, applied to a transmitting end, the method comprising: Sending original information to a receiving end, wherein the original information is original data displayed in any form of sketch, image, or text; The receiving end uses a pre-built knowledge base to perform image restoration according to the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features; If there is a restored image in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; and the data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to perform data communication.

10. The method according to claim 9, wherein: Before sending the original information to the receiving end, the method further includes: receiving a channel state sent from the receiving end; The data format of the original information sent to the receiving end is determined according to the channel state.

11. The method according to claim 9 or 10, wherein: The determining, according to the channel state, the data format of sending the original information to the receiving end comprises: If the channel state is the first state, sending raw data in the form of an image to the receiving end; If the channel state is the second state, sending the original data in the form of sketch to the receiving end; If the channel state is the third state, sending the original data in text form to the receiving end; The data transmission rate of the first state is greater than the data transmission rate of the second state, and the data transmission rate of the second state is greater than the data transmission rate of the third state.

12. The method according to claim 9 or 10, wherein: Before sending the original information to the receiving end, the method further includes: Get the original data; Using an edge extraction model to extract edge structure information from the original data, to obtain the original data displayed in the form of a sketch; A multimodal model is used to extract semantic features from the original data to obtain the original data presented in text form.

13. A data communication method, the method comprising: The sending end sends the image data corresponding to the multiple scenes to the receiving end; The receiving end extracts semantic features from the image data and integrates all semantic features to construct a knowledge base; The transmitting end sends original information to the receiving end according to the current channel state, wherein the original information is original data displayed in any form of sketch, image or text; The receiving end uses the knowledge base to perform image restoration based on the original information to obtain an image restoration result; if a restored image exists in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; and the data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to perform data communication.

14. A data communication device, applied to a receiving end, comprising: A receiving module, used for receiving original information sent from a sending end, wherein the original information is original data displayed in any form of sketch, image and text; A restoration module, used to perform image restoration based on the original information using a pre-built knowledge base to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features; An evaluation module, configured to evaluate the quality of the restored image if there is a restored image in the image restoration result, and obtain a quality evaluation result; The adjustment module is used to adjust the data transmission form of the original information to the receiving end according to the quality evaluation result to perform data communication.

15. A data communication device, applied to a transmitting end, comprising: A sending module, used for sending original information to a receiving end, wherein the original information is original data displayed in any form of sketch, image or text; The receiving end uses a pre-built knowledge base to perform image restoration according to the original information to obtain an image restoration result, wherein the knowledge base includes a collection of multiple semantic features; If there is a restored image in the image restoration result, the quality of the restored image is evaluated to obtain a quality evaluation result; and the data transmission form between the sending end and the receiving end is adjusted according to the quality evaluation result to perform data communication.

16. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

18. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images