A task feed-forward method for key semantic communication of images, an electronic device, a storage medium, and a computer program product

Abstract

The present invention provides a task feedforward-based key semantic communication method for images, belonging to the technical field of semantic communication. The method includes: parsing and processing task feedforward information to determine a target task to be executed; dividing an image into multiple information units according to the task requirement information of the target task, and allocating bandwidth resources of a communication link to the information units according to the importance degree of task requirements; processing the information units by selecting an image compression or output semantic label method according to the allocated bandwidth resources; integrating the processed generated data into data to be transmitted, and inputting the data into the communication link to achieve the feedback transmission of semantic information. The present invention realizes the task-oriented semantic information screening, sorting, and compression processes, improves the retention rate of task-related information and the compression rate of irrelevant information during image data compression, and ensures the feedback transmission of key semantic information related to the task while achieving an extreme compression rate.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic communication, and particularly to a task feed-forward type image key semantic communication method. Background Art

[0002] Semantic communication technology is a newly emerging communication mode in the field of communication. Its main function is to reduce the amount of transmitted data through information processing during the communication process. The main idea of semantic communication is: by extracting the meaning of the original data, filtering out useless, irrelevant, and non-essential information, thereby further compressing the data while retaining the data semantics, and then reducing the amount of information transmitted, reducing the requirement for channel bandwidth. Specifically, semantic communication refers to the communication where semantic reachability is achieved under the precondition that both communication parties have a common "knowledge base". The biggest difference between semantic communication and traditional communication is that semantic communication has knowledge bases existing at the sending end and the receiving end. The sending end extracts semantic information from the information source based on knowledge, and the receiving end reconstructs information based on the semantic information according to knowledge, thereby reducing the amount of information that needs to be transmitted.

[0003] Existing semantic communication schemes and frameworks generally include eight parts: information source, semantic knowledge base, semantic encoder, syntax encoder, channel, syntax decoder, semantic decoder, and information sink. Among them, the five modules of information source, syntax encoder, channel, syntax decoder, and information sink are the classic point-to-point communication system models discussed by Shannon. Semantic communication is a system superimposed on the classic communication model, constituting semantic-level communication. It introduces semantic encoders, semantic decoders, and semantic knowledge bases, and expands the channel and information sink.

[0004] So far, a large number of studies have been carried out on semantic communication technology in aspects such as network architecture, basic theory, coding technology, and security mechanism, and very good achievements have been made, promoting the development of semantic communication technology. Semantic communication has stronger information expression ability. Compared with traditional communication, it pays more attention to the meaning of the transmitted information to ensure that both parties can correctly understand the true intention of the information and achieve the task goal. At the same time, combined with deep learning technology, semantic communication can automatically adapt to various communication environments, optimize coding and transmission strategies, and adopt technologies such as adaptive modulation and automatic repeat request, thereby improving the robustness and adaptability of communication. Through secure sharing of the knowledge base and combined with technologies such as adversarial encryption, semantic communication can improve the privacy protection of data and prevent potential attackers from stealing information.

[0005] Although semantic communication shows great potential in many aspects, there are still many unsolved problems, and its application and development face many challenges. Under low-bandwidth communication conditions, it is difficult for existing technologies to achieve a feed-forward type key semantic feedback with a certain degree of interpretability. Summary of the Invention

[0006] The present invention provides a task feed-forward based key semantic communication method for images, aiming to solve the defect in the prior art that it is difficult to achieve information feedback under low bandwidth.

[0007] In a first aspect, the present invention provides a task feed-forward based key semantic communication method for images, including: parsing and processing task feed-forward information to determine a target task to be executed; the content of the target task includes at least one task requirement information and a task requirement importance corresponding to the task requirement information one by one; segmenting an image into multiple information units according to the task requirement information of the target task, and allocating bandwidth resources of a communication link to the information units according to the task requirement importance; wherein, the size of the allocated bandwidth resources is positively correlated with the task requirement importance; processing the information units by selecting image compression or outputting semantic labels according to the allocated bandwidth resources; integrating the processed data into data to be transmitted and inputting it into the communication link to achieve the feedback of semantic information.

[0008] According to the task feed-forward based key semantic communication method for images provided by the present invention, parsing and processing task feed-forward information to determine a target task to be executed includes: receiving task feed-forward information preset according to requirements; in the case where the requirement is to call a task, calling a target task from a task knowledge base according to the task feed-forward information; in the case where the requirement is to modify the content of a certain task, modifying the content of a certain task according to the task feed-forward information, and taking the modified task as the target task and updating the task knowledge base; in the case where the requirement is to add a new task, adding a new task according to the task feed-forward information, and taking the new task as the target task and updating the task knowledge base.

[0009] According to the task feed-forward based key semantic communication method for images provided by the present invention, segmenting an image into multiple information units according to the task requirement information of the target task includes: segmenting the image by using a semantic segmentation algorithm according to the task requirement information to obtain multiple information units; taking the information units corresponding to the task requirement importance greater than a first preset threshold as key semantic information units, taking the information units corresponding to the task requirement importance less than or equal to the first preset threshold and greater than a second preset threshold as general semantic information units, taking the information units corresponding to the task requirement importance less than or equal to the second preset threshold as redundant semantic information units, and forming a background information unit from other remaining parts.

[0010] According to the task feed-forward based key semantic communication method for images provided by the present invention, allocating bandwidth resources of a communication link to the information units according to the task requirement importance includes: allocating bandwidth resources to the information units according to a preset priority based on the bandwidth resources of the communication link and the task requirement importance; the preset priority is sorted in order from front to back as: key semantic information units, general semantic information units, redundant semantic information units.

[0011] A task feed-forward image key semantic communication method provided by the present invention processes information units by selecting image compression or outputting semantic tags according to the allocated bandwidth resources, including: when the information unit is a key semantic information unit, compressing the key semantic information unit based on image compression to obtain compressed image data; and when the information unit is a general semantic information unit, using the semantic tag of the general semantic information unit as the object to be transmitted.

[0012] A task feed-forward image key semantic communication method provided by the present invention, after the semantic information is fed back, further includes: determining an image data model corresponding to the semantic tag from a preset domain knowledge base for image reconstruction.

[0013] The algorithm for image compression in the task feed-forward image key semantic communication method provided by the present invention is the JEPEG2000 image compression algorithm.

[0014] In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps of any one of the above-mentioned task feed-forward image key semantic communication methods.

[0015] In a third aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of any one of the above-mentioned task feed-forward image key semantic communication methods.

[0016] In a fourth aspect, the present invention further provides a computer program product, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of any one of the above-mentioned task feed-forward image key semantic communication methods.

[0017] The present invention realizes the process of task-oriented semantic information screening, sorting, and compression, improves the retention rate of task-related information and the compression rate of irrelevant information during image data compression, and ensures the feedback of key semantic information related to tasks while achieving an extreme compression rate.

[0018] The task feed-forward image key semantic communication method provided by the present invention designs a semantic feedback mode of task requirement feed-forward, provides an interface for the control end to adjust the unmanned end, and improves the pertinence and effectiveness of image data feedback during task execution.

[0019] The present invention provides support for the process of semantic communication by establishing a task knowledge base and a domain knowledge base, and designs corresponding interaction mechanisms and update mechanisms to ensure the synchronization and completeness of the knowledge base, providing support for the realization of high compression ratio and high task efficiency in semantic communication technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is one of the schematic flowcharts of the task feedforward type image key semantic communication method provided by the present invention;

[0022] Figure 2 It is the schematic structural diagram of the task knowledge base provided by the present invention;

[0023] Figure 3 It is the schematic flowchart of the task feedforward information generation method provided by the present invention;

[0024] Figure 4 It is the schematic diagram of dividing an image to form different types of information units provided by the present invention;

[0025] Figure 5 It is the second schematic flowchart of the task feedforward type image key semantic communication method provided by the present invention;

[0026] Figure 6 It is the schematic structural diagram of the task feedforward type image key semantic communication system provided by the present invention;

[0027] Figure 7 It is the schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0029] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such a process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the said element. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0030] The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually of the same type, and do not limit the number of objects. For example, the first object can be one or multiple.

[0031] The following will be combined with Figures 1 - 7 to describe the task feedforward type image key semantic communication method and device provided by the embodiments of the present invention.

[0032] Figure 1 is one of the flow diagrams of the task feedforward type image key semantic communication method provided by the present invention, as Figure 1 shown, including but not limited to the following steps:

[0033] Step 101: Parse and process the task feedforward information to determine the target task to be executed; the content of the target task includes at least one task requirement information and the task requirement importance corresponding to the task requirement information one by one.

[0034] It should be noted that in the process of implementing the technical solution of the present invention, it mainly involves an unmanned end and a control end. Among them, the unmanned end can be an unmanned system such as a drone or an autonomous vehicle; the control end can be a control system such as a ground control station or a cloud server. Before the task starts, the control end will preset a series of instructions and expected behaviors according to the task objective and environmental conditions. These instructions may include but are not limited to flight routes, data collection types, target tasks, target detection requirements, etc., so as to control the unmanned end and perform semantic feedback.

[0035] Specifically, in the embodiments of the present invention, the control end can generate task feedforward information. Once the task feedforward information is ready, the control end will send it to the unmanned end. After receiving this information, the unmanned end immediately starts to parse and understand the instructions and requirements therein to implement a series of operations such as task invocation, adjustment, and task execution.

[0036] Figure 2 It is a schematic structural diagram of the task knowledge base provided by the present invention. As Figure 2 shown, under a general implementation project (such as the task knowledge base for post-disaster reconstruction), it includes multiple tasks (i.e., subtasks, such as trapped situation analysis). A task includes multiple task requirement information, and each task requirement information corresponds to a task requirement importance. The task requirement importance can be pre-set by experts according to the specific situation of the implementation project.

[0037] Taking the target task of trapped situation analysis as an example, the task requirement information for trapped situation analysis includes: trapped location, trapped personnel, and trapped environment. Further, the task requirement importance corresponding to the trapped location is 8, the task requirement importance corresponding to the trapped personnel is 10, and the task requirement importance corresponding to the trapped environment is 6. Among them, the value range of the task requirement importance is from 0 to 10.

[0038] Step 102: Segment the image into multiple information units according to the task requirement information of the target task, and allocate bandwidth resources of the communication link for the information units according to the task requirement importance.

[0039] Among them, the size of the allocated bandwidth resources is positively correlated with the task requirement importance.

[0040] It can be understood that among the information units segmented in the present invention, there are such information units: the information unit corresponds to a specific area or feature in the image, and these areas or features correspond to the task requirement information.

[0041] For example, for the task of trapped situation analysis, the information unit corresponding to the trapped location, the information unit corresponding to the trapped personnel, and the information unit corresponding to the trapped environment can be obtained; of course, there are also some other irrelevant information units, etc.

[0042] According to the task requirement importance of each information unit, the present invention will allocate corresponding bandwidth resources for them. The information unit with a higher importance will obtain more bandwidth resources to ensure that key information can be transmitted preferentially.

[0043] Step 103: Process the information units by selecting the method of image compression or outputting semantic labels according to the allocated bandwidth resources.

[0044] The present invention can reduce the data volume through image compression to adapt to the allocated bandwidth resources.

[0045] The present invention can also output semantic tags to provide descriptive tags for information units to adapt to the allocated less bandwidth resources.

[0046] For example, the semantic tags can be "person", "road", etc.

[0047] Step 104: Integrate the processed generated data into data to be transmitted, and input it into the communication link to realize the feedback transmission of semantic information.

[0048] Finally, integrate the processed information into information to be transmitted, input it into the communication link for encoding transmission and decoding, and realize the feedback transmission of semantic information. Among them, the data information transmitted through the link is the semantic information of the source image.

[0049] Based on the content of the above embodiments, as an alternative embodiment, the task feed-forward type image key semantic communication method provided by the present invention parses and processes the task feed-forward information to determine the target task to be executed, including: receiving the task feed-forward information preset according to the requirements; in the case where the requirement is to call a task, calling the target task from the task knowledge base according to the task feed-forward information; in the case where the requirement is to modify the content of a certain task, modifying the content of a certain task according to the task feed-forward information, and taking the task after modifying the content as the target task, and updating the task knowledge base; in the case where the requirement is to add a new task, adding a new task according to the task feed-forward information, and taking the new task as the target task, and updating the task knowledge base. The following refers to Figure 3 An implementation manner of the present invention will be described.

[0050] Figure 3 It is a schematic flowchart of the task feed-forward information generation method provided by the present invention. As Figure 3 shown, the control end analyzes the currently faced task, determines whether the current task is included in the task knowledge base. If it is a new task, it is necessary to completely define the task and the content of the task (task requirement information and task requirement importance), and output relevant information; if there is a relevant task in the task knowledge base, it is determined whether the task content is changed. If there is a change, the task requirement information and task requirement importance are updated in time and relevant information is output. If there is no change, the task name is directly output; finally, the requirement feed-forward information of the currently faced task is generated and sent to the unmanned end, and the task can be adjusted at any time during the task execution process to improve the effectiveness of information feedback transmission.

[0051] The requirements generated by the control end include three forms: 1. Define a new task (there is no relevant information in the task knowledge base); 2. Add existing task requirement information (there is a relevant task in the task knowledge base, new requirements); 3. Modify task requirement information (there is a task in the task knowledge base, perform operations such as addition, deletion, and modification on the requirement part).

[0052] The present invention can update and synchronize the knowledge bases of the control end and the unmanned end through task feedforward information, providing an interface for making more full use of the resources of the knowledge base in semantic communication technology, improving the efficiency of semantic communication and reducing the semantic distortion rate.

[0053] Based on the content of the above embodiments, as an alternative embodiment, the task feedforward-based image key semantic communication method provided by the present invention divides an image into multiple information units according to the task requirement information of the target task, including: segmenting the image using a semantic segmentation algorithm according to the task requirement information to obtain multiple information units; taking the information units corresponding to the task requirement importance greater than a first preset threshold as key semantic information units, taking the information units corresponding to the task requirement importance less than or equal to the first preset threshold and greater than a second preset threshold as general semantic information units, taking the information units corresponding to the task requirement importance less than or equal to the second preset threshold as redundant semantic information units, and forming a background information unit from the other remaining parts.

[0054] Figure 4 It is a schematic diagram of dividing an image into different types of information units provided by the present invention. As Figure 4 shown, the present invention can perform object detection through algorithms such as Yolo, match with the requirement information of the task knowledge base, set a first preset threshold k1 and a second preset threshold k2, and classify the types of information units in the above manner.

[0055] Figure 5 It is the second schematic flowchart of the task feedforward-based image key semantic communication method provided by the present invention. The following will further describe the alternative technical solutions of the present invention with reference to Figure 5 this.

[0056] As an alternative embodiment, the task feedforward-based image key semantic communication method provided by the present invention allocates bandwidth resources of a communication link to information units according to the task requirement importance, including: allocating bandwidth resources to information units according to a preset priority based on the bandwidth resources of the communication link and the task requirement importance; the preset priority is sorted in order from front to back as: key semantic information units, general semantic information units, redundant semantic information units.

[0057] Specifically, all information units are classified and sorted according to their importance, and the adaptive semantic information organization module analyzes the bandwidth conditions and performs bandwidth allocation, considering key semantic information units, general semantic information units, and redundant semantic information units in turn. If there is sufficient bandwidth, the redundant semantic information units do not need to be deleted, and the general semantic information units can be adjusted for image data transmission; if the bandwidth is limited, all redundant semantic information units are deleted; if the bandwidth is extremely scarce, the general semantic information units can also be deleted.

[0058] As an alternative embodiment, the task feed-forward image key semantic communication method provided by the present invention processes information units by selecting image compression or outputting semantic labels according to the allocated bandwidth resources, including: when the information unit is a key semantic information unit, compressing the key semantic information unit based on image compression to obtain compressed image data; and when the information unit is a general semantic information unit, using the semantic label of the general semantic information unit as the object to be transmitted.

[0059] Among them, the image compression algorithm can be the JEPEG2000 image compression algorithm, and the compression ratio can be determined according to the allocated bandwidth. For example, if the size of an information unit is 10 bit and the allocated bandwidth is 6 bit, compression with a 60% compression ratio is performed.

[0060] As an alternative embodiment, after the semantic information is fed back, the task feed-forward image key semantic communication method provided by the present invention further includes: determining an image data model corresponding to the semantic label from a preset domain knowledge base for image reconstruction.

[0061] The control end has a comprehensive domain knowledge base, which contains basic image models corresponding to various semantic labels. For information with low importance, semantic labels are transmitted to reduce bandwidth consumption, and the control end calls the image data model in the domain knowledge base based on the semantic label to improve the visual effect of the reconstructed image.

[0062] On the other hand, to further illustrate the technical solution of the present invention, a task feed-forward image key semantic communication system is provided below, as Figure 6 shown. Figure 6 is a schematic structural diagram of the task feed-forward image key semantic communication system provided by the present invention.

[0063] The task feed-forward image key semantic communication system provided by the present invention includes: a control end, an unmanned end, and a communication link; among them, the unmanned end includes two support knowledge bases, namely a task knowledge base and a domain knowledge base, and four information processing units, namely a task parsing unit, a semantic segmentation unit, a semantic screening unit, and a multi-scale compression unit; the receiving end includes two support knowledge bases, namely a task knowledge base and a domain knowledge base, and four information processing units, namely an information supplement unit, an image reconstruction unit, an information presentation unit, and a demand generation unit. The specific composition and principle are as follows.

[0064] (1) Domain knowledge base: The domain knowledge base is constructed by researching the usage background and application fields and organizing the domain knowledge that may be used in the task execution process. The unmanned end can perform in-depth processing on information based on the domain knowledge base to support processes such as redundancy removal; the control end can supplement information based on the domain knowledge base to improve the visual effect of the reconstructed image. During the key semantic feedback process, the sender and receiver complete the synchronization of the domain knowledge base.

[0065] (2) Task knowledge base: The task knowledge base is composed of analyzing the tasks faced by the usage background, organizing tasks, task information requirements, and the importance scores of information requirements. The control end can generate feedforward requirement information to adjust and control the unmanned end by selecting tasks, defining tasks, modifying task requirements, and modifying the importance scores of task requirements. The unmanned end can timely adjust the task and the key points of information processing according to the task feedforward information to ensure the feedback of key semantics related to the task. During the task knowledge feedforward process, the sender and receiver complete the synchronization of the knowledge base.

[0066] (3) Task parsing unit: The task parsing unit is mainly implemented through a matching query algorithm, which matches with the task knowledge base after inputting the task feedforward information. If the task feedforward information is the task name, the task in the task knowledge base is directly called; if the task feedforward information is the task and the modified task requirement information / task requirement importance, the task requirement information / task requirement importance of the corresponding task in the task library is changed and then called; if the task feedforward information is a newly defined task, it is supplemented to the task knowledge base and then called.

[0067] (4) Semantic segmentation unit: The semantic segmentation unit can be implemented by algorithms such as the Segment Anything Model. In specific applications, it should be retrained based on the task background and domain knowledge. The main goal of the semantic segmentation unit is to segment the collected image S into n information units based on task requirements for subsequent in-depth processing.

[0068] (5) Semantic screening unit: The semantic screening unit can be implemented in the form of label matching through a matching algorithm. After inputting all segmented information units and their semantic labels, it performs a matching query with the task knowledge base and the domain knowledge base, and classifies the information units into: key semantic information units, general semantic information units, redundant semantic information units, and background information units.

[0069] (6) Adaptive compression unit: The adaptive compression unit allocates bandwidth resources to information units by analyzing the importance scores of information units and the existing bandwidth resources, and compresses different information units with different compression ratios. Among them, compression algorithms such as JEPEG2000 can be used in the compression process, and if necessary, some non-critical information units can be transmitted or deleted in the form of labels.

[0070] (7) Information supplement unit: The information supplement unit restores the semantic tags to image information units based on the domain knowledge base to provide the overall visual presentation effect of the image.

[0071] (8) Image reconstruction unit: The image reconstruction unit restores all information units to a complete image through image processing algorithms such as Alpha fusion to achieve image reconstruction.

[0072] Figure 7 is a schematic structural diagram of the electronic device provided by the present invention. As Figure 7 shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete mutual communication through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute the task feedforward image key semantic communication method, which includes: parsing and processing the task feedforward information to determine the target task to be executed; the content of the target task includes at least one task requirement information and a task requirement importance corresponding to the task requirement information one by one; segmenting the image into multiple information units according to the task requirement information of the target task, and allocating the bandwidth resources of the communication link for the information units according to the task requirement importance; wherein, the size of the allocated bandwidth resources is positively correlated with the task requirement importance; selecting a method of image compression or outputting semantic tags to process the information units according to the allocated bandwidth resources; integrating the processed generated data into the data to be transmitted and inputting it into the communication link to achieve the feedback of semantic information.

[0073] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the task feedforward image key semantic communication method provided by the above embodiments, which includes: parsing and processing the task feedforward information to determine the target task to be executed; the content of the target task includes at least one task requirement information and a task requirement importance corresponding to the task requirement information one by one; segmenting the image into multiple information units according to the task requirement information of the target task, and allocating the bandwidth resources of the communication link for the information units according to the task requirement importance; wherein, the size of the allocated bandwidth resources is positively correlated with the task requirement importance; selecting a method of image compression or outputting semantic tags to process the information units according to the allocated bandwidth resources; integrating the processed generated data into the data to be transmitted and inputting it into the communication link to achieve the feedback of semantic information.

[0074] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the task feed-forward image key semantic communication method provided in the above embodiments. The method includes: parsing and processing task feed-forward information to determine a target task to be executed; the content of the target task includes at least one task requirement information and a task requirement importance corresponding to the task requirement information one by one; segmenting an image into multiple information units according to the task requirement information of the target task, and allocating bandwidth resources of a communication link to the information units according to the task requirement importance; wherein the size of the allocated bandwidth resources is positively correlated with the task requirement importance; processing the information units by selecting an image compression or output semantic label method according to the allocated bandwidth resources; integrating the processed data into data to be transmitted and inputting it into the communication link to achieve the feedback of semantic information.

[0075] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0076] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0077] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A task-feedforward image key semantic communication method, characterized in that: include: Analyze and process the task feedforward information to determine the target task to be executed; the content of the target task includes at least one task requirement information and the task requirement importance corresponding to the task requirement information; The image is divided into multiple information units according to the task requirement information of the target task, and the bandwidth resources of the communication link are allocated to the information units according to the importance of the task requirement; wherein the size of the allocated bandwidth resources is positively correlated with the importance of the task requirement; According to the allocated bandwidth resources, the information unit is processed by selecting image compression or outputting semantic labels; Integrate the processed data into data to be transmitted and input it into the communication link to realize the return of semantic information; Among them, according to the importance of task requirements, bandwidth resources of communication links are allocated to information units, including: Based on the bandwidth resources of the communication link and the importance of the task requirements, bandwidth resources are allocated to the information units according to the preset priority; The preset priorities are arranged from front to back in the following order: key semantic information unit, general semantic information unit, redundant semantic information unit; According to the allocated bandwidth resources, the information unit is processed by selecting image compression or outputting semantic labels, including: In the case where the information unit is a key semantic information unit, compressing the key semantic information unit based on an image compression method to obtain compressed image data; and When the information unit is a general semantic information unit, the semantic tag of the general semantic information unit is used as the object to be transmitted.

2. The task feedforward image key semantic communication method according to claim 1, characterized in that: Analyze and process the task feedforward information to determine the target task to be executed, including: Receive task feedforward information pre-set according to requirements; When the demand is to call a task, the target task is called from the task knowledge base according to the task feedforward information; When the requirement is to modify the content of a task, the content of the task is modified according to the task feedforward information, and the task after the modification is used as the target task, and the task knowledge base is updated; When the demand is to add a new task, the new task is added according to the task feedforward information, and the new task is used as the target task, and the task knowledge base is updated.

3. The task feedforward image key semantic communication method according to claim 1, characterized in that: The image is segmented into multiple information units according to the task requirements of the target task, including: According to the task requirement information, the image is segmented using the semantic segmentation algorithm to obtain multiple information units; The information unit corresponding to the task requirement importance greater than the first preset threshold is taken as the key semantic information unit, the information unit corresponding to the task requirement importance less than or equal to the first preset threshold and greater than the second preset threshold is taken as the general semantic information unit, the information unit whose task requirement importance is less than or equal to the second preset threshold is taken as the redundant semantic information unit, and the remaining parts constitute a background information unit.

4. The task feedforward image key semantic communication method according to claim 1, characterized in that: After completing the return of semantic information, it also includes: From the preset domain knowledge base, the image data model corresponding to the semantic label is determined for image reconstruction.

5. The task-feedforward image key semantic communication method according to claim 1, characterized in that: The algorithm used for image compression is the JEPEG2000 image compression algorithm.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the task feed-forward image key semantic communication method as claimed in any one of claims 1 to 5 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the task feed-forward image key semantic communication method as claimed in any one of claims 1 to 5 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the task feed-forward image key semantic communication method as claimed in any one of claims 1 to 5 are implemented.

Images